adf3fa493d
Added comments about crawl profiles recrawl cycles
7 years ago
3638e16c2e
More comprehensive log on rejected recrawls caused by date constraint
7 years ago
d47afe6fab
Use a constant for crawler reject reason prefix with specific processing
7 years ago
4e03335625
Added more details to the recrawl job report
7 years ago
d95d393a0d
Add a query link to local Solr to browse selected recrawl candidates
7 years ago
59f7763af6
Display recrawl job report also when job is actively running
7 years ago
6425963cee
Fixed internal tables exact value match iterator
7 years ago
0c9e0b3566
Record recrawl calls to make them schedulable
7 years ago
433e241e4f
Added a report info box about eventual last terminated recrawl job
...
For easier monitoring of recrawls.
7 years ago
b2af25b14f
Added a stop condition to the Recrawl busy thread
7 years ago
421728d25a
Made possible to customize selection query before launching a recrawl
7 years ago
fab6e54fec
Enforced controls (HTTP method, token) on ReIndex and ReCrawl operations
7 years ago
36e9b1c5b3
Fixed SegmentTest test case time dependant occasional failures
...
As highlighted by latest automated Travis builds.
7 years ago
8a4ea1c11e
Added UI switch to control content domain constraint per search request
7 years ago
36a45b3905
Added UI setting for strictness of content-type checking on media search
7 years ago
cedb53be4e
upd to commons-io-2.6
7 years ago
f8071ac8ae
Make TokenizedStringNavigator (used for keyword search facet) active
...
check case insensitive.
As keywords are compared lower case, make sure user input keyword:Key
or keyword:key will be shown as active in facet entry key.
7 years ago
270b77074e
upd to httpclient-4.5.4 and httpmime-4.5.4
7 years ago
6db7f5525b
upd to icu4j-60.2
7 years ago
e6907fdab3
Added optional search parameter/setting to control content domain filter
...
Thus allowing to choose at configuration or per search request, whether
extending or not results beyond strict content domain filter (image,
video, audio or application).
Related graphical controls to be added to user interface.
7 years ago
f52217c939
Enable full size images preview for users with extended search rights
7 years ago
d42c1773c8
Added UI setting for optional encryption with https on p2p searches
7 years ago
09c4ee56a7
Added optional https support for remote crawl and profile operations
7 years ago
5db1c9155a
Do locale independant case conversion on hosts, schemes, and file exts.
...
Required for proper operation when the default system locale is Turkish,
as dottless and dotted i characters have specific case conversion rules
in this language.
7 years ago
1c4803e40a
Enable optional https support for /yacy/transferURL API calls.
...
Also updated some Javadoc and consistently use Switchboard instance as a
constructor parameter where relevant.
7 years ago
79a2ba306a
Updated links to Java Regular Expressions documentation to version 8
7 years ago
c94bc82f6a
upd to commons-compress-1.15
7 years ago
c6e1befbca
Restored peer URL host name stripping removed from previous commit.
...
Still useful for peers with IPv6 addresses.
7 years ago
17e004599d
Started implementing optional https preference for protocol operations
...
Introduced through the new configurable setting
network.unit.protocol.https.preferred, defaulting to false for now.
Let choose to prefer using https when available on remote peers to
perform YaCy protocol operations including notably hello or transferRWI.
Not yet implemented for every YaCy protocol operations.
7 years ago
2bc61f5657
Merge pull request #149 from Scre13/bugfix_default_settings
...
Fixed loading default thread load setting in Performance Settings of Queues and Processes.
7 years ago
bb3d3fe074
fixed default loading default settings; load was populated with wrong value
7 years ago
20bba135fe
Show hide or show public surftip button depending on current config status,
...
to show the button to switch the status (hiding button of current status)
7 years ago
b907819cb4
Merge branch 'master' of https://github.com/yacy/yacy_search_server.git
7 years ago
25573bd5ab
added a crawl filter based on <div> tag class names
...
When a crawl is started, a new field to exclude content from scraping is
available. The field can be identified with the class name of div tags.
All text contained in such a div tag where the configured class name(s)
match are not indexed, while the remaining page is indexed.
7 years ago
640fed2a9c
Removed Java 1.8 no more necessary version checking (fixes issue #147 )
...
Java 1.8 is by the way now a prerequisite to run from latest sources.
7 years ago
d95b288f19
Removed use of deprecated Jetty IPAccessHandler for client filtering.
...
Upgraded to InetAccessHandler.
Added InetPathAccessHandler extension to InetAccessHandler to maintain
path patterns capability previously available in IPAccessHandler but
lost in InetAccessHandler.
Filtering on IPv6 addresses is now supported.
Support for deprecated pattern formats such as "192.168." and
"192.168.1.1/path" has been removed, but startup automated migration
should convert such patterns eventually present in serverClient.
7 years ago
cc7a93e6b6
remove deprecated jetty continuation class from urlproxyservlet
...
(was a long time carry over, while not supporting async requests)
7 years ago
607b39b427
Merge branch 'master' of https://github.com/yacy/yacy_search_server.git
...
Conflicts:
htroot/yacysearchitem.java
7 years ago
4355de0f3c
(more!) evaluation of XRealIP from nginx reverse proxy
7 years ago
e5b4799838
upd to Jetty-9.4.8.v20171121
7 years ago
f9cba827c0
Made "tld:" modifier case insensitive and IDN complient.
...
Thus allowing typing internationalized top-level domains with non ASCII
characters as tld: modifier.
7 years ago
a4494d6e01
Improved support for internationalized domain names on "site:" modifier
...
Allow typing directly internationalized domain names including non ASCII
characters in the search field.
Search is done using the ASCII Compatible Encoding (ACE) representation.
7 years ago
d07006bac4
Do locale independant case conversion on "filetype:" query modifier.
7 years ago
8fbf25d1ed
Made "site:" query modifier case insensitive.
7 years ago
867388e05b
Refactored 'site:' query modifier parsing into a dedicated function.
7 years ago
c5c3cc1274
Use HTTP Post operation for resetting memory monitoring state.
...
Fixes issue #145
Also added textual hint on the button, and display it only when it makes
sense, that is to say when the memory state is 'exhausted'.
7 years ago
0704b1d644
upd to httpcore-4.4.8
7 years ago
bfe753acea
Merge pull request #144 from him2him2/_fic_HTTPS
...
Update HTTP -> HTTPS in README.md
7 years ago
c9d80b5b77
Prefer fine URL match over approximate URL mask regex on final filtering
...
Also prevent adding a redundant and CPU costly Solr url mask filter
query when possible
7 years ago
0a120787e3
Improved accuracy of URLs search filters : protocol, tld, host, file ext
7 years ago
luccioman