7f5526e6ef
allow larger no-proxy expressions
12 years ago
1228a5798d
you can now search for '*' to get just ALL entries in the search index
...
as result list. This makes sense if you intend to search just by using
the navigation tools to cut the data set into navigation 'slices'.
12 years ago
1f33c30d7b
re-integrating useForHost method (lost sometime?) to get the noProxy
...
pattern working again. Without using this method all remote urls
including the localhost had been accessed through the configured proxy
12 years ago
f1a9c2e604
fix Servlet template on conditional file include with use of conditional template pattern in included template file (example IndexCreateQueues_p.html)
...
see bug http://bugs.yacy.net/view.php?id=215
12 years ago
a4a780b871
- fix for bad url conversion in bookmarks when using smb urls
...
- fix for localhost hosts in solr schema host handling
12 years ago
e80dfeca23
- making blacklist path part case insensitive (solving http://bugs.yacy.net/view.php?id=171 )
...
- blacklist test adding explicite response text "not blocked" if no blacklist match
12 years ago
e2d499be9e
remove NOT NEEDED reference to solr.YaCySchema from ConfigurationSet to be able to use ConfigurationSet for other conf files (than solr.keys.default.list).
12 years ago
a3cd3852ab
introduced a better place to update the lastacc time value in latency
12 years ago
864abcd33d
removed Latency update after URL selection because that causes
...
a completely wrong behaviour when cache fresh cases appear. Makes
re-crawling MUCH faster!
12 years ago
4491072256
- clear the search cache when altering the solr boosts
...
- better positions for submit buttons
12 years ago
2b7d46bc1f
using a filter query for the site parameter in GSA api
12 years ago
dd241d03bb
latency fix: only set last-visit time if access was actually by the
...
robot
12 years ago
118233a7e6
fix for bad xml in gsa result when doing a query with quotes
12 years ago
1e002ab18e
added another blacklist-cleaner into balancer
12 years ago
10527e28ae
fix for wrong display of error urls in HostBrowser
12 years ago
756772fbd3
fix for waitingtime computation for intranet configuration
12 years ago
fa27e5820f
- check blacklist (again) when taking urls from the crawl stack because
...
the blacklist may get extended during crawling
- removed debug output
12 years ago
5f5d66921e
patch for funny symbols in url paths (like tilde)
12 years ago
adfecc6ba8
more robustness during shutdown
12 years ago
d4bfe9339e
Brute-force attempt to start solr in case of a memory problem.
...
I don't actually know if this is correct. It is a desperate try to get
YaCy running on production servers which must get alive even with
strange hacks like this. This is also related to a forum posting in
http://forum.yacy-websuche.de/viewtopic.php?t=4528&p=27135#p27135
12 years ago
8aa08261a7
update to Solr Boost handling
12 years ago
908ad2f174
Added a new servlet to configure the solr ranking using field boosts
12 years ago
a598fb6227
renamed Ranking_p.html to RankingRWI_p.html
...
because there will be another Ranking servlet as well at next
12 years ago
a01e47b992
enhanced exists()-method for solr; should reduce a lot of IO during DHT
...
target selection
12 years ago
72f165d58b
added a Boost class which stores solr query boost values. The class can
...
be configured using the yacy.init file. The boost information is taken
from the configuration each time when a query to solr is done.
12 years ago
ea033f8f8e
added number of characters in url to default index to be able to use
...
this field for ranking
12 years ago
b5ee88c6af
added more logging to get info which url causes performance problems
12 years ago
1faa045dc1
fix: prevent regex pattern compile error for blacklist import for path '*' (extend it to '.*')
12 years ago
bb20691d4f
fix: respect config setting of "show Nav Top-Menu" in HostBrowser.html for public users (as hostbrowser is now available in search results)
12 years ago
6cf33f899c
prevent Solr "version conflict" on update by set Solr "_version_" field to 0 (=no version check)
12 years ago
acd98bebb7
improvements in GSA result writer
12 years ago
3de784c8dd
replaced more split and replaceAll missing pattern pre-compilation with
...
pre-compiled pattern
12 years ago
8fc3679c66
using more pre-compile pattern for split methods
12 years ago
d48e9788d2
enhanced search result processing behavior
...
- query less at one time; query more often
- in between the small queries, evaluate results
- remove fields from search results which are not needed
12 years ago
bf512e6350
Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
12 years ago
469efcdb9d
fix: display and calculate authors and namespace search navigator if configured (otherwise skip overhead)
...
(leave hosts, topics and not in ConfigPortal included filetype, protocoll navigator untouched)
12 years ago
eca68fa197
added debug code to crawler monitor
12 years ago
205f8b222b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
c54cb85422
added link to
...
http://docs.oracle.com/javase/6/docs/api/java/util/regex/Pattern.html
to the /RegexTest.html servlet
12 years ago
ee612e8b93
start the local search only if this peer is doing a remote search or
...
when it is doing a local search and the peer is old
12 years ago
d465773a37
- removed multi-add of documents (no used)
...
- inserted specialized code for size request
12 years ago
a1a4d9aa94
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java
12 years ago
b7004043ea
- added a field cache for solr queries which call only for a single
...
value
- fixed a version conflict exception within a solr add request
12 years ago
5aa5202adf
fixes for filesystem indexing
12 years ago
bf42179982
introduced more structure in HostBrowser, table view, better counting,
...
distinguishing of error cases (fail/excluded)
12 years ago
efd2c4622d
added a new fail type attribute for the index to distinguish two
...
separate fail types: network fail and forced exclusion (i.e. by robots
or forwarding rules).
12 years ago
5e182a566f
- added another enumeration method in kelondro data structure to get a
...
more random access to data for the balancer
- added random access inside the balancer
12 years ago
4eab3aae60
removed overhead by preventing generation of full search results when
...
only the url is requested
12 years ago
a114bb23bb
- using edismax in gsa interface
...
- generating less field data for gsa search results
- using a boost query in gsa interface to move double content to the end
of the result list
12 years ago
d6b82840f8
added a feature to find similarities in documents.
...
This uses an enhanced version of the Nutch/Solr TextProfileSignatue.
As a result, a signature of the document is written to the solr search
index. Additionally for each time when a signature is written, it is
checked if the singature exists already in the index. If the signature
does not exist, the document is marked as unique. The unique attribute
can now be used to sort document lists and bring duplicates to the end
of a result list.
To enable this, a large portion of the search api to Solr had to be
changed. This affected mainly caching of 'exists' searches to enhance
the check for existing signatures and do this without actually doing a
solr query.
Because here the first time a long number is used as value in the Solr
store, also the value naming in the YaCySchema had to be adopted and
normalized. This caused that many files had to be changed.
12 years ago
orbiter