f32aa9a49c
prevent merge of blobs that can't be handled in memory
12 years ago
bbd242afb4
fix for a NPE
12 years ago
8d944f6517
nowrap from gaston in forum
...
http://forum.yacy-websuche.de/viewtopic.php?p=26815#p26815
12 years ago
24d9db1613
snippet retrieval loading processes may use a smaller minimum load time
...
value than crawling processes. This speeds up the search result
preparation dramatically.
12 years ago
ef488a15f7
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
1687737771
Abstraction of HandleMap and HandleSet
12 years ago
76b037a20a
check content domain fix:
...
search image/media should not show pages containing image/media
search text should show all/text but image/media
12 years ago
9cd409682f
close augmented stream if filled from cache to get its content
...
use augmented stream if proxyAugmentation is set only
12 years ago
e432bb9cd9
better calculation of possible saving in HeapReader index data structure
12 years ago
9549984c65
documentation/comments
12 years ago
beb6425f0c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
83c93e9209
no translation of queue-links
12 years ago
3bcd9d622b
cleaned up classes and methods which are either superfluous at this time
...
or will be superfluous or subject of complete redesign after the
migration to solr. Removing these things now will make the transition to
solr more simple.
12 years ago
6f1ddb2519
Moved solr index-add method to the same method where the YaCy index is
...
written. Also done some code-cleanup.
12 years ago
315d83cfa0
cleanup
12 years ago
1f41d9c6f5
bugfix for a NPE
12 years ago
76202f068e
extended abstraction of local and remote solr index using one front-end
...
for index administration and querying.
12 years ago
d3f243e2e1
fixed node type calculation for principal peers
12 years ago
7ec7341f60
added user-authentication protection to solr search (same as implemented
...
for yacysearch)
12 years ago
e2a97ef8f6
better explain how to access the embedded solr
12 years ago
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
12 years ago
cba4ab862e
fix for http://bugs.yacy.net/view.php?id=202
12 years ago
b76836db7b
Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
12 years ago
36c9875b6e
removed localized number formatting from num-results_totalcount response (this is only used in xml and json where localized format is not valid)
12 years ago
0640a6f7e6
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
69e743d9e3
- more abstraction for the RWI index as preparation for solr integration
...
- added options in search index to switch parts of the index on or off
12 years ago
6cc5d1094e
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
05a3ffd03a
patches to ensure that solr connectors are active ony if they have a
...
solr object assigned and vice versa
13 years ago
5a3c829872
embedded solr is only initiated if it is activated with
...
IndexFederated_p.html
13 years ago
161005ceaa
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bf4968d748
source change in classpath
13 years ago
3a350a2f83
partial html fix for
...
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4454
13 years ago
49ee31f837
added classpath for htroot/solr
13 years ago
97b7bcf2a6
added a solr search index
...
- by default, a (empty) solr storage instance is created at
SEGMENTS/solr_36
- the index is written if in /IndexFederated_p.html the flag "embedded
solr search index" is switched on
- a standard solr query interface is available now with a new servlet at
http://127.0.0.1:8090/solr/select
To test this, do the following:
- switch to webportal mode
- switch on the feature as described
- do a crawl. this fills the solr index. The normal YaCy search will NOT
work now!
- do a solr query, like:
http://127.0.0.1:8090/solr/select?q= *:*
http://127.0.0.1:8090/solr/select?q=text_t:Help
play with different search fields as you can see in
/IndexFederated_p.html
You can use the standard solr query attributes as described in
http://wiki.apache.org/solr/SearchHandler
13 years ago
f0a079ac9f
allow larger log entries
13 years ago
9b48c9fe2e
removed a crawler overhead (terminated loop which searches greatest
...
stack that has zero-waiting urls). This should cause a slightly faster
crawl for crawl stacks with many different domains in the crawl queue.
13 years ago
784a4abb18
enhancement in internal data organization which should generate less
...
synchronizations in database access
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
c00a3cf74d
less usage of generic logger to avoid logger generation overhead
13 years ago
a196f24f60
prevent enqueueing of non-loggeable logging entries
13 years ago
482afed07c
reduced logging overhead (a bit)
13 years ago
e76159040b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bbfa497a3c
replaced more size() > 0 by !isEmpty()
13 years ago
58e7d1952f
reduction of logging to prevent too much IO caused be logging
13 years ago
83da68c4c1
fixed a memory leak inside the logger which appeared if the log was
...
writter faster that the logger is able to print this out to its out
stream. A very large collection of unwritten log outputs had been seen
during strong crawling. The new ArrayBlockingQueue is limited to prevent
this case.
13 years ago
e3aa05b9dd
added creation of subpath pattern when crawl start is 'from file'
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
28b30231c3
fix for url matcher of multiple amp& in an url, see:
...
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=4439&p=26650#p26650
13 years ago
aef9dd0350
- removed cleaning of blacklist cache on startup
...
- added cleaning of blacklist cache if cache is modified in interface
- extended cache saving to all cache types
- moved cache location to DATA/LISTS
- fixed static file path which was relative to the application path but
should be relative to data path - which is different in debian and mac
implementations
13 years ago
c7afa8bc48
using SwitchboardConstants for solr attributes
13 years ago
sixcooler