16e9d4d1dd
added a restart hint
12 years ago
d725782440
turned severe message to warning message about network failure events
12 years ago
b3a54d5b1c
fix for wrong class name in log
12 years ago
2d36a7eaf5
- do not create a new query for all remote peers
...
- no document search this time
- adjusted banner and network to not show 'WORDS' but DHT Chunks. This
is to avoid confusion for robinson peers which do not create Word
Entries
12 years ago
4af0839be2
use appropriate ranking for each search situation:
...
- when using the /date modifier, a date ranking profile is used
- when using a site: modifier, a ranking profile supporting longer urls
is used
12 years ago
b8ed66a55d
added all clickdepth computations for source and target paths in
...
webstructure core
12 years ago
6300730d7f
refactoring of clickdepth computation as preparation for clickdepth
...
computation of webgraph links
12 years ago
2080fc7406
removed unused tag fields
12 years ago
7804c12976
fix error msg in ConfigHeuristics_p
12 years ago
230a12bfe2
adjust Opensearch discover function to new webgraph Solr schema
12 years ago
6b13dd0d3d
added clickdepth field writing for webgraph core (unfinished)
12 years ago
47114910d5
fix for possible memory leaks
12 years ago
addba047e2
changes in ranking computation
...
- an existing ranking servlet for solr was extended. It is now possible
to set boost values for fields, boost functions and boost queries.
- The ranking can have different instances, but currently only the first
one is used
- added an abstraction layer for fields which can be used for search and
those fields can be edited in the solr ranking configruation
- the ranking value from solr within the field score is used to combine
remote search requests, which all are created using the same locally
defined boost values
- reduced the number of fields which are used for search (makes it
faster)
- replaced some text fields by string fields (makes indexing faster)
- removed classes which had no use
- made a large number of experiments for a better ranking and created a
temporary setting which prefers hits inside titles
- adjusted also the RWI-based ranking computation to 'prefer title'
- made special cases like for portal search where no post-processing and
post-ranking is wanted: this keeps the original ranking order as done by
Solr
- fixed many bugs with old settings for ranking
12 years ago
38f46eb33d
set RootNodeFlag only if EmbeddedSolr is connected (as RootNodes may receive direct Solr queries)
12 years ago
2962f2b9e9
Merge branch 'master' of git://gitorious.org/yacy/rc1.git
12 years ago
ab74d559fb
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
4490133909
removed target_tag_s (superfluous)
12 years ago
cd197bb555
fix for NPE if surrogates do not exist
12 years ago
6ae30f9d0f
replace the terminateOldSessions - return immediate time from fixed 3 sec to requested minage parameter
12 years ago
68e739a90b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
3d9ce9cd04
- added more selection criteria for network seed list
...
- enhanced up script
12 years ago
168e8d9b4d
added/fixed missing DOCTYPE line (submitted by Thomas)
12 years ago
252bb51f98
fix for wrong mime type in noload crawler
12 years ago
25300913fa
fixes to search debugging after testing with the different search
...
debugging options
12 years ago
81380ae5c8
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
c2fde018b5
concurrent snippet fetching from solr results which do not have snippets
12 years ago
b1140e3d82
added debug switches for detailed search testing
12 years ago
cdbfddf091
added filter queries for better image, audio and video results
12 years ago
587ef83eab
added missing cleanup statements for short memory cases during search
12 years ago
2562f052b9
do not put the fulltext field text_t into the search cache because it is
...
not used there and uses a lot of memory
12 years ago
2b6c79d347
in method exists() also use the new caching-stacks for
...
documents/metadata
12 years ago
ae734b3f8d
enhanced the search result processing
...
- no waiting time at the end
- switched on 'classic' snippet production and verification (again)
12 years ago
2d472a39f4
DHT-transferred metadata and crawl receipts now also use the delayed
...
search cache to prevent that too much IO load is on the peer during
search.
12 years ago
0d7b4bc891
better protection against OOM during search flush and fixed missing
...
result push
12 years ago
221ed7d764
- enhanced concurrency during search without IO blocking
...
- introduced a second queue to flush remote search results (now: old
metadata structure from DHT peers)
- fixed result counters
12 years ago
2714b59f38
*) For some reason this seems to fix a ClassCastException on my system
...
(OpenJDK).
12 years ago
3b1d9dc884
made index storage from DHT search result concurrently. This prevents
...
blocking by high CPU usage during search. Also: removed query from Solr
for DHT search results; results are taken from the pending queue.
12 years ago
f13c0b2abd
fix for search
12 years ago
0f7ea7ad9f
- enhanced solr.add procedure for mass adds
...
- removed unused solr access classes
- made snippet generation for documents aus YaCy RWI/DHT concurrent (as
it was before the search process removation)
- reduced the number of remote results in settings file because the
processing of such mass documents add is too CPU-intensive (in Solr)
12 years ago
7ff10bdb1b
fix of page navigation for formatted totalcount numbers
12 years ago
08d28eed1a
Übersetzung des Domain Navigators als Anbieter Navigator; ist als Nutzen
...
besser erklärbar
12 years ago
f327ffedb4
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
9c09fd7d0b
better/less requests to local solr; the request is made in chunks which
...
are exactly at only that size which is needed to present the current
search result page. This will also cause that next solr request are made
automatically during switching to next pages.
12 years ago
840fa22135
disabled clickdepth computation during craling since that is repeated
...
during clean-up phase.
12 years ago
a734fbc4a5
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
d74472f562
corrected result counter
12 years ago
2555542f7a
removed the dns prefetch because that was not soo useful
12 years ago
aa3c26c62e
added recrawl/reload to CrawlStartSite for a timeout of 3 days
12 years ago
c1b7e61882
added option to create empty vocabularies
12 years ago
e0edad689d
fix link to IndexSchema_p.html
12 years ago
Michael Peter Christen