yacy_search_server

History

Michael Peter Christen 788288eb9e added the generation of 50 (!!) new solr field in the core 'webgraph'. The default schema uses only some of them and the resting search index has now the following properties: - webgraph size will have about 40 times as much entries as default index - the complete index size will increase and may be about the double size of current amount As testing showed, not much indexing performance is lost. The default index will be smaller (moved fields out of it); thus searching can be faster. The new index will cause that some old parts in YaCy can be removed, i.e. specialized webgraph data and the noload crawler. The new index will make it possible to: - search within link texts of linked but not indexed documents (about 20 times of document index in size!!) - get a very detailed link graph - enhance ranking using a complete link graph To get the full access to the new index, the API to solr has now two access points: one with attribute core=collection1 for the default search index and core=webgraph to the new webgraph search index. This is also avaiable for p2p operation but client access is not yet implemented.		12 years ago
..
AbstractScraper.java	redesign of the QueryParams class: introduced QueryGoal which holds the	12 years ago
AbstractTransformer.java	Now used synchronization in all close() methods to make sure all objects	13 years ago
CharacterCoding.java	…
ContentScraper.java	added the generation of 50 (!!) new solr field in the core 'webgraph'.	12 years ago
ContentTransformer.java	- replaced all length() == 0 and size() == 0 with isEmpty()	13 years ago
EmbedEntry.java	added the generation of 50 (!!) new solr field in the core 'webgraph'.	12 years ago
Evaluation.java	refactoring + parser bugfixes	13 years ago
ImageEntry.java	added the generation of 50 (!!) new solr field in the core 'webgraph'.	12 years ago
Scraper.java	free some memory after parsing html	13 years ago
ScraperInputStream.java	added the generation of 50 (!!) new solr field in the core 'webgraph'.	12 years ago
ScraperListener.java	…
Transformer.java	…
TransformerWriter.java	fix in html parser and bookmark generation	12 years ago