You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 788288eb9e added the generation of 50 (!!) new solr field in the core 'webgraph'. ... The default schema uses only some of them and the resting search index has now the following properties: - webgraph size will have about 40 times as much entries as default index - the complete index size will increase and may be about the double size of current amount As testing showed, not much indexing performance is lost. The default index will be smaller (moved fields out of it); thus searching can be faster. The new index will cause that some old parts in YaCy can be removed, i.e. specialized webgraph data and the noload crawler. The new index will make it possible to: - search within link texts of linked but not indexed documents (about 20 times of document index in size!!) - get a very detailed link graph - enhance ranking using a complete link graph To get the full access to the new index, the API to solr has now two access points: one with attribute core=collection1 for the default search index and core=webgraph to the new webgraph search index. This is also avaiable for p2p operation but client access is not yet implemented.		12 years ago
..
Blacklist.java	clear some more caches if running out of memory	12 years ago
BlacklistFile.java	Rewrote all String blacklist types to enum 'BlacklistType', closes bug	13 years ago
FilterEngine.java	- replaced all length() == 0 and size() == 0 with isEmpty()	13 years ago
LoaderDispatcher.java	added the generation of 50 (!!) new solr field in the core 'webgraph'.	12 years ago
RegexHelper.java	*) Invalid crawl profiles (containing invalid mustmatch/mustnotmatch filters) will be moved from active crawls to invalid crawls (new file: DATA/INDEX/freeworld/QUEUES/crawlProfilesInvalid.heap). This file can not be edited yet, but it shoudl be easy to extend the CrawlProfileEditor accordingly.	14 years ago