You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen d6b82840f8 added a feature to find similarities in documents. ... This uses an enhanced version of the Nutch/Solr TextProfileSignatue. As a result, a signature of the document is written to the solr search index. Additionally for each time when a signature is written, it is checked if the singature exists already in the index. If the signature does not exist, the document is marked as unique. The unique attribute can now be used to sort document lists and bring duplicates to the end of a result list. To enable this, a large portion of the search api to Solr had to be changed. This affected mainly caching of 'exists' searches to enhance the check for existing signatures and do this without actually doing a solr query. Because here the first time a long number is used as value in the Solr store, also the value naming in the YaCySchema had to be adopted and normalized. This caused that many files had to be changed.		12 years ago
..
FTPLoader.java	…
FileLoader.java	added a feature to find similarities in documents.	12 years ago
HTTPLoader.java	replaced the custom robots.txt loader by the standard http loader	12 years ago
ImporterException.java	…
RSSLoader.java	…
Request.java	replaced the custom robots.txt loader by the standard http loader	12 years ago
Response.java	added a feature to find similarities in documents.	12 years ago
SMBLoader.java	added a feature to find similarities in documents.	12 years ago
SitemapImporter.java	…
URLRewriterLibrary.java	…