You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/net/yacy/crawler/data
reger fb1fcc2b03
handle noarchive tag, skip writing page to cache
11 years ago
..
Cache.java handle noarchive tag, skip writing page to cache 11 years ago
CrawlProfile.java fix for subpath crawl filter 11 years ago
CrawlQueues.java added an option to set 'obey nofollow' for links with rel="nofollow" 11 years ago
Latency.java - added a new Crawler Balancer: HostBalancer and HostQueues: 11 years ago
NoticedURL.java made number of open files in crawler configurable and increased default 11 years ago
ResultImages.java fix for image alt attachment to AnchorURLs in html parser. 11 years ago
ResultURLs.java migrated the index export methods from the old metadata to solr. Now 12 years ago