You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/net/yacy/crawler
Michael Peter Christen 69f85265e1
added an option to put image links to the crawl queue and handle these
11 years ago
..
data added a feature to set a collection for a crawl result based on a 11 years ago
retrieval added a feature to set a collection for a crawl result based on a 11 years ago
robots Redesign of crawler identification and robots steering. A non-p2p user 11 years ago
Balancer.java Redesign of crawler identification and robots steering. A non-p2p user 11 years ago
CrawlQueue.java Redesign of crawler identification and robots steering. A non-p2p user 11 years ago
CrawlStacker.java added an option to put image links to the crawl queue and handle these 11 years ago
CrawlSwitchboard.java Redesign of crawler identification and robots steering. A non-p2p user 11 years ago
HarvestProcess.java fix for wrong display of error urls in HostBrowser 12 years ago