You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen e4cbe9232d fixed a crawler bug where a double-occurring url was not re-crawled ... because the double-check error was written to the error-db and never deleted. No the error-db is cleared on every start and these double-messages are not written to the error-db any more.		12 years ago
..
data	fixed a crawler bug where a double-occurring url was not re-crawled	12 years ago
retrieval	Redesign of crawler identification and robots steering. A non-p2p user	12 years ago
robots	Redesign of crawler identification and robots steering. A non-p2p user	12 years ago
Balancer.java	Redesign of crawler identification and robots steering. A non-p2p user	12 years ago
CrawlQueue.java	Redesign of crawler identification and robots steering. A non-p2p user	12 years ago
CrawlStacker.java	fixed a crawler bug where a double-occurring url was not re-crawled	12 years ago
CrawlSwitchboard.java	Redesign of crawler identification and robots steering. A non-p2p user	12 years ago
HarvestProcess.java	fix for wrong display of error urls in HostBrowser	12 years ago