You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen eca9380e3d bugfix for crawler double-check: if an url is redirected, the ... redirect-target was not double-checked. This is now done by replacing the redirect-URL on the crawl queue again (where it is double-checked)		11 years ago
..
data	fix for subpath crawl filter	11 years ago
retrieval	bugfix for crawler double-check: if an url is redirected, the	11 years ago
robots	do not store failure of loading of robots.txt into the index as a fail	11 years ago
Balancer.java	- added a new Crawler Balancer: HostBalancer and HostQueues:	11 years ago
CrawlStacker.java	bugfix for crawler double-check: if an url is redirected, the	11 years ago
CrawlSwitchboard.java	typo	11 years ago
HarvestProcess.java	fix for wrong display of error urls in HostBrowser	12 years ago
HostBalancer.java	npe fix in crawler	11 years ago
HostQueue.java	fix for malformed hostpath names in crawl balancer	11 years ago
LegacyBalancer.java	special strategy for balancer: do not remove targets with zero wait time	11 years ago