You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 4f83d5f18c added the new field harvestkey_s to the collection index and the ... webgraph index which is temporary filled with the crawl profile key. This is used to select a set of documents for post-processing as soon as a crawl is finished. Now the postprocessing for a specific crawl is started when that specific crawl is finished and not at the end of all post-processing steps.		12 years ago
..
data	added the new field harvestkey_s to the collection index and the	12 years ago
retrieval	- removed ZURL data structure; removed also the ZURL data file	12 years ago
robots	- the webgraph shall store all links which appear on a web page and not	12 years ago
Balancer.java	refactoring	12 years ago
CrawlStacker.java	better and consistent deletions for error urls	12 years ago
CrawlSwitchboard.java	when profiles are cleaned, there shall be first a callback showing which	12 years ago
HarvestProcess.java	fix for wrong display of error urls in HostBrowser	12 years ago
HostQueue.java	Added new data structure to be used by the balancer (not used yet).	12 years ago
HostQueues.java	Added new data structure to be used by the balancer (not used yet).	12 years ago