You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

luccioman 6cec2cdcb5 Use unredirected robots.txt URL when adding an entry to the table.		7 years ago
..
data	Added HT Cache basic statistics (hit rate)	8 years ago
retrieval	Support parsing gzip files from servers with redundant headers.	7 years ago
robots	Use unredirected robots.txt URL when adding an entry to the table.	7 years ago
Balancer.java	Fixed display of crawler pending URLs counts in HostBrowser.html page.	8 years ago
CrawlStacker.java	Factored code re-implementing DigestURL.hosthash() method.	8 years ago
CrawlStarterFromSraper.java	Advanced Crawl from local file : better processing of large files.	8 years ago
CrawlSwitchboard.java	remove wrong log line in CrawlSwitchboard	8 years ago
FileCrawlStarterTask.java	Crawl from local file : faster task end when manually terminating crawl.	8 years ago
HarvestProcess.java	…
HostBalancer.java	Fixed display of crawler pending URLs counts in HostBrowser.html page.	8 years ago
HostQueue.java	to prevent crawler to concurrently access and alter same crawl queue	8 years ago
IllegalCrawlProfileException.java	Crawl from local file : faster task end when manually terminating crawl.	8 years ago
LegacyBalancer.java	use supplied url port to get robots.txt in crawlers hostqueue	9 years ago
RecrawlBusyThread.java	init Recrawl job chunk size to max crawl loader during job start, to use some system preferences	9 years ago