You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 25573bd5ab added a crawl filter based on <div> tag class names ... When a crawl is started, a new field to exclude content from scraping is available. The field can be identified with the class name of div tags. All text contained in such a div tag where the configured class name(s) match are not indexed, while the remaining page is indexed.		7 years ago
..
data	added a crawl filter based on <div> tag class names	7 years ago
retrieval	added a crawl filter based on <div> tag class names	7 years ago
robots	…
Balancer.java	…
CrawlStacker.java	…
CrawlStarterFromScraper.java	Updated a license header typo.	7 years ago
CrawlSwitchboard.java	added a crawl filter based on <div> tag class names	7 years ago
FileCrawlStarterTask.java	fix typo	7 years ago
HarvestProcess.java	…
HostBalancer.java	…
HostQueue.java	…
IllegalCrawlProfileException.java	…
LegacyBalancer.java	…
RecrawlBusyThread.java	Customized Threads with generic name for easier monitoring.	7 years ago