You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 25573bd5ab added a crawl filter based on <div> tag class names ... When a crawl is started, a new field to exclude content from scraping is available. The field can be identified with the class name of div tags. All text contained in such a div tag where the configured class name(s) match are not indexed, while the remaining page is indexed.		7 years ago
..
Blacklist.java	Properly close file output streams even on exceptions scenarios.	8 years ago
BlacklistFile.java	applying precompiled CommonPattern.COMMA.split to all places where	10 years ago
BlacklistHelper.java	Fixed blacklist returned location URL on empty parameters	7 years ago
BlacklistHostAndPath.java	Blacklist import and update performance improvements.	8 years ago
FilterEngine.java	- the webgraph shall store all links which appear on a web page and not	12 years ago
LoaderDispatcher.java	added a crawl filter based on <div> tag class names	7 years ago
RegexHelper.java	avoid NPE in regex checker	12 years ago