You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 2de159719b added an option to set 'obey nofollow' for links with rel="nofollow" ... attribute in the <a> tag for each crawl. This introduces a lot of changes because it extends the usage of the AnchorURL Object type which now also has a different toString method that the underlying DigestURL.toString. It is therefore not advised to use .toString at all for urls, just just toNormalform(false) instead.		10 years ago
..
Blacklist.java	refactore URIMetadataNode to further unify interaction with index	11 years ago
BlacklistFile.java	Rewrote all String blacklist types to enum 'BlacklistType', closes bug	13 years ago
BlacklistHelper.java	First draft of a blacklist API.	11 years ago
FilterEngine.java	- the webgraph shall store all links which appear on a web page and not	11 years ago
LoaderDispatcher.java	added an option to set 'obey nofollow' for links with rel="nofollow"	10 years ago
RegexHelper.java	avoid NPE in regex checker	12 years ago