You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 2de159719b added an option to set 'obey nofollow' for links with rel="nofollow" ... attribute in the <a> tag for each crawl. This introduces a lot of changes because it extends the usage of the AnchorURL Object type which now also has a different toString method that the underlying DigestURL.toString. It is therefore not advised to use .toString at all for urls, just just toNormalform(false) instead.		11 years ago
..
AlternativeDomainNames.java	…
ChunkedInputStream.java	…
ChunkedOutputStream.java	…
ContentLengthInputStream.java	…
HTTPDFileHandler.java	…
HTTPDProxyHandler.java	…
HTTPDemon.java	…
MultiOutputStream.java	…
ProxyLogFormatter.java	…
RobotsTxtConfig.java	…
TemplateEngine.java	…