You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 2de159719b added an option to set 'obey nofollow' for links with rel="nofollow" ... attribute in the <a> tag for each crawl. This introduces a lot of changes because it extends the usage of the AnchorURL Object type which now also has a different toString method that the underlying DigestURL.toString. It is therefore not advised to use .toString at all for urls, just just toNormalform(false) instead.		11 years ago
..
AlternativeDomainNames.java	…
ChunkedInputStream.java	…
ChunkedOutputStream.java	- the webgraph shall store all links which appear on a web page and not	12 years ago
ContentLengthInputStream.java	…
HTTPDFileHandler.java	removed unused variables and warnings	11 years ago
HTTPDProxyHandler.java	added an option to set 'obey nofollow' for links with rel="nofollow"	11 years ago
HTTPDemon.java	- remove empty http0_9 status text array	11 years ago
MultiOutputStream.java	…
ProxyLogFormatter.java	…
RobotsTxtConfig.java	…
TemplateEngine.java	fix for parsing of numeric value in case that boolean values are given	11 years ago