Michael Peter Christen 2de159719b added an option to set 'obey nofollow' for links with rel="nofollow" ... attribute in the <a> tag for each crawl. This introduces a lot of changes because it extends the usage of the AnchorURL Object type which now also has a different toString method that the underlying DigestURL.toString. It is therefore not advised to use .toString at all for urls, just just toNormalform(false) instead.		11 years ago
..
cora	…
crawler	…
document	…
kelondro/io	…
peers/operation	…
search/snippet	added an option to set 'obey nofollow' for links with rel="nofollow"	11 years ago