You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 2de159719b added an option to set 'obey nofollow' for links with rel="nofollow" ... attribute in the <a> tag for each crawl. This introduces a lot of changes because it extends the usage of the AnchorURL Object type which now also has a different toString method that the underlying DigestURL.toString. It is therefore not advised to use .toString at all for urls, just just toNormalform(false) instead.		11 years ago
..
content	…
importer	…
language	…
parser	added an option to set 'obey nofollow' for links with rel="nofollow"	11 years ago
AbstractParser.java	…
Condenser.java	…
Document.java	added an option to set 'obey nofollow' for links with rel="nofollow"	11 years ago
ImageParser.java	…
LargeNumberCache.java	…
LibraryProvider.java	…
Parser.java	…
Phrase.java	…
SentenceReader.java	…
SnippetExtractor.java	…
TextParser.java	…
WordTokenizer.java	…