You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 5e31bad711 - the webgraph shall store all links which appear on a web page and not ... all unique links! This made it necessary, that a large portion of the parser and link processing classes must be adopted to carry a different type of link collection which carry a property attribute which are attached to web anchors. - introduction of a new URL class, AnchorURL - the other url classes, DigestURI and MultiProtocolURI had been renamed and refactored to fit into a new document package schema, document.id - cleanup of net.yacy.cora.document package and refactoring		11 years ago
..
RobotsTxt.java	- the webgraph shall store all links which appear on a web page and not	11 years ago
RobotsTxtEntry.java	- the webgraph shall store all links which appear on a web page and not	11 years ago
RobotsTxtParser.java	- the webgraph shall store all links which appear on a web page and not	11 years ago