You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

luccioman 47af33a04c Advanced Crawl from local file : better processing of large files. ... Applied strategy : when there is no restriction on domains or sub-path(s), stack anchor links once discovered by the content scraper instead of waiting the complete parsing of the file. This makes it possible to handle a crawling start file with thousands of links in a reasonable amount of time. Performance limitation : even if the crawl start faster with a large file, the content of the parsed file still is fully loaded in memory.		8 years ago
..
AbstractScraper.java	strong redesign of html parser: object recursion is now made using a	11 years ago
AbstractTransformer.java	skip creation of unused Bluelist contenttransformer	10 years ago
CharacterCoding.java	fix for bad URL decoding	11 years ago
ContentScraper.java	Advanced Crawl from local file : better processing of large files.	8 years ago
ContentScraperListener.java	Advanced Crawl from local file : better processing of large files.	8 years ago
ContentTransformer.java	skip creation of unused Bluelist contenttransformer	10 years ago
EmbedEntry.java	- the webgraph shall store all links which appear on a web page and not	12 years ago
Evaluation.java	added a new way of content browsing in search results:	10 years ago
IconEntry.java	Fixed license headers on files created to improve favicon management.	8 years ago
IconLinkRelations.java	Fixed license headers on files created to improve favicon management.	8 years ago
ImageEntry.java	fix for image alt attachment to AnchorURLs in html parser.	11 years ago
Scraper.java	strong redesign of html parser: object recursion is now made using a	11 years ago
ScraperInputStream.java	Refactoring : use StandardCharsets constants instead of hard-coded	9 years ago
ScraperListener.java	Advanced Crawl from local file : better processing of large files.	8 years ago
Transformer.java	strong redesign of html parser: object recursion is now made using a	11 years ago
TransformerWriter.java	add links with image extension not automatically to image links.	9 years ago