You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 659178942f - Redesigned crawler and parser to accept embedded links from the NOLOAD ... queue and not from virtual documents generated by the parser. - The parser now generates nice description texts for NOLOAD entries which shall make it possible to find media content using the search index and not using the media prefetch algorithm during search (which was costly) - Removed the media-search prefetch process from image search		13 years ago
..
retrieval	- Redesigned crawler and parser to accept embedded links from the NOLOAD	13 years ago
Balancer.java	- Redesigned crawler and parser to accept embedded links from the NOLOAD	13 years ago
Cache.java	refactoring	13 years ago
CrawlProfile.java	fixed default must-match filter for full domain crawls - the old filter	13 years ago
CrawlQueues.java	- Redesigned crawler and parser to accept embedded links from the NOLOAD	13 years ago
CrawlStacker.java	new indexing strategy: ALL links that appear anywhere are indexed, not	13 years ago
CrawlSwitchboard.java	…
ImporterException.java	…
Latency.java	refactoring: removed dependency from switchboard in Balancer/CrawlQueues	13 years ago
NoticedURL.java	refactoring: removed dependency from switchboard in Balancer/CrawlQueues	13 years ago
RSSLoader.java	…
ResourceObserver.java	more logging in resource observer	13 years ago
ResultImages.java	ConcurrentLinkedQueue has a VERY long return time on the .size() method.	13 years ago
ResultURLs.java	…
RobotsTxt.java	…
RobotsTxtEntry.java	…
RobotsTxtParser.java	…
SitemapImporter.java	…
ZURL.java	ConcurrentLinkedQueue has a VERY long return time on the .size() method.	13 years ago