You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/de/anomic/crawler
orbiter 4588b5a291
- fixed document number limitation for crawls that restrict the number of documents per domain
14 years ago
..
retrieval - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
Balancer.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
CrawlProfile.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
CrawlQueues.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
CrawlStacker.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
CrawlSwitchboard.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
ImporterException.java added final where possible 17 years ago
Latency.java preparations to move the HTCache into cora: 15 years ago
NoticedURL.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
RSSLoader.java * fix system update if urls are in blacklist (for example for very general blacklists like *.de) 14 years ago
ResourceObserver.java same units for memory observer configuration (MiB) 14 years ago
ResultImages.java *) cleaning up the code a little bit 14 years ago
ResultURLs.java - fixed document number limitation for crawls that restrict the number of documents per domain 14 years ago
RobotsEntry.java - replaced pdfbox and fontbox version 1.1.0 with 1.2.1 15 years ago
RobotsTxt.java - moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed 15 years ago
SitemapImporter.java enhanced crawler: 14 years ago
ZURL.java * fix system update if urls are in blacklist (for example for very general blacklists like *.de) 14 years ago
robotsParser.java added a sitemap entry parser and loader for sitemaps 15 years ago