You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/de/anomic/crawler
f1ori ee3cef91e8
* fix filesize in ftp crawls
14 years ago
..
retrieval * fix filesize in ftp crawls 14 years ago
Balancer.java fixed crawl queue delete function 14 years ago
CrawlProfile.java custom + generic skins: 15 years ago
CrawlQueues.java * fix system update if urls are in blacklist (for example for very general blacklists like *.de) 14 years ago
CrawlStacker.java - fixed numerous bugs 14 years ago
CrawlSwitchboard.java fixed a number of small bugs: 15 years ago
ImporterException.java added final where possible 17 years ago
Latency.java preparations to move the HTCache into cora: 15 years ago
NoticedURL.java enhanced crawler: 14 years ago
RSSLoader.java * fix system update if urls are in blacklist (for example for very general blacklists like *.de) 14 years ago
ResourceObserver.java change in handling of the all-visible home path for storage in YaCy: 15 years ago
ResultImages.java *) cleaning up the code a little bit 14 years ago
ResultURLs.java - added a tag cloud to search results (using the topics) 15 years ago
RobotsEntry.java - replaced pdfbox and fontbox version 1.1.0 with 1.2.1 15 years ago
RobotsTxt.java - moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed 15 years ago
SitemapImporter.java enhanced crawler: 14 years ago
ZURL.java * fix system update if urls are in blacklist (for example for very general blacklists like *.de) 14 years ago
robotsParser.java added a sitemap entry parser and loader for sitemaps 14 years ago