You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
yacy_search_server/source/de/anomic/crawler
orbiter 844f158686
- removed dependencies in header framework:
15 years ago
..
retrieval - removed dependencies in header framework: 15 years ago
AbstractImporter.java - cleanup, removed unused imports 15 years ago
Balancer.java redirect uncaught exceptions to logging + small other changes 15 years ago
CrawlProfile.java added the new crawl scheduling function to the crawl start menu: 15 years ago
CrawlQueues.java ... migrating to HttpComponents-Client-4.x ... 15 years ago
CrawlStacker.java fixed crawler bug caused by NPE in logging 15 years ago
CrawlSwitchboard.java added the new crawl scheduling function to the crawl start menu: 15 years ago
Importer.java refactoring: 17 years ago
ImporterException.java added final where possible 17 years ago
ImporterManager.java *) some minor changes for better code readability 15 years ago
Latency.java better (and corrected) recognition of intranet and internet-addresses. This corrects the isLocal property that is used by network definitions to restrict index ranges to local and global addresses. Address locations (intranet or internet) had been partly identified by the top level domain of the host address. Since intranet addresses can also be addressed using a host name that is in a country domain it is necessary to do a dns resolving for each check. The check is supported by a local dns cache so the intranet/internet check should not affect network traffic too much. To ensure that the cache works properly the cache class was upgraded to better concurrency data structures. 15 years ago
NoticedURL.java - better url double check in crawler 15 years ago
ResourceObserver.java allow global search if res. observer disabled index transmission 15 years ago
ResultImages.java redesign of parser interface: 15 years ago
ResultURLs.java - more abstraction (HashMap -> Map) 15 years ago
RobotsEntry.java redesign of remote proxy settings 15 years ago
RobotsTxt.java refactoring of the cora package to get subpackages for http and ftp (smb to come) 15 years ago
SitemapImporter.java applied code changes that are recommended by PMD 15 years ago
ZURL.java fixed crawler bug caused by NPE in logging 15 years ago
robotsParser.java - fixed a bug in robots.txt parser 15 years ago