You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

luccioman aa9ddf3c23 Added control over Robots.txt active threads maximum number. ... When starting a crawl from a file containing thousands of links, configuration setting "crawler.MaxActiveThreads" is effective to prevent saturating the system with too many outgoing HTTP connections threads launched by the crawler. But robots.txt was not affected by this setting and was indefinitely increasing the number of concurrently loading threads until most ot the connections timed out. To improve performance control, added a pool of threads for Robots.txt, consistently used in its ensureExist() and massCrawlCheck() methods. The Robots.txt threads pool max size can now be configured in the /PerformanceQueus_p.html page, or with the new "robots.txt.MaxActiveThreads" setting, initialized with the same default value as the crawler.		8 years ago
..
java/net/yacy	Added control over Robots.txt active threads maximum number.	8 years ago
net/yacy/data	Updated license header file name	8 years ago
parsertest	update to metadata-extractor-2.7.0.jar	10 years ago
viewImageTest	Added links to more image test suites.	9 years ago
words	- moved test data from /bin to /test/words	15 years ago
ImageViewerPerfTest.java	Fixed license headers on files created to improve favicon management.	8 years ago
ImageViewerTest.java	Fixed license headers on files created to improve favicon management.	8 years ago
yacysearchitemTest.java	Fixed maven compilation error	8 years ago