You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Michael Peter Christen 0833937c1c better balancing and duetime-cumputation also for no-delay intranet ... hosts		12 years ago
..
RobotsTxt.java	- added concurrency for robots.txt loading	12 years ago
RobotsTxtEntry.java	better balancing and duetime-cumputation also for no-delay intranet	12 years ago
RobotsTxtParser.java	replaced the custom robots.txt loader by the standard http loader	12 years ago