Commit Graph

14 Commits (3b6229d2f31a229ea737ddc97e44ce1ebe653604)

Author SHA1 Message Date
danielr 17b7845eb5 * refactoring
17 years ago
danielr 3bb870bfcd added final where possible
17 years ago
orbiter c3d461d191 - removed superfluous copyright statement
17 years ago
orbiter 3ca98fee42 removed superfluous copyright statement
17 years ago
orbiter 05c26d58d9 fixed missing remove operation in balancer
17 years ago
orbiter 606b323a2d fixed bug that appeared when a new crawl ist started
17 years ago
orbiter 28d5703f8a - fixed a bug in Robots.txt loader which could have caused that robots.txt files had been loaded from the same domain more than once
17 years ago
orbiter a6719dfd2b - refactoring of robots parser
17 years ago
orbiter 474e29ce4a added options to configure the 'corporate identity'-icons, the home page link and the greeting line from
17 years ago
orbiter 474659a71f - modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order
17 years ago
orbiter b928ae492a some code-cleanup and possible speed enhancements in different core methods
17 years ago
danielr 7feae906aa - organize imports
17 years ago
orbiter dd75b3cabc - patch for bad profiles
17 years ago
orbiter 1689030ee8 refactoring: moved all crawler classes into their own package
17 years ago