Commit Graph

7 Commits (0f5fe8cc530a0c86d86456bf39604ba798b3aaba)

Author SHA1 Message Date
orbiter a6719dfd2b - refactoring of robots parser
17 years ago
orbiter 474e29ce4a added options to configure the 'corporate identity'-icons, the home page link and the greeting line from
17 years ago
orbiter 474659a71f - modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order
17 years ago
orbiter b928ae492a some code-cleanup and possible speed enhancements in different core methods
17 years ago
danielr 7feae906aa - organize imports
17 years ago
orbiter dd75b3cabc - patch for bad profiles
17 years ago
orbiter 1689030ee8 refactoring: moved all crawler classes into their own package
17 years ago