Commit Graph

8 Commits (f8f05ecba7885fe304448d5d22e7b9ab1aa1b6e5)

Author SHA1 Message Date
Michael Peter Christen 2d9e577ad0 replaced the custom robots.txt loader by the standard http loader
12 years ago
Michael Peter Christen a33e2742cb - removed unnecessary synchronized and deadlock in crawler
12 years ago
orbiter 8952153ecf update to Balancer algorithm:
12 years ago
Michael Peter Christen ae6feb5610 showing the web structure graph as animation in the crawl monitor
12 years ago
Michael Peter Christen 21fe8339b4 - enhanced generation of url objects
12 years ago
Michael Peter Christen 5f0ab25382 removed the option to prevent removal of & parts inside of the
12 years ago
Michael Peter Christen a06930662c replaced some more .getBytes() with UTF8/ASCII.getBytes()
12 years ago
Michael Peter Christen 00c1c777fa refactoring
13 years ago