9 Commits (d2d5be032dc255b24b74e9ccf0a328d375c62e33)

Author	SHA1	Message	Date
Michael Peter Christen	a3cd3852ab	introduced a better place to update the lastacc time value in latency	12 years ago
Michael Peter Christen	864abcd33d	removed Latency update after URL selection because that causes ... a completely wrong behaviour when cache fresh cases appear. Makes re-crawling MUCH faster!	12 years ago
Michael Peter Christen	756772fbd3	fix for waitingtime computation for intranet configuration	12 years ago
Michael Peter Christen	0fe8be7981	enhaced data structures for balancer and latency computation which ... should produce a bit better prognosis about forced waiting times.	12 years ago
Michael Peter Christen	b2ffd49817	less latency	12 years ago
Michael Peter Christen	0833937c1c	better balancing and duetime-cumputation also for no-delay intranet ... hosts	12 years ago
Michael Peter Christen	2d9e577ad0	replaced the custom robots.txt loader by the standard http loader	12 years ago
orbiter	8952153ecf	update to Balancer algorithm: ... - create a load list from the current list of known hosts - do not create this list for each Balancer.pop access - create the list from those hosts which have a zero-waiting time - select 1/3 from that list which have the most urls waiting - get hosts from the wainting list in random order - fixes for some delta-time computations - always load all urls from hosts which have never been loaded before	12 years ago
Michael Peter Christen	00c1c777fa	refactoring	13 years ago