removed a crawler overhead (terminated loop which searches greatest

stack that has zero-waiting urls). This should cause a slightly faster
crawl for crawl stacks with many different domains in the crawl queue.
pull/1/head
Michael Peter Christen 13 years ago
parent 784a4abb18
commit 9b48c9fe2e

@ -458,7 +458,7 @@ public class Balancer {
byte[] besturlhash = null; byte[] besturlhash = null;
String besthost = null; String besthost = null;
Map<String, byte[]> zeroWaitingCandidates = new HashMap<String, byte[]>(); Map<String, byte[]> zeroWaitingCandidates = new HashMap<String, byte[]>();
while (i.hasNext()) { while (i.hasNext() && zeroWaitingCandidates.size() < 10) {
entry = i.next(); entry = i.next();
// clean up empty entries // clean up empty entries

Loading…
Cancel
Save