7c1feefb28
introduced a default 10 second time-out in rwi normalization time
...
uring search process to prevent endless deadlocks after a very long
running search
13 years ago
8d997d55b6
better logging
13 years ago
43c2c6e588
better logging
13 years ago
c15fcde1c8
add-on to latest commit
13 years ago
cf47d94888
performance hack to parse numbers inside of substrings without actually
...
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
13 years ago
7e0ddbd275
added a "fromCache" flag in Response object to omit one cache.has()
...
check during snippet generation. This should cause less blockings
13 years ago
c6a09eab0b
synchronization needed
13 years ago
6696cb1313
bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer
...
SeedDB.lookupByName searche for lowercase peerNames, while MapColumnIndex.getIndex uses peername as is in the keyset.
Changed the index init to insert lowercase peer names as key
13 years ago
f294f2e295
bugfix to http://bugs.yacy.net/view.php?id=181
...
tried to make a bit less 'noise' to dns server
also included: less processes in snippet fetch to reduce load during
search on small computers
13 years ago
acf8d521a2
fix for http://bugs.yacy.net/view.php?id=126
13 years ago
fa735f4f04
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
3e1bc9477f
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
6f8a2fef1f
small speed enhancement using a column factory
13 years ago
d10627d591
More sync in close() methods
...
Conflicts:
source/net/yacy/kelondro/logging/GuiHandler.java
source/net/yacy/kelondro/workflow/InstantBusyThread.java
13 years ago
fbb946f913
Made a method static (Eclipse suggested it), removed unused import, pk=null check does now output a warning in logfile
13 years ago
89142d1e8d
removed (not all) warnings
13 years ago
15db703808
added missing serialization to remove all warnings
13 years ago
1795a7325b
made HandleSet serializable
13 years ago
a093ccf5eb
Now used synchronization in all close() methods to make sure all objects
...
are 'closed' in an ordered way
Conflicts:
source/de/anomic/http/server/ChunkedInputStream.java
source/de/anomic/http/server/ChunkedOutputStream.java
source/de/anomic/http/server/ContentLengthInputStream.java
source/net/yacy/cora/protocol/Domains.java
source/net/yacy/cora/services/federated/solr/SolrShardingConnector.java
source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java
source/net/yacy/document/content/dao/PhpBB3Dao.java
source/net/yacy/document/parser/html/AbstractTransformer.java
source/net/yacy/kelondro/blob/BEncodedHeap.java
source/net/yacy/kelondro/blob/HeapReader.java
source/net/yacy/kelondro/index/RAMIndexCluster.java
source/net/yacy/kelondro/io/ByteCountInputStream.java
source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java
source/net/yacy/kelondro/table/SQLTable.java
13 years ago
0cf3d36eae
more tolerance in case of corrupted file
13 years ago
34f4225d7e
less 'wellformed' calls without asserts
13 years ago
ba6aaabc51
refactoring + parser bugfixes
13 years ago
e32055aa15
added stub classes for
...
- a new database for url reference data ('seen links')
- a new database extending the references to the full url metadata
attributes set which shall replace the old metadata database if it is
finished
- migration help classes stub to use old and new metadata databases
simultanously
13 years ago
2fc8ecee36
ConcurrentLinkedQueue has a VERY long return time on the .size() method.
...
See
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html
and the following test programm:
public class QueueLengthTimeTest {
public static long countTest(Queue<Integer> q, int c) {
long t = System.currentTimeMillis();
for (int i = 0; i < c; i++) {
q.add(q.size());
}
return System.currentTimeMillis() - t;
}
public static void main(String[] args) {
int c = 1;
for (int i = 0; i < 100; i++) {
Runtime.getRuntime().gc();
long t1 = countTest(new ArrayBlockingQueue<Integer>(c), c);
Runtime.getRuntime().gc();
long t2 = countTest(new LinkedBlockingQueue<Integer>(), c);
Runtime.getRuntime().gc();
long t3 = countTest(new ConcurrentLinkedQueue<Integer>(),
c);
System.out.println("count = " + c + ": ArrayBlockingQueue =
" + t1 + ", LinkedBlockingQueue = " + t2 + ", ConcurrentLinkedQueue = "
+ t3);
c = c * 2;
}
}
}
13 years ago
213c8d97f2
use less proccesses in process pool
13 years ago
c639248c23
protection against strange answers from remote peers during search
13 years ago
1cd711d005
added classes for citation references (for new citation ranking)
13 years ago
e0f1e7d904
added new citation reference data structure that shall be used for a
...
citation ranking
13 years ago
e18a4f6b74
more tolerant merge iterator
13 years ago
7e4e3fe5b6
free some memory after parsing html
13 years ago
4540174fe0
memory hacks
13 years ago
b4409cc803
small redesign of blob column index and usage
13 years ago
d5c1f2746e
performance hack
13 years ago
803963aebd
performance hack: better space grow in CharBuffer (speeds up html
...
parser)
13 years ago
e2f8f263e8
changed storage of search words: keep order
13 years ago
0b67a0a5d8
added a column index for tables in blob files. This is heavily used
...
during receiving of DHT submissions and when answering remote search
requests. Both events together may have caused IO-deadlocking and this
commit shall fix that.
13 years ago
e3bb73c3d6
serialized some database access methods
13 years ago
2ea585d616
fix for host navigator
13 years ago
ef78f22ee1
performance hack
13 years ago
a02fdf8625
better error messages
13 years ago
c6ba44468e
timeout = 5000 instead 3000
13 years ago
8776b84c10
*) small fix to make password change function of reconfigureYACY.sh work
...
again
13 years ago
4901cee3cc
suppress auto-tagged subject entries when sending out or receiving
...
metadata from other peers
13 years ago
985b78cf89
correct 'avaiable()' to use max of young / eden
13 years ago
4da8746275
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
c9aaa9e00a
respect non-reserved Memory in GenerationMemoryStrategy
...
and enable it again
13 years ago
37f2d1b3e9
replaced Thread initialization with ExecutorService pool for delete
...
method. This is much faster and produces less blocking when using the
Compressor class which is used by the HTCache. I.e. picture search is
much faster now.
13 years ago
0d6176804b
emergency disabling of GenerationMemoryStrategy because of non-working
...
available-method
13 years ago
87f0210480
enriched log output to find NPE in HeapReader
13 years ago
254adea51c
small fixes
13 years ago
Michael Peter Christen