9bf0d7ecb9
added a new collection type 'dht' to all documents from the peer-to-peer
...
interface to distinguish rich and poor document data.
This also reverts some changes from commit
796770e070
10 years ago
3cd7deb3b8
do not flush non-errors to stdout because this is a concurrency issue.
...
the flush-call appeared very often in thread dumps with high load, so
this hopefully gives some performances
10 years ago
1a764135be
one more Thread Dump fix for new bootstrap css style
11 years ago
bb21d825f9
fix for thread dump line spacing
11 years ago
3c3cb78555
- removed a lot of garbage and bloated code from GuiHandler.
...
- transformed log lines to String before they are stored because the
storage space is about 1:250 (45kb for one line before transformation,
180 bytes afterwards)
- this saves up to 10MB RAM so we can increase the number of lines to
1000 again.
11 years ago
5afa6e3aee
Automatically flush the log cache if a short memory status is reached.
...
For the default of 200 lines this can flush about 10MB.
11 years ago
6aabc4e5c8
reduced logging line memory, 10000 lines had filled up 450MB! grrr.
...
(thank you, a bomb from the past)
11 years ago
336f86394c
replaced StringBuffer with StringBuilder
11 years ago
841a28ae76
Added 'final' for all exception blocks as this helps the Java compiler
...
to optimize memory usage
Conflicts:
source/net/yacy/search/Switchboard.java
11 years ago
5c6946dd5f
replaced usage of log4j by ConcurrentLog where possible
12 years ago
5878c1d599
- refactoring of log to ConcurrentLog:
...
jdk-based logger tend to block
at java.util.logging.Logger.log(Logger.java:476) in concurrent
environments. This makes logging a main performance issue. To overcome
this problem, this is a add-on to jdk logging to put log entries on a
concurrent message queue and log the messages one by one using a
separate process.
- FTPClient uses the concurrent logging instead of the log4j logger
12 years ago
35fa718b77
testing to use solr for portalsearch caused some bugfixing but no full
...
success: try to comment out the solr search request in
yacy-portalsearch.js
12 years ago
1baf498d59
- show more lines in online log
...
- reverse order is default now
12 years ago
2f536cb54d
code cleanup: removed unised methods and made more methods and objects
...
private
12 years ago
00c1c777fa
refactoring
12 years ago
f0a079ac9f
allow larger log entries
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
a196f24f60
prevent enqueueing of non-loggeable logging entries
13 years ago
482afed07c
reduced logging overhead (a bit)
13 years ago
83da68c4c1
fixed a memory leak inside the logger which appeared if the log was
...
writter faster that the logger is able to print this out to its out
stream. A very large collection of unwritten log outputs had been seen
during strong crawling. The new ArrayBlockingQueue is limited to prevent
this case.
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
f32de94723
more logging
13 years ago
b0c408788b
made class methods static where possible
13 years ago
e0d8643226
- performance hacks
...
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
13 years ago
8d997d55b6
better logging
13 years ago
43c2c6e588
better logging
13 years ago
3e1bc9477f
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
d10627d591
More sync in close() methods
...
Conflicts:
source/net/yacy/kelondro/logging/GuiHandler.java
source/net/yacy/kelondro/workflow/InstantBusyThread.java
13 years ago
89142d1e8d
removed (not all) warnings
13 years ago
a093ccf5eb
Now used synchronization in all close() methods to make sure all objects
...
are 'closed' in an ordered way
Conflicts:
source/de/anomic/http/server/ChunkedInputStream.java
source/de/anomic/http/server/ChunkedOutputStream.java
source/de/anomic/http/server/ContentLengthInputStream.java
source/net/yacy/cora/protocol/Domains.java
source/net/yacy/cora/services/federated/solr/SolrShardingConnector.java
source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java
source/net/yacy/document/content/dao/PhpBB3Dao.java
source/net/yacy/document/parser/html/AbstractTransformer.java
source/net/yacy/kelondro/blob/BEncodedHeap.java
source/net/yacy/kelondro/blob/HeapReader.java
source/net/yacy/kelondro/index/RAMIndexCluster.java
source/net/yacy/kelondro/io/ByteCountInputStream.java
source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java
source/net/yacy/kelondro/table/SQLTable.java
13 years ago
87f0210480
enriched log output to find NPE in HeapReader
13 years ago
5a55397f99
some last-minute performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
7598a9e26b
fix for thread dump
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7992 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
8eef8722d1
update to ThreadDump analysis: freerunner and thread state recognition
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7990 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
57d5529a01
performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7977 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
31283ecd07
- added a search option to filter only specific network protocols. i.e. get only results from ftp servers. Just add '/ftp' to your search.
...
for example search for "passwd /ftp". This can also be done with /http /https and /smb
- fixed some search throttling processes that should protect your peer against search DoS or strong search load
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7794 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
7db208c992
performance hacks: more pre-allocated StringBuilder
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7790 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
d27a0a67ff
fix in log initialization according to hint from Dominic
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7728 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
2b5f8585bf
performance hack for Balancer and ip address parsing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7608 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
b1d133b69f
another anhancement to the ThreadDump function: better multiple dumps and filtering out of not interesting dump parts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7606 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
859c99886c
fix for multiple thread dump
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7603 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
c2a968c23f
fix for bug in formatting in ThreadDump
...
and added hint for linux/Mac users that they may use the LOCKED feature using the start option -l
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7601 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
0324de1467
removed debug line
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7597 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
1aba7869bf
patch for Windows: do not use the thread lock feature from previous commit if used on Windows
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7596 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
0a11727374
added new feature for Thread dump:
...
"THREADS WITH STATES: LOCK FOR OTHERS"
will show only such threads that lock other threads. This is the 'opposite part' of the blocked threads.
Because that this uses a thread dump that is produced with a kill -3 on the PID of the process and such thread dumps are written by the Java core outside of System.out and Sytem.err it is necessary to read the dump from a log in the file system. Such a log is only written if YaCy is started with startYACY.sh on a linux system. That means:
this feature is only available on linux and Mac OS X if YaCy is started with ./startYACY.sh -l
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7595 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
e6c3507b17
disabled some of the previous changes (did not work in openjdk)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7591 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
f9e5c21083
update to thread dump logs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7590 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
96bb33ed9b
added default size to StringBuffer in logger (and it is not possible to replace the StringBuffer with a StringBuilder...)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7573 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
e1b6916423
always try to guess the size of a StringBuilder to prevent too many memory re-allocations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7572 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
bf27a72d53
*) set SVN properties
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7564 6c8d7289-2bf4-0310-a012-ef5d649a1542
14 years ago
Michael Peter Christen