e432bb9cd9
better calculation of possible saving in HeapReader index data structure
12 years ago
9549984c65
documentation/comments
12 years ago
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
12 years ago
69e743d9e3
- more abstraction for the RWI index as preparation for solr integration
...
- added options in search index to switch parts of the index on or off
12 years ago
f0a079ac9f
allow larger log entries
13 years ago
784a4abb18
enhancement in internal data organization which should generate less
...
synchronizations in database access
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
a196f24f60
prevent enqueueing of non-loggeable logging entries
13 years ago
482afed07c
reduced logging overhead (a bit)
13 years ago
e76159040b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bbfa497a3c
replaced more size() > 0 by !isEmpty()
13 years ago
83da68c4c1
fixed a memory leak inside the logger which appeared if the log was
...
writter faster that the logger is able to print this out to its out
stream. A very large collection of unwritten log outputs had been seen
during strong crawling. The new ArrayBlockingQueue is limited to prevent
this case.
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
1addbc792c
use less memory for md5 cache
13 years ago
f32de94723
more logging
13 years ago
8efc1c1078
- fixed a memory leak (or bad usage) during parsing/snippet fetch
...
- more logging for errors
13 years ago
b0c408788b
made class methods static where possible
13 years ago
5bd3c90907
- removed unnecessary semicolons
...
- added default case for switch
13 years ago
132afaf687
removed unaccessible code
13 years ago
7c1ba99755
removed more unused method parameters
13 years ago
83701a1b4c
removed unused ImageReference package
13 years ago
0301aba1e9
removed unused method parameters
13 years ago
d3964253ae
- added @SuppressWarnings to unused servlet method parameters
...
- removed unnecessary casts
- removed unnecessary throw statements
13 years ago
ea10766bfd
cleaned unnecessary nested code
13 years ago
1481037820
replaced non-generic array with collection
13 years ago
613b45f604
- better data structures in secondary search
...
- fixed a big memory leak in secondary search
13 years ago
8a82609360
- smaller caches to save memory
...
- close cloneable iterators to free memory
13 years ago
ce8d4b87d9
fixes for new eclipse 'Juno' warning 'Resource leak'.
13 years ago
0c345d1559
giving threads name so its easier to see whats happening during
...
debugging and within a thread dump
13 years ago
b9d42fd9c8
using com.google.common.io.Files instead of homebrew methods
13 years ago
de3ef8ad73
removed unimportant warnings
13 years ago
9264d8b4af
removed old navigation practice using subject tags in favor of
...
triplestore-tags
13 years ago
61bb52d55c
- using http://purl.org/dc/terms/references to refer from an
...
auto-annotated document to a 'pseudo-linked' document which has an url
created with an object-prefix as defined in the vocabulary file
13 years ago
8b53771db2
changed behavior of navigation processing:
...
- vocabulary annotation is not done any more into the metadata of urldb
- vocabularies are written into the jena triplestore using a rdf
vocabulary
- vocabularies for rdf tripel must be updated; refactoring done
- with the new navigation tags in the triplestore a faster
pre-urldb-lookup is possible: navigation is processed now within the RWI
during pre-ranking retrieval
- added also a Owl vocabulary stub to add the plain-text url to the
triplestore using the owl:sameas predicate
13 years ago
bef823c247
close the reader if finished
13 years ago
9cbfc1a1c0
augmentedProxy, which forwards every proxy request to a
...
rewrite engine to customize existing webpages. originally implemented by
Florian Richter.
Conflicts:
source/de/anomic/http/server/HTTPDProxyHandler.java
13 years ago
3b992e6b00
using utf8 String compression in Webstructure database
13 years ago
2280a7b276
- changed initialization order to prefer allocation of memory for table
...
files first
- bugfixes in memory amount calculation
13 years ago
0746308bc2
only the metadata tables shall be able to use the tail cache
13 years ago
7ec9bef0c3
fix for OOM
13 years ago
41c02cb10e
- less restrictions for usage of Table RAM copy
...
- new limit to use the table copy (instead of flag): 400MB available. If
less is available, then a copy is never used. If more is available, then
it can be used if there is a remaining space of at least 200MB
- flush caches more often: flush the Digest cache
13 years ago
b8f56a9803
npe bugfix
13 years ago
ba10caf89a
lazy initialization of database tables
13 years ago
701b9a28a0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
htroot/PerformanceMemory_p.java
13 years ago
10c9c17d51
fixed handlemap spread factor and null iterator handling
13 years ago
b0095c8d3c
flush the compressor cache when a cleanup is done
13 years ago
96e9d77270
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
13 years ago
00f2df1120
a variety of possible memory leak fixes
13 years ago
3dd8376825
added automatic cleaning of cache if metadata and file database size is
...
not equal. It might happen that these data is different because one of
that caches is cleaned after a while or when it is too big. The metadata
is then not cleaned, but now wiped after a checkup process at every
application start. This should cause a bit less memory usage.
13 years ago
6bb07afcc3
accept also files with other file prefix; used to read 'foreign' cache
...
files
13 years ago
Michael Peter Christen