1db476c67e
fix for bad table iteration
10 years ago
bc275dca07
added network history graph image /NetworkHistory.png which can show
...
many different statistics about the history of the peer.
10 years ago
3ac31614a3
added option to reverse-sort YaCy tables (internal API change only)
11 years ago
ea6c9e9b07
reduce mem buffer overhead for gap files during r/w
...
(they are typically small compared to idx allowing to use smaller buffersize -> set to 16k records)
11 years ago
501d55cd35
removed superfluous assert
11 years ago
97983ba89f
fixed generics warnings for generic array instantiation that appeared
...
after migration to Java 7
11 years ago
a3542f29b4
npe fix
11 years ago
c48d2a2a02
npe fix
11 years ago
5b83887da8
npe fix
11 years ago
8b44fcf0f4
added missing @Override annotation
11 years ago
fdaeac374a
- enhanced postprocessing speed and memory footprint (by using HashMaps
...
instead of TreeMaps)
- enhanced memory footprint of database indexes (by introduction of
optimize calls)
- optimize calls shrink the amount of used memory for index sets if they
are not changed afterwards any more
11 years ago
25a6c05008
experimental removal of synchronization. This should work for all cases
...
where the size() and isEmpty() method is used only for statistics, which
happens at many locations in YaCy. If these methods are used for
structual reasons (like accessing the last element in an array) then it
may fail or cause other problems. As far as visible, this is not the
case.
11 years ago
5695280edd
removed superfluous synchronization
11 years ago
c3dcbdc8d5
try to recover from an OOM during citation index reading and fail-over
...
to second solr core in case of unrecoverable OOM.
11 years ago
7b69c438f7
more methods for the table class
11 years ago
5e31bad711
- the webgraph shall store all links which appear on a web page and not
...
all unique links! This made it necessary, that a large portion of the
parser and link processing classes must be adopted to carry a different
type of link collection which carry a property attribute which are
attached to web anchors.
- introduction of a new URL class, AnchorURL
- the other url classes, DigestURI and MultiProtocolURI had been renamed
and refactored to fit into a new document package schema, document.id
- cleanup of net.yacy.cora.document package and refactoring
12 years ago
47b1c81d08
- refactoring
...
- generalized writing of url attributes to solr documents
- added more url attributes to error documents
12 years ago
13433d41a1
Log this exception better
...
Conflicts:
source/net/yacy/kelondro/blob/Tables.java
12 years ago
aeac2fb763
replaced more containsKey() -> get() usages by a simple get(), followed
...
by a test for NULL. This should increase the application speed and
reduces the lookup time for the affected methods by 50%
12 years ago
841a28ae76
Added 'final' for all exception blocks as this helps the Java compiler
...
to optimize memory usage
Conflicts:
source/net/yacy/search/Switchboard.java
12 years ago
5878c1d599
- refactoring of log to ConcurrentLog:
...
jdk-based logger tend to block
at java.util.logging.Logger.log(Logger.java:476) in concurrent
environments. This makes logging a main performance issue. To overcome
this problem, this is a add-on to jdk logging to put log entries on a
concurrent message queue and log the messages one by one using a
separate process.
- FTPClient uses the concurrent logging instead of the log4j logger
12 years ago
14186e815e
npe fix
12 years ago
e20450e798
patch in HTCache and CitationIndex loading in case that a file is
...
broken: do not crash; instead ignore the file and delete it.
12 years ago
a1c989002b
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4652
...
generate dht data even if dht receive and dht transmission is switched
off
12 years ago
e1bfe9d07a
- reduction of the concurrently running processes to make YaCy more
...
adjusted to smaller and 1-core devices.
- the workflow processor now starts no process at all. these are started
as soon as parser/condenser/indexing queues are filled.
- better abstraction
12 years ago
089dee1770
- generalized SchemaConfiguration into super-class Configuration and
...
adopted other classes which used the configuration-only access for that
class
- removed many warnings
- adjusted logging
12 years ago
712cc37c40
if maxFileSize < 0 then the file size limit is without limit.
12 years ago
354f0d9acd
moved static method from ClusteredScoreMap to MapDataMining because it
...
was not used in the ClusteredScoreMap class but only in MapDataMining
12 years ago
276dd6452b
removed warnings
12 years ago
2f536cb54d
code cleanup: removed unised methods and made more methods and objects
...
private
12 years ago
116f429e35
fix for java.lang.RuntimeException: TableColumnIndex not available...
13 years ago
8219a445f3
refactoring
13 years ago
00c1c777fa
refactoring
13 years ago
563d584420
removed more dependencies in cora from kelondro
13 years ago
e65cecc419
- updated lucene libraries to 3.6.1
...
- added lucene-grouping which enables faceted search; try this:
http://localhost:8090/solr/select?q=*:*&start=0&rows=3&facet=true&facet.field=host_s
13 years ago
4d29f59a27
removed warnings
13 years ago
d31a632951
- added dmoz RDF dump importer
...
- added indexing to Tables columns to support larger bookmark
collections
- added RDF output (HTTP) for public bookmarks at /YMarks.rdf
- YMarkRDF also provides a Jena RDF Model as "internal" API
- various other changes/fixes for YMarks (mainly backend)
13 years ago
2094df2e4e
- correct length computation for BStringObject (bugfix suggested by
...
apfelmaennchen)
- using ASCII for string conversion for Strings generated from Integer
13 years ago
94a334f128
another fix to the Solr metadata reading process and to the shutdown
...
process
13 years ago
f32aa9a49c
prevent merge of blobs that can't be handled in memory
13 years ago
1687737771
Abstraction of HandleMap and HandleSet
13 years ago
e432bb9cd9
better calculation of possible saving in HeapReader index data structure
13 years ago
9549984c65
documentation/comments
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
482afed07c
reduced logging overhead (a bit)
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
b0c408788b
made class methods static where possible
13 years ago
0301aba1e9
removed unused method parameters
13 years ago
ea10766bfd
cleaned unnecessary nested code
13 years ago
8a82609360
- smaller caches to save memory
...
- close cloneable iterators to free memory
13 years ago
Michael Peter Christen