Commit Graph

225 Commits (4eddabee4221f0d8deec55a157b520e1e1c140ce)

Author SHA1 Message Date
Michael Peter Christen b0c408788b made class methods static where possible
13 years ago
Michael Peter Christen 0301aba1e9 removed unused method parameters
13 years ago
Michael Peter Christen ea10766bfd cleaned unnecessary nested code
13 years ago
Michael Peter Christen 8a82609360 - smaller caches to save memory
13 years ago
Michael Peter Christen 0c345d1559 giving threads name so its easier to see whats happening during
13 years ago
Michael Peter Christen de3ef8ad73 removed unimportant warnings
13 years ago
Michael Peter Christen bef823c247 close the reader if finished
13 years ago
cominch 9cbfc1a1c0 augmentedProxy, which forwards every proxy request to a
13 years ago
Michael Peter Christen ba10caf89a lazy initialization of database tables
13 years ago
Michael Peter Christen 701b9a28a0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 10c9c17d51 fixed handlemap spread factor and null iterator handling
13 years ago
Michael Peter Christen b0095c8d3c flush the compressor cache when a cleanup is done
13 years ago
Michael Peter Christen 96e9d77270 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 3dd8376825 added automatic cleaning of cache if metadata and file database size is
13 years ago
Michael Peter Christen 6bb07afcc3 accept also files with other file prefix; used to read 'foreign' cache
13 years ago
Michael Peter Christen 461a0ce052 removed warnings
13 years ago
reger 6696cb1313 bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer
13 years ago
Roland 'Quix0r' Haeder a093ccf5eb Now used synchronization in all close() methods to make sure all objects
13 years ago
Michael Peter Christen ba6aaabc51 refactoring + parser bugfixes
13 years ago
Michael Peter Christen 213c8d97f2 use less proccesses in process pool
13 years ago
Michael Peter Christen b4409cc803 small redesign of blob column index and usage
13 years ago
Michael Peter Christen 0b67a0a5d8 added a column index for tables in blob files. This is heavily used
13 years ago
Michael Peter Christen 37f2d1b3e9 replaced Thread initialization with ExecutorService pool for delete
13 years ago
Michael Peter Christen 0d6176804b emergency disabling of GenerationMemoryStrategy because of non-working
13 years ago
Michael Peter Christen 87f0210480 enriched log output to find NPE in HeapReader
13 years ago
Michael Christen 216a287a85 Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r
13 years ago
Michael Christen 575dbbaa93 enhancements in Blob retrieval: try to use less CPU resources by testing
13 years ago
Roland 'Quix0r' Haeder fa08ed5ae5 Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check
13 years ago
Michael Christen c04bfaa51b refactoring
13 years ago
Michael Peter Christen 613ab6a69d added BEncodedHeapBag and BEncodedHeapShard which are storage container
13 years ago
Michael Christen 1f4afb4dc0 performance hacks
13 years ago
Michael Christen e9dc99fe15 added rules to set specific RWIs as private RWIs which are not
13 years ago
Michael Christen 078fcde0dd bad initialization
13 years ago
Michael Christen 044f83feed added some pauses into the search process which shall produce
13 years ago
Michael Christen d35bdc2df6 removed npe
13 years ago
Michael Christen 9cd469e6d6 added pull request from als plus an NPE fix
13 years ago
orbiter 83335c3b09 fix for http://bugs.yacy.net/view.php?id=78
13 years ago
orbiter 35a9e8f307 - fixed network graphic
13 years ago
Al Sutton 8993cac4d8 Initial performance improvements
13 years ago
orbiter 5a55397f99 some last-minute performance hacks
13 years ago
orbiter 05f34a3fa7 added a full, complete, database insert, update and delete API for the tables.
13 years ago
orbiter 3a15e58e28 - increased stability when opening the robots table
13 years ago
orbiter 57d5529a01 performance hacks
13 years ago
orbiter 2842ce30d6 added synchronization in ReferenceContainer and logging for shrinking
13 years ago
sixcooler ecb4986b38 refactored stuff from last commit to ReferenceContainer
13 years ago
sixcooler f7c4abfdd7 limit references per blob & term to the 100.000 youngest
13 years ago
orbiter 51cf697acd refactoring: moved all score-related classes to new ranking package
13 years ago
sixcooler 5cd07d7f84 early freeing resources on deleting index reference if search-verification fails (aka Switchboard.cleanupJob)
13 years ago
orbiter 1912d0cccc changed handling of RowSet element retrieval: until today all elements had been copied from the underlying byte[] arrays into a new Entry object that again had a copy of a portion of that byte[] in its own bye[]. There was an option to just refer to the underlying byte[] with a pointer but that was almost never used. This commit now changes an interface to the Row class where it is now necessary to tell if a copy is always required. Fortunately the copy is only needed in very rare cases. That means that this change should cause much less memory allocation; it is expected that this happens especially during search situations.
14 years ago
orbiter 0c1b29f3c9 - applied many small performance hacks
14 years ago
orbiter fe0c08455b more concurrency (enhancement) hacks
14 years ago
orbiter 4bea3f9714 hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources:
14 years ago
orbiter e28bd0d038 fix for some possible causes of memory leaks
14 years ago
orbiter 10e2f588f8 - enhanced ybr ranking computation
14 years ago
orbiter 3ed4a09368 small features, some bug fixes and performance hacks
14 years ago
orbiter b45701d20f this is a re-implementation of the YaCy Block Rank feature
14 years ago
orbiter dc54915df4 fix for very bad compare
14 years ago
orbiter b77b8cac0c - enhanced html parser: recognized much more details in the content
14 years ago
orbiter 17530ca7b5 fix for bug http://bugs.yacy.net/view.php?id=10
14 years ago
orbiter b1a8d0c020 enhancements to web cache and less strict caching rules
14 years ago
orbiter a35d513bd8 fix for not-deleted .gap and .idx files
14 years ago
orbiter 8f11d3a5bb redesigned the ScoreMap classes:
14 years ago
orbiter 30aed9824a moved getBytes() to UTF8.getBytes() to use a default String encoding
14 years ago
orbiter e1b6916423 always try to guess the size of a StringBuilder to prevent too many memory re-allocations
14 years ago
low012 3b40b98256 *) set SVN properties
14 years ago
orbiter cb1f49d0f2 replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'.
14 years ago
orbiter 8d14916c74 more patches for a better out-of-memory management
14 years ago
orbiter 993b9bc1a8 memory/performance hacks, less synchronization, better concurrency
14 years ago
orbiter 42d90664f3 - fixed a memory leak in the httpc.post method (no finish)
14 years ago
orbiter b1781d7aae some more performance hacks
14 years ago
orbiter b2f147d28e performance hack: excluded map encoding in many cases from synchronization block, especially when doing an iteration
14 years ago
orbiter 5e186e0122 continuing the fight against deadlocks during time formatting: better caching.
14 years ago
orbiter 19b2a50578 - enhanced date formatter cache
14 years ago
orbiter 48a61c39a3 speed hacks in BLOB ArrayStack:
14 years ago
orbiter 804ae2275b - do not delete idx and gap files if the heap is not modified
14 years ago
orbiter 5e45ded8e2 - removed locks from WordReference
14 years ago
orbiter d84b4a072e healing for some OOM problems
14 years ago
orbiter 6083f2f171 fix for (false) oom
14 years ago
orbiter fe93caac5a added flags and administration options to show advanced search and to show search result attributes (for each search result)
14 years ago
orbiter eb12e15738 moved all Double values to Float values because of
14 years ago
orbiter 090c73e32e catch a OOM in HeapReader iteration
14 years ago
orbiter 10ae8d961b - cora package has now no dependencies to other yacy packages and becomes a 'base' package (refactoring)
14 years ago
orbiter b2ed4cfaf8 more small bugfixes and light refactoring
14 years ago
low012 9b3fae9496 *) cleaning up the code a little bit
14 years ago
sixcooler b87bf88ac8 using less memory on merging and rewriting blobs
14 years ago
orbiter 4c50d3428e smaller file size for array stacks to support smaller deletion sizes
14 years ago
orbiter becc463d8a enhanced did-you-mean
14 years ago
orbiter 445619f3ec added a submenu ConfigHTCache_p.html to set the size of the HTCache separately from the proxy configuration.
14 years ago
orbiter ca738ac924 - added a tag cloud to search results (using the topics)
14 years ago
orbiter e4d561971e added more score cluster options and made score cluster usage more transparent
14 years ago
orbiter 7cd9d9d22a - enhanced DidYouMean computation using a faster count on index entries; this causes that results can be ranked better
14 years ago
orbiter 09c208a3ab patch for corrupted database files (just work on and forget key)
14 years ago
orbiter 8da4eb5de6 addition to patch in SVN 7111
14 years ago
orbiter 37baa8bae3 - fixes for concurrency exceptions and failed database integrity verification
14 years ago
orbiter 83ac07874f - corrected return value of put() methods (not used anywhere, so it did not harm before)
14 years ago
orbiter 7dbc357593 patch to identify corrupted database files
14 years ago
orbiter 5fe828fa06 - replaced pdfbox and fontbox version 1.1.0 with 1.2.1
14 years ago
orbiter 24502fe3de performance hacks
14 years ago
orbiter d865ef77a8 removed re-read of index in case of a bad index. This may not solve the problem but it applies a 100% CPU problem on the peer. I'm afraid bad index files must be abandoned, and cannot be fixed this way.
14 years ago
orbiter b2c9db48ea Performance enhancement
14 years ago