Commit Graph

783 Commits (58802e4201ade767c33bb2b3769984fa74040f03)

Author SHA1 Message Date
orbiter 138422990a - removed useCell option: the indexCell data structure is now the default index structure; old collection data is still migrated
16 years ago
orbiter 16baa7ad24 To translate a mediawiki dump into the YaCy surrogate format do the following:
16 years ago
orbiter 5195c94838 two patches for performance enhancements of the index handover process from documents to the index cache:
16 years ago
orbiter 9416f5c26f more speed test cases: kelondro provides map functions that are more than 20% faster than standard java classes and use less than halve of the memory of java classes:
16 years ago
orbiter b53790abb1 more performance hacks: 10% more speed for Base64.compare() which is really often used in YaCy code
16 years ago
orbiter 8ffb9889e1 some fixes and performance hacks
16 years ago
orbiter dfb96ecb72 more fixes
16 years ago
orbiter 1b8d346b4c fixes in connection with transiton to byte[] hashes
16 years ago
orbiter 996572de95 quickfix
16 years ago
orbiter 380ed2dac0 performance and debugging additions
16 years ago
f1ori 76af84d732 * add custom comparator to ScoreCluster for byte[]
16 years ago
f1ori 2f860a2564 * convert byte[] hashes to string for log output
16 years ago
orbiter fbcbcc5bdb export of yacy document objects as dublin core record in xml
16 years ago
orbiter d7cbf4cdd4 more performance hacks: less overhead in word hash computation
16 years ago
orbiter 29e96c1a60 bugfixes and performance hacks
16 years ago
f1ori 44daec7936 * introduce signatures to autoupdate
16 years ago
orbiter 538e375901 replaced old caching method for computed word hashes with a better method. The word hash computation is a new performance bottleneck (after the IO bottleneck was removed with the IndexCell data structure) and a better caching for word hashes was necessary.
16 years ago
orbiter e16c25ddf7 (peak-) performance hacks
16 years ago
orbiter 63cd152969 fixes
16 years ago
orbiter c8624903c6 full redesign of index access data model:
16 years ago
orbiter bc80dc913a added new surrogate reader (surrogates are parsed documents on batches)
16 years ago
orbiter 8a24350036 - fix for join method with new generalized RWI data structure (caused by latest commit)
16 years ago
orbiter 89ec3acb3e - full abstraction of index content type: the kelondro full text index may now also contain indexes about other content than text, i.e. navigation indexes or reverse linking indexes.
16 years ago
orbiter 8862a2fed0 ups
16 years ago
orbiter de68948bc5 better handling of free memory computation and emrgency cache flush for index cell
16 years ago
orbiter b81c7467d8 protection against too many files in RICELL in case of massive emergency dumps caused by low memory
16 years ago
orbiter 14361f1ca4 added log message for index generation in HeapReader
16 years ago
orbiter 44e01afa5b - refactoring
16 years ago
orbiter 82fb60a720 increased memory limit for emergency cache flush
16 years ago
lotus 596e6215dc fix in case of white space in path name
16 years ago
orbiter b887f4a116 keep more free mem
16 years ago
orbiter c2359f20dd refactoring: better abstraction of reference and metadata prototypes.
16 years ago
orbiter ab656687d7 more strict BLOB initialization .. may also help to save some ram
16 years ago
orbiter f6691411b5 - migration of files from SplitTable (which are used for the URL-DB) to a different file name format.
16 years ago
orbiter f21a8c9e9c a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.
16 years ago
orbiter 7ba078daa1 - added fast site-operator
16 years ago
orbiter b4126432bc hardening of index dump write process
16 years ago
orbiter 9bfb2641db - removed deprecated threads
16 years ago
orbiter 0139988c04 - added writing of temporary file names and renaming to final file name when index dump/merge are done. Interrupted merges can be cleaned up.
16 years ago
orbiter 3621aa96ab - added a memory protection for the IndexCell migration
16 years ago
orbiter 568e8f1741 fix in unmountBLOB
16 years ago
orbiter 9da69d6b68 - better selection of files to be merged
16 years ago
orbiter d39a5b42ca more care about open file handles. Now files also close on windows and can be deleted afterwards.
16 years ago
orbiter 029495e64d fixed bug introduced in SVN 5756 in EcoTable.put()
16 years ago
orbiter 587838bd09 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5758 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter d2e2420a68 - added another file selection method for index cell merge
16 years ago
orbiter 96eaecda3e - added migration class to go from index collections to the index cell data structure.
16 years ago
orbiter 0f0b4aec75 better index cell merge logic
16 years ago
orbiter fa07234d4e fix for clear method: now deletes files
16 years ago
borg-0300 c450e3746b svn attributes added
16 years ago