Commit Graph

3554 Commits (06ed4ef7b3bf986ab973e858677cd01f6f042810)

Author SHA1 Message Date
borg-0300 06ed4ef7b3 * better picture handling
16 years ago
orbiter 5a634cab23 removed generation of anchor link sets in document types that describe container formats.
16 years ago
low012 f1244264b8 *) hopefully fixed bug reported in http://forum.yacy-websuche.de/viewtopic.php?t=2057
16 years ago
orbiter 2e3186189b fix for mediawikiIndex surrogate producer + added concurrency
16 years ago
apfelmaennchen 6f5ea7b1a8 small fix for previous post
16 years ago
apfelmaennchen 138a0747e3 added serverObjects.putJSON as JSON has very particulare encoding requirements
16 years ago
orbiter d977dd9a96 fix for surrogate loader
16 years ago
orbiter 9cb68353da fix for bug in ProfilingGraph for ppm >> 10000 ppm (!)
16 years ago
orbiter 9e4db75aac reduced internal logging and reduced memory that internal logging can use
16 years ago
orbiter c10c257255 attempt to fix a deadlock situation where the IODispatcher did not work.
16 years ago
orbiter 09987e93fd fixed some more bad handling of byte[]
16 years ago
orbiter 1bcc1450cb more explaining error message in case of IOExceptions during html parsing
16 years ago
orbiter fe51f4d668 less synchronization may help to prevent deadlocks
16 years ago
orbiter 58802e4201 added missing success test in storeDocumentIndex,
16 years ago
orbiter 171e62bee5 addition to the fix from last commit (which did not work)
16 years ago
orbiter 059949a0d1 tried to fix problem with snippet fetch for second search page when verify=false
16 years ago
lotus b08991e278 moved some constants, rename of Tray class
16 years ago
orbiter 138422990a - removed useCell option: the indexCell data structure is now the default index structure; old collection data is still migrated
16 years ago
orbiter 1b9e532c87 some concurrency for wikipedia dump reader
16 years ago
lotus 25d2160288 small fix
16 years ago
orbiter 16baa7ad24 To translate a mediawiki dump into the YaCy surrogate format do the following:
16 years ago
orbiter 0b2c98edc9 some more work on the wikipedia-dump exporter (not finished yet)
16 years ago
orbiter 5195c94838 two patches for performance enhancements of the index handover process from documents to the index cache:
16 years ago
orbiter 9416f5c26f more speed test cases: kelondro provides map functions that are more than 20% faster than standard java classes and use less than halve of the memory of java classes:
16 years ago
orbiter b53790abb1 more performance hacks: 10% more speed for Base64.compare() which is really often used in YaCy code
16 years ago
orbiter 8ffb9889e1 some fixes and performance hacks
16 years ago
orbiter dfb96ecb72 more fixes
16 years ago
orbiter 1b8d346b4c fixes in connection with transiton to byte[] hashes
16 years ago
f1ori 0b0a46d35a * fix transferRWI as suggested by celle (thanks!)
16 years ago
orbiter 996572de95 quickfix
16 years ago
orbiter 380ed2dac0 performance and debugging additions
16 years ago
lotus 635b0a9da7 code-split
16 years ago
orbiter fa3adbbfc6 added domain checks to surrogate reader and RWI transfer receiver to prevent spaming using surrogates
16 years ago
f1ori 76af84d732 * add custom comparator to ScoreCluster for byte[]
16 years ago
lotus ab0030d7a7 allow dht-out for remote-crawl processing peers on default settings
16 years ago
low012 d1116c049f *) added new method "contains()" to Blacklist interface
16 years ago
f1ori 08445e42f0 * don't throw exception, in case of bad charset in http-header
16 years ago
f1ori 2f860a2564 * convert byte[] hashes to string for log output
16 years ago
f1ori d93a2a6552 * ignore whitespaces so you can copy&paste signatures better
16 years ago
orbiter fbcbcc5bdb export of yacy document objects as dublin core record in xml
16 years ago
orbiter d7cbf4cdd4 more performance hacks: less overhead in word hash computation
16 years ago
orbiter 29e96c1a60 bugfixes and performance hacks
16 years ago
orbiter 4e97a31009 corrections in dublin core syntax
16 years ago
f1ori 44daec7936 * introduce signatures to autoupdate
16 years ago
orbiter 538e375901 replaced old caching method for computed word hashes with a better method. The word hash computation is a new performance bottleneck (after the IO bottleneck was removed with the IndexCell data structure) and a better caching for word hashes was necessary.
16 years ago
orbiter 9e853e1977 partly reverting SVN 5818: identical comparator required for join operator
16 years ago
orbiter e16c25ddf7 (peak-) performance hacks
16 years ago
orbiter 63cd152969 fixes
16 years ago
orbiter 7dfe7e7cc6 fixed some problems with surrogate reader. This is now ready for testing.
16 years ago
orbiter 3a1364ed5c removed example lines from SurrogateReader sources; added additional example file
16 years ago