Commit Graph

3654 Commits (c879783008594364cbc6647adc37cab591755bfb)

Author SHA1 Message Date
orbiter 27eb8d62cb - new development cycle
16 years ago
orbiter b7457d3807 patch for http://forum.yacy-websuche.de/viewtopic.php?p=14720#p14720
16 years ago
orbiter bffbe43e09 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14522#p14522
16 years ago
orbiter f133d6065c fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
16 years ago
lotus 82af994041 added missing loglevel
16 years ago
orbiter ad9762746d no exception in case of uniq() time-out, see also
16 years ago
orbiter 1efe686e3f fix for http://forum.yacy-websuche.de/viewtopic.php?p=13960#p13960
16 years ago
lotus 13fb84ab81 you can define your default number of search results displayed by search.items
16 years ago
orbiter f2e4d156e8 removed debug messages
16 years ago
orbiter 709bfc2cd4 added a memory check in http post protocol
16 years ago
orbiter c01d6f43e1 - fixed problem with thread dump if no arguments are given
16 years ago
orbiter a49edd9415 fix for bug in search with site: constraint
16 years ago
orbiter c1e5fad9a7 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14767#p14767
16 years ago
orbiter 8ee3a94e82 fix for non-caching of sitehash, see http://forum.yacy-websuche.de/viewtopic.php?p=14440#p14440
16 years ago
borg-0300 21930d05ed fix for [B@...
16 years ago
orbiter b6ba387e01 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14751#p14751
16 years ago
orbiter 4338dcf936 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2093&hilit=
16 years ago
lotus bad7ce9286 experimental option trayIcon.force for unsupported platforms. java 1.6 needed
16 years ago
low012 ea27853c59 *) some refactoring
16 years ago
low012 d164b42604 *) cosmetics
16 years ago
orbiter 17150b2950 fixed bug in snippet computation
16 years ago
orbiter 89aeb318d3 enhanced the wikimedia dump import process
16 years ago
orbiter 5fb77116c6 added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see
16 years ago
hermens df733af4fa Try not to loose content from ram during IndexCell.delete by moving ram.delete after the dangerous operations on the array (array.get and array.delete)
16 years ago
hermens ac72005f2f Let IndexCell.remove remove entries from the ram portion of the DB as well.
16 years ago
orbiter 8ba7ff5353 a fix and another speed enhancement for the RWI cache
16 years ago
orbiter 05f077e85f added stack trace output to solve problem in
16 years ago
orbiter 71a4cadf31 better and more performant synchronization in SimpleARC, the caching object for word hashes. Speeds up indexing.
16 years ago
orbiter e6773cbb33 better handling of RWI cache for concurrency and less overhead when writing new entries -> even more indexing speed
16 years ago
orbiter c097531e3d added a catch Exception to all thread to check if any of them silently dies without any other notification
16 years ago
orbiter 083533e5ec fix for bugs in IODispatcher
16 years ago
orbiter 21fbca0410 better scaling of HEAP dump writer for small memory configurations;
16 years ago
orbiter 6e0b57284d better care for states of the IODispatcher
16 years ago
orbiter 1db9cdd4e4 fixed bug in writing of robots.txt entries in case that host names exceeded 64 characters and some other problems
16 years ago
f1ori bde88b684a * splitt off yacyRelease from yacyVersion
16 years ago
orbiter 057ce14c8e more fixes (character encoding, parser exceptions, http client failure, blob writing)
16 years ago
orbiter d2ac0aa682 - fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling
16 years ago
lotus 1351d903a1 don't follow links like mailto:
16 years ago
orbiter e88a66bcae temporary disabling computation of all sublinks (check needed)
16 years ago
low012 ff5f82d780 *) removed description of removed commands from wikiHelp ([= =])
16 years ago
orbiter eacf95213a fix for crawling of mailto-links
16 years ago
orbiter 9c6ac43f66 fixes for wiki parser
16 years ago
orbiter 3a64c9d02f - fix for problem with concurrency when computing word hashes
16 years ago
orbiter d3f8aa5a2a set of small fixes
16 years ago
low012 78ffb61297 *) got rid of unnecessary variable which might also fix IndexOutOfBoundsException
16 years ago
orbiter d31e6f9c14 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14457#p14457
16 years ago
orbiter 8d6212233b fix for IODispatcher
16 years ago
orbiter f678472f46 fix for quote problem in json output
16 years ago
orbiter d079d6dfdb small changes in surrogate reader, wiki code and portal test
16 years ago
orbiter 07f09742bb set of small fixes and comments
16 years ago
borg-0300 06ed4ef7b3 * better picture handling
16 years ago
orbiter 5a634cab23 removed generation of anchor link sets in document types that describe container formats.
16 years ago
low012 f1244264b8 *) hopefully fixed bug reported in http://forum.yacy-websuche.de/viewtopic.php?t=2057
16 years ago
orbiter 2e3186189b fix for mediawikiIndex surrogate producer + added concurrency
16 years ago
apfelmaennchen 6f5ea7b1a8 small fix for previous post
16 years ago
apfelmaennchen 138a0747e3 added serverObjects.putJSON as JSON has very particulare encoding requirements
16 years ago
orbiter d977dd9a96 fix for surrogate loader
16 years ago
orbiter 9cb68353da fix for bug in ProfilingGraph for ppm >> 10000 ppm (!)
16 years ago
orbiter 9e4db75aac reduced internal logging and reduced memory that internal logging can use
16 years ago
orbiter c10c257255 attempt to fix a deadlock situation where the IODispatcher did not work.
16 years ago
orbiter 09987e93fd fixed some more bad handling of byte[]
16 years ago
orbiter 1bcc1450cb more explaining error message in case of IOExceptions during html parsing
16 years ago
orbiter fe51f4d668 less synchronization may help to prevent deadlocks
16 years ago
orbiter 58802e4201 added missing success test in storeDocumentIndex,
16 years ago
orbiter 171e62bee5 addition to the fix from last commit (which did not work)
16 years ago
orbiter 059949a0d1 tried to fix problem with snippet fetch for second search page when verify=false
16 years ago
lotus b08991e278 moved some constants, rename of Tray class
16 years ago
orbiter 138422990a - removed useCell option: the indexCell data structure is now the default index structure; old collection data is still migrated
16 years ago
orbiter 1b9e532c87 some concurrency for wikipedia dump reader
16 years ago
lotus 25d2160288 small fix
16 years ago
orbiter 16baa7ad24 To translate a mediawiki dump into the YaCy surrogate format do the following:
16 years ago
orbiter 0b2c98edc9 some more work on the wikipedia-dump exporter (not finished yet)
16 years ago
orbiter 5195c94838 two patches for performance enhancements of the index handover process from documents to the index cache:
16 years ago
orbiter 9416f5c26f more speed test cases: kelondro provides map functions that are more than 20% faster than standard java classes and use less than halve of the memory of java classes:
16 years ago
orbiter b53790abb1 more performance hacks: 10% more speed for Base64.compare() which is really often used in YaCy code
16 years ago
orbiter 8ffb9889e1 some fixes and performance hacks
16 years ago
orbiter dfb96ecb72 more fixes
16 years ago
orbiter 1b8d346b4c fixes in connection with transiton to byte[] hashes
16 years ago
f1ori 0b0a46d35a * fix transferRWI as suggested by celle (thanks!)
16 years ago
orbiter 996572de95 quickfix
16 years ago
orbiter 380ed2dac0 performance and debugging additions
16 years ago
lotus 635b0a9da7 code-split
16 years ago
orbiter fa3adbbfc6 added domain checks to surrogate reader and RWI transfer receiver to prevent spaming using surrogates
16 years ago
f1ori 76af84d732 * add custom comparator to ScoreCluster for byte[]
16 years ago
lotus ab0030d7a7 allow dht-out for remote-crawl processing peers on default settings
16 years ago
low012 d1116c049f *) added new method "contains()" to Blacklist interface
16 years ago
f1ori 08445e42f0 * don't throw exception, in case of bad charset in http-header
16 years ago
f1ori 2f860a2564 * convert byte[] hashes to string for log output
16 years ago
f1ori d93a2a6552 * ignore whitespaces so you can copy&paste signatures better
16 years ago
orbiter fbcbcc5bdb export of yacy document objects as dublin core record in xml
16 years ago
orbiter d7cbf4cdd4 more performance hacks: less overhead in word hash computation
16 years ago
orbiter 29e96c1a60 bugfixes and performance hacks
16 years ago
orbiter 4e97a31009 corrections in dublin core syntax
16 years ago
f1ori 44daec7936 * introduce signatures to autoupdate
16 years ago
orbiter 538e375901 replaced old caching method for computed word hashes with a better method. The word hash computation is a new performance bottleneck (after the IO bottleneck was removed with the IndexCell data structure) and a better caching for word hashes was necessary.
16 years ago
orbiter 9e853e1977 partly reverting SVN 5818: identical comparator required for join operator
16 years ago
orbiter e16c25ddf7 (peak-) performance hacks
16 years ago
orbiter 63cd152969 fixes
16 years ago
orbiter 7dfe7e7cc6 fixed some problems with surrogate reader. This is now ready for testing.
16 years ago
orbiter 3a1364ed5c removed example lines from SurrogateReader sources; added additional example file
16 years ago