boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
5828 Commits
1 Branch
20 Tags
851 MiB
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '222850414e'
${ noResults }
Commit Graph

3719 Commits (222850414e19f7e24f403ca202c7ec776bdfd48e)

Author SHA1 Message Date
orbiter c72a5cf326 added stub for PHPBB3 extraction code using direct access to mySQL
16 years ago
orbiter e735d3a69f fix for http://forum.yacy-websuche.de/viewtopic.php?p=15175#p15175
16 years ago
orbiter 63a0255166 - refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index
16 years ago
orbiter f246928c20 first attempt to add 'real' Navigation to yacy search results: host navigation
16 years ago
orbiter 54b9e99c01 - more information about peer tags
16 years ago
orbiter 26a46b5521 increased default maximum file size for database files to 2GB
16 years ago
orbiter addecdb18c simplified code, removed one unused method in all implementing classes
16 years ago
borg-0300 47fce9020c small change (Orbiter's wish)
16 years ago
borg-0300 e07b14e5d7 finally a working fix for 5960
16 years ago
borg-0300 3ebb904d2c fix for 5960, http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2119
16 years ago
lotus 734680dc70 initialize the ResourceObsever in own thread
16 years ago
orbiter e005cfea37 fix for bug in -incell option of URLAnalysis
16 years ago
orbiter a7e392f31b The collection index will not be supported any more.
16 years ago
orbiter a2f48863fc - added prototype for navigation index
16 years ago
lotus 47fd226bdb proper parsing of sentences
16 years ago
orbiter 27eb8d62cb - new development cycle
16 years ago
orbiter b7457d3807 patch for http://forum.yacy-websuche.de/viewtopic.php?p=14720#p14720
16 years ago
orbiter bffbe43e09 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14522#p14522
16 years ago
orbiter f133d6065c fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
16 years ago
lotus 82af994041 added missing loglevel
16 years ago
orbiter ad9762746d no exception in case of uniq() time-out, see also
16 years ago
orbiter 1efe686e3f fix for http://forum.yacy-websuche.de/viewtopic.php?p=13960#p13960
16 years ago
lotus 13fb84ab81 you can define your default number of search results displayed by search.items
16 years ago
orbiter f2e4d156e8 removed debug messages
16 years ago
orbiter 709bfc2cd4 added a memory check in http post protocol
16 years ago
orbiter c01d6f43e1 - fixed problem with thread dump if no arguments are given
16 years ago
orbiter a49edd9415 fix for bug in search with site: constraint
16 years ago
orbiter c1e5fad9a7 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14767#p14767
16 years ago
orbiter 8ee3a94e82 fix for non-caching of sitehash, see http://forum.yacy-websuche.de/viewtopic.php?p=14440#p14440
16 years ago
borg-0300 21930d05ed fix for [B@...
16 years ago
orbiter b6ba387e01 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14751#p14751
16 years ago
orbiter 4338dcf936 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2093&hilit=
16 years ago
lotus bad7ce9286 experimental option trayIcon.force for unsupported platforms. java 1.6 needed
16 years ago
low012 ea27853c59 *) some refactoring
16 years ago
low012 d164b42604 *) cosmetics
16 years ago
orbiter 17150b2950 fixed bug in snippet computation
16 years ago
orbiter 89aeb318d3 enhanced the wikimedia dump import process
16 years ago
orbiter 5fb77116c6 added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see
16 years ago
hermens df733af4fa Try not to loose content from ram during IndexCell.delete by moving ram.delete after the dangerous operations on the array (array.get and array.delete)
16 years ago
hermens ac72005f2f Let IndexCell.remove remove entries from the ram portion of the DB as well.
16 years ago
orbiter 8ba7ff5353 a fix and another speed enhancement for the RWI cache
16 years ago
orbiter 05f077e85f added stack trace output to solve problem in
16 years ago
orbiter 71a4cadf31 better and more performant synchronization in SimpleARC, the caching object for word hashes. Speeds up indexing.
16 years ago
orbiter e6773cbb33 better handling of RWI cache for concurrency and less overhead when writing new entries -> even more indexing speed
16 years ago
orbiter c097531e3d added a catch Exception to all thread to check if any of them silently dies without any other notification
16 years ago
orbiter 083533e5ec fix for bugs in IODispatcher
16 years ago
orbiter 21fbca0410 better scaling of HEAP dump writer for small memory configurations;
16 years ago
orbiter 6e0b57284d better care for states of the IODispatcher
16 years ago
orbiter 1db9cdd4e4 fixed bug in writing of robots.txt entries in case that host names exceeded 64 characters and some other problems
16 years ago
f1ori bde88b684a * splitt off yacyRelease from yacyVersion
16 years ago
orbiter 057ce14c8e more fixes (character encoding, parser exceptions, http client failure, blob writing)
16 years ago
orbiter d2ac0aa682 - fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling
16 years ago
lotus 1351d903a1 don't follow links like mailto:
16 years ago
orbiter e88a66bcae temporary disabling computation of all sublinks (check needed)
16 years ago
low012 ff5f82d780 *) removed description of removed commands from wikiHelp ([= =])
16 years ago
orbiter eacf95213a fix for crawling of mailto-links
16 years ago
orbiter 9c6ac43f66 fixes for wiki parser
16 years ago
orbiter 3a64c9d02f - fix for problem with concurrency when computing word hashes
16 years ago
orbiter d3f8aa5a2a set of small fixes
16 years ago
low012 78ffb61297 *) got rid of unnecessary variable which might also fix IndexOutOfBoundsException
16 years ago
orbiter d31e6f9c14 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14457#p14457
16 years ago
orbiter 8d6212233b fix for IODispatcher
16 years ago
orbiter f678472f46 fix for quote problem in json output
16 years ago
orbiter d079d6dfdb small changes in surrogate reader, wiki code and portal test
16 years ago
orbiter 07f09742bb set of small fixes and comments
16 years ago
borg-0300 06ed4ef7b3 * better picture handling
16 years ago
orbiter 5a634cab23 removed generation of anchor link sets in document types that describe container formats.
16 years ago
low012 f1244264b8 *) hopefully fixed bug reported in http://forum.yacy-websuche.de/viewtopic.php?t=2057
16 years ago
orbiter 2e3186189b fix for mediawikiIndex surrogate producer + added concurrency
16 years ago
apfelmaennchen 6f5ea7b1a8 small fix for previous post
16 years ago
apfelmaennchen 138a0747e3 added serverObjects.putJSON as JSON has very particulare encoding requirements
16 years ago
orbiter d977dd9a96 fix for surrogate loader
16 years ago
orbiter 9cb68353da fix for bug in ProfilingGraph for ppm >> 10000 ppm (!)
16 years ago
orbiter 9e4db75aac reduced internal logging and reduced memory that internal logging can use
16 years ago
orbiter c10c257255 attempt to fix a deadlock situation where the IODispatcher did not work.
16 years ago
orbiter 09987e93fd fixed some more bad handling of byte[]
16 years ago
orbiter 1bcc1450cb more explaining error message in case of IOExceptions during html parsing
16 years ago
orbiter fe51f4d668 less synchronization may help to prevent deadlocks
16 years ago
orbiter 58802e4201 added missing success test in storeDocumentIndex,
16 years ago
orbiter 171e62bee5 addition to the fix from last commit (which did not work)
16 years ago
orbiter 059949a0d1 tried to fix problem with snippet fetch for second search page when verify=false
16 years ago
lotus b08991e278 moved some constants, rename of Tray class
16 years ago
orbiter 138422990a - removed useCell option: the indexCell data structure is now the default index structure; old collection data is still migrated
16 years ago
orbiter 1b9e532c87 some concurrency for wikipedia dump reader
16 years ago
lotus 25d2160288 small fix
16 years ago
orbiter 16baa7ad24 To translate a mediawiki dump into the YaCy surrogate format do the following:
16 years ago
orbiter 0b2c98edc9 some more work on the wikipedia-dump exporter (not finished yet)
16 years ago
orbiter 5195c94838 two patches for performance enhancements of the index handover process from documents to the index cache:
16 years ago
orbiter 9416f5c26f more speed test cases: kelondro provides map functions that are more than 20% faster than standard java classes and use less than halve of the memory of java classes:
16 years ago
orbiter b53790abb1 more performance hacks: 10% more speed for Base64.compare() which is really often used in YaCy code
16 years ago
orbiter 8ffb9889e1 some fixes and performance hacks
16 years ago
orbiter dfb96ecb72 more fixes
16 years ago
orbiter 1b8d346b4c fixes in connection with transiton to byte[] hashes
16 years ago
f1ori 0b0a46d35a * fix transferRWI as suggested by celle (thanks!)
16 years ago
orbiter 996572de95 quickfix
16 years ago
orbiter 380ed2dac0 performance and debugging additions
16 years ago
lotus 635b0a9da7 code-split
16 years ago
orbiter fa3adbbfc6 added domain checks to surrogate reader and RWI transfer receiver to prevent spaming using surrogates
16 years ago
f1ori 76af84d732 * add custom comparator to ScoreCluster for byte[]
16 years ago
lotus ab0030d7a7 allow dht-out for remote-crawl processing peers on default settings
16 years ago
low012 d1116c049f *) added new method "contains()" to Blacklist interface
16 years ago
f1ori 08445e42f0 * don't throw exception, in case of bad charset in http-header
16 years ago
f1ori 2f860a2564 * convert byte[] hashes to string for log output
16 years ago
f1ori d93a2a6552 * ignore whitespaces so you can copy&paste signatures better
16 years ago
orbiter fbcbcc5bdb export of yacy document objects as dublin core record in xml
16 years ago
orbiter d7cbf4cdd4 more performance hacks: less overhead in word hash computation
16 years ago
orbiter 29e96c1a60 bugfixes and performance hacks
16 years ago
orbiter 4e97a31009 corrections in dublin core syntax
16 years ago
f1ori 44daec7936 * introduce signatures to autoupdate
16 years ago
orbiter 538e375901 replaced old caching method for computed word hashes with a better method. The word hash computation is a new performance bottleneck (after the IO bottleneck was removed with the IndexCell data structure) and a better caching for word hashes was necessary.
16 years ago
orbiter 9e853e1977 partly reverting SVN 5818: identical comparator required for join operator
16 years ago
orbiter e16c25ddf7 (peak-) performance hacks
16 years ago
orbiter 63cd152969 fixes
16 years ago
orbiter 7dfe7e7cc6 fixed some problems with surrogate reader. This is now ready for testing.
16 years ago
orbiter 3a1364ed5c removed example lines from SurrogateReader sources; added additional example file
16 years ago
orbiter 9050a3c4c5 alpha version of surrogate reading and indexing.
16 years ago
orbiter b15b059c0d fix for latest commit
16 years ago
orbiter c8624903c6 full redesign of index access data model:
16 years ago
f1ori dd6b5005ff * fix missing charset handling in getpageinfo_p
16 years ago
orbiter bd5f4c78d8 - added default profile for surrogate indexing
16 years ago
orbiter ad78e3a59f - less lines in rssTerminal
16 years ago
orbiter bc80dc913a added new surrogate reader (surrogates are parsed documents on batches)
16 years ago
orbiter 12d81e98eb - fixed bad search results when searching for empty string
16 years ago
orbiter 8a24350036 - fix for join method with new generalized RWI data structure (caused by latest commit)
16 years ago
orbiter e58320a507 added more info in log fore debugging
16 years ago
orbiter 89ec3acb3e - full abstraction of index content type: the kelondro full text index may now also contain indexes about other content than text, i.e. navigation indexes or reverse linking indexes.
16 years ago
borg-0300 7a48090fcf - fix for "uk" language
16 years ago
orbiter dc2af61bc9 allow up to 50 results from remote peers
16 years ago
orbiter c0e8ed5461 fixed problem with not http client
16 years ago
orbiter 8862a2fed0 ups
16 years ago
orbiter de68948bc5 better handling of free memory computation and emrgency cache flush for index cell
16 years ago
f1ori fcb77c3140 * added .im (Isle of Man) to TLD-list
16 years ago
orbiter b81c7467d8 protection against too many files in RICELL in case of massive emergency dumps caused by low memory
16 years ago
orbiter d4d87d90c4 - extended experimental wikipedia dump parser
16 years ago
orbiter c3aff2521e fix for NPE
16 years ago
orbiter 57c00dd8c9 fix for bad filtering of common http error
16 years ago
orbiter 14361f1ca4 added log message for index generation in HeapReader
16 years ago
orbiter c08f9b36a4 refactoring of wiki parser.
16 years ago
orbiter 44e01afa5b - refactoring
16 years ago
orbiter 82fb60a720 increased memory limit for emergency cache flush
16 years ago
low012 9180617dd9 *) Classes to handle import of lists (especially blacklists) from XML files, not used yet, but will be used soon.
16 years ago
lotus 596e6215dc fix in case of white space in path name
16 years ago
orbiter b887f4a116 keep more free mem
16 years ago
orbiter c2359f20dd refactoring: better abstraction of reference and metadata prototypes.
16 years ago
orbiter ab656687d7 more strict BLOB initialization .. may also help to save some ram
16 years ago
orbiter 5b138ada16 fixes to web structure reference collection and url construction
16 years ago
orbiter a29a11e526 added evaluation of incoming links in webstructure api
16 years ago
orbiter f6691411b5 - migration of files from SplitTable (which are used for the URL-DB) to a different file name format.
16 years ago
shostakovich 1f37cc6107 Robots.txt is now reused after one day. See forum-topic:
16 years ago
orbiter f21a8c9e9c a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.
16 years ago