Commit Graph

3335 Commits (1a51d9fcfdba3f922efdba4082bee6b3dc5ee05e)

Author SHA1 Message Date
orbiter b098522977 some very small advances to index utf-8 (not working yet), inserted also debugging code
16 years ago
orbiter 2f49666908 integrated the character decoding into the parser, removed old code
16 years ago
orbiter 49293c1358 fix for deadlock in new encoder :-(
16 years ago
orbiter 0edec2b760 FULL redesign of algorithms in htmlTools to encode/decode strings from/to unicode and html.
16 years ago
orbiter 958ec20cd0 removed specialized umlaute-handling in html parser. This has to be replaced by something that is able to transfer all possible html encodings into utf-8. Please see SVN 5293 for test cases.
16 years ago
f1ori 2e53cbc66a should compile now
16 years ago
f1ori f3bf2e379e should compile again
16 years ago
f1ori dd8441f102 fix bug: data from plasmaParser is allready converted to UTF-8
16 years ago
orbiter 6941bf42b1 performance hacks
16 years ago
orbiter 9b0c4b1063 redesign of parts of the new BLOB buffer
16 years ago
orbiter 1778fb420d - added some performance tweaks to the new BLOB buffer
16 years ago
orbiter 9663e61449 added another class to handle BLOB writings to the new HTCACHE data storage:
16 years ago
orbiter 382226da94 fix for bug introduced in SVN 5281: parameters were switched
16 years ago
danielr f2fd043797 refactoring (moved duplicate code into methods)
16 years ago
danielr c612046e5e r5278 java 1.5 compatible
16 years ago
f1ori af71ec93bf ops, forgot to import something
16 years ago
f1ori 9e65e9141c * always use UTF-8 for encoding hashes
16 years ago
orbiter 826ca79735 refactoring and new architecture to store the files of the web cache:
16 years ago
danielr f095137238 - respecting httpdMaxBusySessions (refusing new connections if limit is hit)
16 years ago
orbiter 8ba33f104e fix for npe
16 years ago
orbiter 998861acfd - some refactoring in BLOBHeap to enable more gap processing functions
16 years ago
lotus 9d50bfd0b3 fix for npe: http://forum.yacy-websuche.de/viewtopic.php?p=10562
16 years ago
orbiter 766cad6e93 enhancement in memory management of BLOB Heap files / merging of deleted entries
16 years ago
orbiter 7860d5d632 fix for bug in seed list management (cause was bad class overloading, only visual effects!)
16 years ago
orbiter ffed5fc415 fixed problem with lost peers in database
16 years ago
orbiter 6fb865fbdc - fix of bug in iterator in kelondroBLOBHeap which caused bug in crawl profile listing
16 years ago
orbiter 2d65887723 - fix for bug in new profile handling
16 years ago
orbiter ff68f394dd fix for problem with balancer and lost crawl profiles:
16 years ago
lotus fb8d9850ea fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1462
16 years ago
lotus 0d1a2f6183 fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1461
16 years ago
orbiter 9ac16f565b - fixed several bugs in database management functions
16 years ago
orbiter 820a03f9d6 - removed some warnings
16 years ago
lotus fe2792e9ce use accept-language header instead of user agent for language detection
16 years ago
orbiter c8bdd965ec - larger update time for status page
16 years ago
lotus dda771db9d - search result layout
16 years ago
orbiter ce4715e305 removed indexing of anchor links and tagging such words as part of urls (that was wrong)
16 years ago
orbiter ce57de6cb3 - fixed re-setting of DHT Send/Receive settings
16 years ago
lotus 31c31e54e4 new tray icon image for different icon sizes (e.g. linux)
16 years ago
f1ori 9589dfe080 * removed trayicon popupmenu title
16 years ago
lotus 5a637f004d localized tray
16 years ago
lotus 9d4f0325e1 - removed shutdown from search page (we have it in tray now!)
16 years ago
lotus 214277dad6 - revert r5202
16 years ago
f1ori 7afa084207 * add nativ java trayicon, using reflections
16 years ago
apfelmaennchen b97ff24b43 bookmarksDB / xbel.xml:
16 years ago
orbiter 6e7d113eac fix for wrong index initialization after network switch
16 years ago
lotus 0a0cc3bf67 added missing classes to build target "run"
16 years ago
orbiter 7b35d54c6c fixed some problems with network switching (was not completely 'clean')
16 years ago
orbiter f0b42e5a98 fixed NPE
16 years ago
orbiter 8e0de7f180 update to language statistic evaluation:
16 years ago
orbiter 1198eeecc7 added language selection to search query:
16 years ago
orbiter 00c1535f84 added ranking and evaluation of language type in a search
16 years ago
lotus a81cb78211 finally some putHTML on htroot/xml/
16 years ago
orbiter bfcf9b7aa3 - added language detection using metadata from documents: html and odt documents provide this information
16 years ago
apfelmaennchen 5e8bd0f29c small fixes to getpageinfo_p.xml and htmlFilterContentScraper.java with respect to keyword extraction
16 years ago
apfelmaennchen 5b2a57bfd0 - /xml/util/getpageinfo_p.xml added <desc> and <lang> tags
16 years ago
orbiter e1f67262f7 - added and removed some debugging output
16 years ago
orbiter ce2a7ed116 integrated language detection classes into condenser environment
16 years ago
orbiter 2b13705839 fixed a mistake in indexing queue processing: documents had been parsed before it was checked if they should be indexed or not. parsing was not necessary for this check, so the check was moved in the queue in front of the document parsing
16 years ago
orbiter 21dbb39afa switched two balancer cases
16 years ago
orbiter 1bbf362cef update to the crawl balancer: better organization and better crawl delay prediction
16 years ago
orbiter ddcf285499 - fixed a bug in performance setting (did not work with german translation)
16 years ago
orbiter 0cd0fee546 fixed bug with wrong proxy result enqueueing. See:
16 years ago
orbiter 670244849d fix for http://forum.yacy-websuche.de/viewtopic.php?p=9835#p9835
16 years ago
lotus fd9233244e configurable free disk space via disk.free
16 years ago
orbiter 25a62cdc3f small fixes
16 years ago
lotus 73f233bb11 * set resource observer to 1000MB
16 years ago
orbiter 5fbccfd75e fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1366&p=9348#p9348
16 years ago
orbiter a28faabfd2 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1351&p=9242#p9242
16 years ago
apfelmaennchen 7b63c66a08 - bugfix in bookmarksDB.Tag.hasPublicItems()
16 years ago
orbiter 1fb1665e71 increased dht interval to avoid peer selection failure
16 years ago
orbiter 1eb813bd43 shifted index deletion-on-exit rule to the class where the errors are produced
16 years ago
f1ori ba76995d2c * fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1415
16 years ago
f1ori bea6c13139 * with r5137 robotParser didn't work at all -> fix
16 years ago
lotus 3ded1efe84 kelondroExceptionCounter didn't work
16 years ago
f1ori ae677e1738 * fix problem in robotparser, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1421&p=9742
16 years ago
lotus 383d89481e count errors before deleting collection.index
16 years ago
lotus 0bb4fbc403 delete corrupted collecion.index on exit for rebuild on next start
16 years ago
lotus b68d06a6e8 performance settings based on network's remote crawl speed
16 years ago
danielr d60b2b198d proxy fixed 'not modified' http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1419
16 years ago
f1ori bd0318ba81 * YaCy only supports gzip-encoding, so remove any other encoding from request
16 years ago
orbiter bb5c898441 enhancements to localsearch behavior
16 years ago
orbiter 42e2d195ac added hint from http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1294
16 years ago
orbiter 39964e88fa fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1329#p9121
16 years ago
orbiter 3f3673b6e5 extended balancer:
16 years ago
orbiter 3c6e8d2015 set default ppm when network is switched
16 years ago
orbiter 3288c19c1a reduce remote crawl PPM for fresh peers in freeworld to 6 PPM
16 years ago
lotus 5ce9a100bb fix(2) for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1416
16 years ago
danielr cf29ca19d4 possible fix for POST character encoding http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1374
16 years ago
danielr a2eeb6138c fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1416
16 years ago
orbiter d09ddabd09 corrected a design mistake (5-byte hashes not necessary)
16 years ago
orbiter c97d0fcee7 modified the domain list export function:
16 years ago
orbiter 77ee0765a4 - added domain statistic generation to IndexControlURLs_p.html servlet
16 years ago
orbiter 80a7bc93d6 - added statistical evaluation about domains that appear during crawling
16 years ago
orbiter 4fbee21cea - added fetch-ahead again (had been removed in last commit)
16 years ago
lotus 423a89ebe8 * fix if yacy was installed to a path with whitespace
16 years ago
orbiter fc03b0437a fixed a error case where a second search after a first search with a different search word failed
16 years ago
orbiter eca171ba2e fix for case where javascript was not filtered by the html parser
16 years ago
lotus e645bae29f display table in log
16 years ago
orbiter ead39064c5 fixed problem with wrong result number calculation
16 years ago
hermens 2437beb96c fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1360&p=9321#p9321
16 years ago