boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
2776 Commits
1 Branch
20 Tags
1023 MiB
Commit Graph

1833 Commits (78d65e128ef1b7db29559996381c8d45e43e46de)

Author SHA1 Message Date
theli c5d3020941 *) better errorhandling for last commit 19 years ago
theli d0a5a53789 *) changes needed for multi-language support 19 years ago
orbiter d82875c72b removed removal of 'funny symbols' that may have caused utf-8 problems 19 years ago
orbiter 26ab1fa885 fixed null pointer exception 19 years ago
theli b0e8ff6eda *) some TODO makers for UTF-8 problem 19 years ago
orbiter 41e27b85b7 fix for crawler condition 19 years ago
orbiter 0ee7e45413 bugfix for merge method (caused by bad refactoring) 19 years ago
orbiter 5c2f30eaca adjustments to dhtInCache write 19 years ago
theli 9ecf7f0da2 *) some TODO makers for UTF-8 problem 19 years ago
theli e2f8339827 *) some bugfixes for UTF-8 related problems 19 years ago
orbiter c89d8142bb replaced old 'kCache' by a full-controlled cache 19 years ago
orbiter 6e2907135a bugfixes for remote search server part 19 years ago
orbiter cf9884e22b first attempt to implement a secondary search 19 years ago
theli 2a06ce5538 *) next bugfix for UTF-8 19 years ago
theli bdc51591ae *) UTF-8 Bug solved (hopefully) 19 years ago
theli ef751b9d33 *) removing all string operations from the template engine 19 years ago
orbiter 7ef80c1026 more debugging 19 years ago
orbiter b251076e64 avoid ConcurrentModificationException 19 years ago
orbiter 75b198bc02 - updated references to indexContainer 19 years ago
orbiter 0bed3b9ac3 removed superfluous interface 19 years ago
orbiter b7e7808ea6 wordmigration now works also for new index database 19 years ago
theli a0ddf2ec11 *) AbstractCrawlWorker.java: delete already downloaded data on crawling error 19 years ago
orbiter 4f9e42d5ed more changes towards better join-search 19 years ago
orbiter a7281a9b4d fix for last commit 19 years ago
orbiter 82a6054275 - fixed bug with new indexAbstract generation 19 years ago
theli fded1f4a5d *) better handling of maximum file size limit in crawler 19 years ago
orbiter 416b4e5c6b ups 19 years ago
orbiter 309accb983 memory control for ymage generation: 19 years ago
orbiter 74d1dea30b changes towards better join-search 19 years ago
orbiter ae4e8ce03e - cut for 'probably last html-interface version': version number update 19 years ago
orbiter 64bed59ee8 enhancements to ranking 19 years ago
theli 63893003be *) Adding settings page for the crawler which allows to specify a file size limit and the timeout to use. 19 years ago
auron_x 06b1365066 *) fixed existing protection against divbyzero and removed the new one 19 years ago
orbiter 94d7ced900 fix for last ranking commit 19 years ago
orbiter cc97a3e9c6 fixed possibly bug with indexOutOfBoundsException 19 years ago
orbiter 03835c2ee8 enhanced search result computation 19 years ago
orbiter 809960ddc6 avoid division by zero 19 years ago
orbiter ac3419b65f better debugging for indexOutOfBoundException bug 19 years ago
orbiter 75b03a4580 fix for new ArrayIndexOutOfBoundException 19 years ago
orbiter a8bc768206 enhancements to ranking evaluation 19 years ago
auron_x a82e926c5d *) fix for wrong totalPPM-calculation 19 years ago
theli 33898ae7e9 *) ResourceInfoFactory.java: Bugfix for classNotFoundException 19 years ago
theli 406e170e25 *) more verbose error message 19 years ago
theli b298474e22 *) Bugfix needed because of changed plasmaCrawlLURL.load behavior 19 years ago
orbiter c2e6cc8c6b small part of Bosts patch 19 years ago
orbiter 96c6e4e322 - enhancements to detailed search page 19 years ago
orbiter 9340dbb501 fixed all possible problems with nullpointer exception for LURLs 19 years ago
theli a5ed86105b *) bugfix for handling of ResourceInfo object in proxy 19 years ago
hermens ff4362b02d some more fixes for new plasmaCrawlLURL.load behavior 19 years ago
hermens 7aeadbe7cc another NullPointerException in http.ResourceInfo 19 years ago
orbiter 141f9e5bb4 fix for new plasmaCrawlLURL.load behavior 19 years ago
orbiter 1e7fd48afd added size method to ftpc 19 years ago
hermens 087f7511f8 prevent NullPointerException in http.ResourceInfo 19 years ago
orbiter a2525072f2 bugfix for kelondroRow - property generation 19 years ago
hydrox 59a5511dbb *) added missing static Strings as requested by theli 19 years ago
theli 6578564c9a *) Ignore more hop by hop http headers 19 years ago
theli b44514242a *) crawler/ftp/CrawlWorker.java: better errorhandling 19 years ago
theli 7d7f30139c *) crawler/ftp/CrawlWorker.java: delete old cache file 19 years ago
theli 4ae0f122f8 *) ResourceInfo.java: License header added 19 years ago
theli 043edfa4d8 *) ftp/ResourceInfo.java ResourceInfo object for ftp resources added 19 years ago
orbiter 4866868c0e added write cache for LURLs 19 years ago
orbiter 8a0e35618b enhancements to search result preparation 19 years ago
theli 5c1bb53d2a Missing description for last commit 19 years ago
theli dae763d8e3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2495 6c8d7289-2bf4-0310-a012-ef5d649a1542 19 years ago
theli 4825bfaaf3 *) Bugfix for PrintWriter Problem 19 years ago
orbiter d4c5e2af01 html-dirlist can now also be generated from existing connections 19 years ago
theli 7930839594 *) URL.java: userinfo was not taken over when generating a new url from a base url and a rel. path 19 years ago
orbiter 17ba468165 added html dirlisting generation in ftpc.java: 19 years ago
theli 7a35b8e237 *) direct access to responseheaders of sbQueue.Entry removed to make it more http independent 19 years ago
theli ffbf416e76 *) direct access to requestheader of htCache.Entry removed to make it more http independent 19 years ago
theli 3870d615e3 *) setting htCache.Entry fields to private 19 years ago
theli 393a7d10be *) setting htCache.Entry fields to private 19 years ago
theli ab5a9bee66 *) adding some copyright headers 19 years ago
theli 5847492537 *) next step of restructuring for new crawlers 19 years ago
orbiter 6cce47e217 test of ftp-urls in URL class 19 years ago
theli fce9e7741b *) next step of restructuring for new crawlers 19 years ago
theli e3f0136606 *) next step of restructuring for new crawlers 19 years ago
theli 9ded4e8d5a *) Bugfix for name resolution in proxy mode 19 years ago
theli 1c8300fcec *) Bugfix for name resolution in proxy mode 19 years ago
theli 4e2a950ac9 *) next step of restructuring for new crawlers 19 years ago
theli 09b106eb04 *) next step of restructuring for new crawlers 19 years ago
theli eb9b138986 *) next step of restructuring for new crawlers 19 years ago
theli 1395aae742 *) starting restructuring which is needed to add crawlers for additional protocols 19 years ago
theli b4acbdaa97 *) better handling of server shutdown 19 years ago
theli f3ac4dbbb9 *) better handling of server shutdown 19 years ago
theli 959b779aba *) avoid performance loss if log level is greater than 'fine' 19 years ago
auron_x 57dda1a92c *)again fixing for wrong version display, now totally working with double instead of float 19 years ago
auron_x 479b74e1dd *) fix for stupid mistake in new ppm-calc which caused decimal digits beeing written to seedinfo 19 years ago
auron_x 348258a557 *) changed PPM-calculation to be much more accurate 19 years ago
orbiter 18b6876860 new cache flush configuration settings 19 years ago
hermens f0278b4092 Bugfix for / by zero when the AssortmentCluster is empty 19 years ago
orbiter 14e0bb0dcf allow more references per word for new db 19 years ago
orbiter 985dcbde7f changed some parameters that may cause better memory usage and more indexing speed 19 years ago
orbiter b7f4a1521b added options to switch on or off the kelondroFlexTable for NURL, EURL and PreNURL 19 years ago
orbiter c26da4893b turned back NURL usage of kelondroTree, kelondroFlexTable has still problems with deleted entries 19 years ago
orbiter db1eae0227 * simplified initialization of database objects 19 years ago
hermens 0b73f2b132 Repair DNS prefetch during cacheScan 19 years ago
orbiter 27a159b401 * documentation update 19 years ago
theli f80f776b89 *) Trying to solve NullpointerException problem in function addURLtoErrorDB 19 years ago
orbiter d78b824e85 fixed problem with default path after first start-up 19 years ago
hydrox 1c99b5a484 *)fixed logging for urldbcleanup 19 years ago
orbiter 135e019883 removed one superfluous line from last commit 19 years ago
orbiter 1591a55963 added object cache miss-cache use for remove method 19 years ago
orbiter 8f3f4ab0eb enhanced synchronisation in plasmaWordIndex 19 years ago
orbiter f933f00f09 another patch to URL protocol handling for 'news', 'nntp' etc: 19 years ago
orbiter 4c6e00d80a more bugfixes for URL class, see: 19 years ago
orbiter 23dd972608 fixed memory calculation in performanceMemory web page 19 years ago
orbiter b7dc251948 fixed bugs in url class: 19 years ago
orbiter 1ce3c22761 better memory control: 19 years ago
orbiter 39b4c26bdc more memory control: 19 years ago
orbiter 3e9d509c39 some small fixes 19 years ago
orbiter 276225d79e fix for URL class 19 years ago
orbiter eb633c0a4f server threads must now supply a method that can be called in case 19 years ago
orbiter f5720cb2fa removed most synchronization in wordIndex (for testing) 19 years ago
orbiter 0187c60010 because of a bug in the JRE 1.4.2 there was no memory protection 19 years ago
auron_x 4eca0f8830 *) fixed PPM calculation for multiple indexer-threads 19 years ago
orbiter cfb51fdef1 less synchronization in plasmaWordIndex 19 years ago
orbiter d6a928c2da quickfix for http://www.yacy-forum.de/viewtopic.php?t=2705 19 years ago
orbiter 6ad471ef96 * applied many compiler warning recommendations 19 years ago
allo cf1186597b utf fix from theli 19 years ago
hydrox 9da3aa74d3 silly me, fix for the fix as advised by theli 19 years ago
hydrox bb3d9a5582 *) e.getMessage().indexOf() can only be used if there is actually an ExceptionMessage. 19 years ago
hydrox 7a54010a9c *) Iterators can't be casted to IndexContainer 19 years ago
theli 5e0b6f8f83 *) sorting peer name list on Blacklist_p.html 19 years ago
orbiter cd5f7e137c fixed problem with NURL-generation upon first startup 19 years ago
orbiter 8418af141a added several consistency checks and small changes 19 years ago
theli 9d13aeca13 *) removing class. does not work so far 19 years ago
theli 95a84ae469 *) adding missing classes 19 years ago
theli eee44be602 *) adding an interface for customized blacklist classes 19 years ago
orbiter 6d2f15971a there is a very strange error that causes that the kelondroRecords structure 19 years ago
theli d2e8e76218 *) now it's possible to configure the yacy blacklist separately for dht, search, proxy, crawler 19 years ago
orbiter 9ae9062bd3 * disabled new kelondroFlex table for NURLs 19 years ago
orbiter 689bbcf9cd replaced kelondroTree db for NURLs by new kelondroFlexTable 19 years ago
orbiter 7fbba41962 synchronization fixes 19 years ago
orbiter 328f9859a5 more synchronization in plasmaWordIndex 19 years ago
orbiter f43c90fa98 fixed handling of null referer in crawlOrder 19 years ago
orbiter 130e6d4719 generalized index object for eurl, nurl and lurl to prepare move 19 years ago
orbiter acdf24877f more synchronization against outOfMemoryError in wordIndex 19 years ago
orbiter 95160d7f2c fixed size computation of index elements from the collection index 19 years ago
orbiter 26116cabde added missing rowdef assignment 19 years ago
orbiter cfbacbbf08 reverted change in robotsParser 19 years ago
orbiter abf22f6e60 removed url normalform computation from htmlFilterContentScraper. 19 years ago
orbiter 740d49751d * strict type and size check in kelondroRow handling 19 years ago
orbiter 314021453f * more logging 19 years ago
allo a52f36787f better templatedebugging 19 years ago
allo 3480d36417 added some debug code 19 years ago
orbiter 61b151b083 * added another auto-fix for collection index inconsitency check 19 years ago
orbiter 0bbbd129ef small fix for exception message 19 years ago
orbiter 718fbc2dae enhancements in kelondroCollectionIndex: 19 years ago
orbiter f58283def2 better control of index flush 19 years ago
orbiter 4be21a3cab ups 19 years ago
orbiter 80b6c90d54 enhancements to prevent blocking during dht transfer receive 19 years ago
theli 9f298083cd *) adding more urls to the error url 19 years ago
hermens d56f06401e - Cache known URLs during indexReceive to avoid getting blocked during loadedURL.exists() whenever possible 19 years ago
theli c09f734d06 *) offer router configuration on ConfigBasic.html 19 years ago
hermens dcbb4d0a6b Display the size of HashBlacklistedCache on PerformanceMemory page. 19 years ago
orbiter d799622da1 better flush limit for index collections 19 years ago
orbiter d468d665c9 some changes that may help to prevent deadlocks that cause an OutOfMemoryError 19 years ago
theli d54767f634 *) last step of removing embedded html from dir class 19 years ago
orbiter 279b1d969d Integrated new indexing data structure 'collections' into the main class 19 years ago
orbiter 4ff742e42d implemented indexCollectionRI 19 years ago
orbiter 01f95eccd3 re-write of kelondroCollectionIndex. This is the data structure that 19 years ago
orbiter ebc2233092 * implemented (finished) class indexRowSetContainer 19 years ago
orbiter 9183d21f25 renamed new index class to old name 19 years ago
orbiter c4e922885a replaced indexURLEntry by new class that uses a kelondroRow.Entry object 19 years ago
orbiter 0b7112f8b2 fix for missing topLevelClone in indexRAMCacheRI.wordContainerIterator 19 years ago
orbiter e357599f92 * fixed problem with indexContainer iteration from RAM: 19 years ago
theli 57fe5cc671 *) code cleanup 19 years ago
allo 4e9f02c8ec integration of Michaels string-extraction. 19 years ago
orbiter 8b77afd72c some fixes to new container merger 19 years ago
orbiter 830167596a bugfix for 19 years ago
theli 839806a775 *) serverPortForwardingUpnp.java: code cleanup, license header added 19 years ago
theli 03230cd887 *) removing old port forwarding classes 19 years ago
theli 6e676224d0 *) adding support for upnp 19 years ago
orbiter 417ed5102e redesign of database iterators: 19 years ago
theli 0db237467f *) bugfix for URL generation from file 19 years ago
orbiter ad692fc6c7 implemented option to extract nurls from the database 19 years ago
orbiter 7fd90ca7c8 * strict handling of NURL entry element generation, storage and stacking 19 years ago
orbiter 5f72be2a95 some redesign of EURL storage 19 years ago
orbiter 1ed3e2daef added option to extract domains and/or urls from the eurl database 19 years ago
orbiter 7e0a130fb5 new indexURLEntry class 'indexURLEntryNew', to replace old class 19 years ago
orbiter 58df8b7bbf a large collection of different changes 19 years ago
orbiter e20ff77c10 another bugfix in new url class 19 years ago
orbiter 685430a1b5 bugfix in new URL class, better loggin for domain extraction 19 years ago
orbiter 79af283f6c better debugging in new URL class for wrong port numbers 19 years ago
allo 1b2ea58ee9 wrong substring invocation. 19 years ago
orbiter e4f1820b58 protection against too long authentication strings in switchboard 19 years ago
orbiter b3f7e62e03 better handling of whitespace 19 years ago
orbiter 4149939c02 better handling of whitespace for gettext quotation 19 years ago
orbiter 97fa6788a1 added gettext support: 19 years ago
theli b3c569f706 *) renaming of function getTransferedEntitySpeed to getTransferedEntrySpeed to avoid confusion 19 years ago
orbiter 67edd80884 removed tabs 19 years ago
allo 67c486a023 some example Code, how supertemplates can be used. 19 years ago
orbiter 5214f571cd simplified method call in balancer 19 years ago
allo 7b0e2521bb Support for a supertemplate, which can do all thing, a normal template can do. 19 years ago
orbiter 4bd626572b added hashCode and compareTo to new URL class 19 years ago
orbiter abb5264929 fix for 19 years ago
theli a70cbd959b *) further improvements for the anomic.net.url class 19 years ago
theli 8a1f1d96b3 *) Bugfix for url concatenation. Relative urls with / or http:// at the beginning 19 years ago
rramthun ca33eaa442 - Some spelling 19 years ago
allo 8795875800 dirlisting for all empty directories. 19 years ago
orbiter 7935f27038 enhanced synchronization in balancer 19 years ago
orbiter 3879a0ecd0 replaced java.net.URL usage by use of new class de.anomic.net.URL 19 years ago
orbiter 07900366ac deactivated cache-initialization for file-indexes (files in WORDS) 19 years ago
orbiter 40aa735520 fixe timing problem causing too long delay during initialization of kelondroTree objects 19 years ago
orbiter d2bb3f442e fixed timing problem causing a division by zero exception 19 years ago
allo 6acb6a4d8f tiny performance optimization 19 years ago
allo 2bdf1fc360 totalPPM 19 years ago
theli 24a02cbeef *) Bugfix for not parsable application/xhtml+xml resources if 19 years ago
orbiter b0ca5fa784 some correction algorithm for preload time computation during assortment open 19 years ago
orbiter e22cbaee97 - extended logging for preload 19 years ago
orbiter 671fd9a5c9 work towards new indexing database structure 19 years ago
orbiter 92f4cb4d73 added option to configure the start-up delay time for kelondro database files. 19 years ago
orbiter ce9dd3e76d some work in the index construction zone (no effect yet) 19 years ago
theli fe617d7e54 *) adding function to return the protocol type of a ssl connection 19 years ago
orbiter 018b3e0832 added pause option to server threads. 19 years ago
orbiter e1a52bea22 added a class stub for the new database structure: 19 years ago
orbiter 3b69b35bf2 added pre-load of node cache entries to kelondroRecords 19 years ago
orbiter 85d575e928 enhancements to kelondroRow and kelondroColumn 19 years ago
orbiter ab1ed053f5 another small correction 19 years ago
orbiter b92561fb67 removed unused code 19 years ago
orbiter eadbd56fc5 small adjustment to last commit 19 years ago
orbiter e9765ac4e6 introduced bulk read for node iterator in kelondroRecords 19 years ago
orbiter 6643da3fbd bugfix for http://www.yacy-forum.de/viewtopic.php?p=23463#23463 19 years ago
orbiter 866d53ed70 fix for DNS block bug 19 years ago
orbiter 6af70febef - added kelondroTree index option to kelondroFlexTable 19 years ago
orbiter dd2865178a major bugfix (searched a whole week for the bug) for 19 years ago
orbiter f9b9d085c4 just changed testing code 19 years ago
theli b594ee9a5a *) Adding possibility to configure if the http proxy should send the 19 years ago
orbiter ef84fc4956 added IOException to size() and row() 19 years ago
orbiter 84dfd76a6a kelondroFlex bugfix 19 years ago
hydrox 8ba8e2b7d9 *) added cache for blacklists urlhashs recieved by DHT. DHT does not request URLs listed in this cache. 19 years ago
hermens 53cbcc6d6e Implement emergency break in index receive when the limit of the ramCache is exceeded by more than cacheLimit 19 years ago
orbiter e40987ecab removed default memory reservation for testing 19 years ago
orbiter 4cc6e6551f bugfix 19 years ago
orbiter 66964dc015 removed high/med/low from kelondroRecords cache control. 19 years ago
allo 6866bc2758 be quiet! 19 years ago
borg-0300 4c6083b264 network picture; 19 years ago
borg-0300 955915385a network picture; 19 years ago
borg-0300 027fa8ab1c network picture; 19 years ago
orbiter 41c4641612 added some profiling to kelondro caching classes 19 years ago
orbiter dd560e4b2f finetuning 19 years ago
orbiter 5b1d77cd4b some enhancements to caching 19 years ago
theli b20496e42b *) make DHT DoS check configurable (requested by KoH) 19 years ago
orbiter 650c7e9e55 some enhancements to caching 19 years ago
hermens d4645062bc Correct usage of vhost in wget/wput requests: 19 years ago
orbiter bde748f68f - integrated buffer in RowCollectionSet which speeds up the new 19 years ago
orbiter d083b15c4b tests/fixes 19 years ago
orbiter ce9a98ee25 experiments with buffer for collection class 19 years ago
orbiter 6aa0f6eb0f better abstraction, less synchronization 19 years ago