Commit Graph

205 Commits (bb2095fe39ec4a57560d13cf83c470814a4df1f6)

Author SHA1 Message Date
borg-0300 bf14e6def5 *) proxyCache, proxyCacheSize can be changed under 'Proxy Indexing' 20 years ago
theli 2a081c9ee5 *) Adding additional logging message for "NURL.entry() == null" Bug 20 years ago
theli b70de495a0 *) Remembering Crawler-isPaused setting 20 years ago
theli e569a84dc0 *) Using the same configuration settings for all indexing threads on server Startup 20 years ago
theli 17be77a468 *) Bugfix for "Crawler data will not be removed from htcache if content parsing failed" 20 years ago
allo eb6365c069 local Bootstrapping bug. 20 years ago
theli 330eae7cf3 *) Normalizing CrawlerStartURL now before crawling is started 20 years ago
theli d4a045d7b1 *) Trying to solve "de.anomic.plasma.plasmaSwitchboard.deQueue': null" Bug 20 years ago
orbiter 25f632dbd9 more DHT bugfixes and better logging of DHT effects 20 years ago
orbiter 5cb00889d9 enhancements to dht selection, search and search presentation 20 years ago
orbiter ba0a486328 moved printStackTrace() to logging 20 years ago
orbiter cd10370992 several bugfixes and dht selection / logging improvement 20 years ago
orbiter c8a7a85ce2 fix for http://www.yacy-forum.de/viewtopic.php?p=7384#7384 20 years ago
orbiter 7db543a9fa fixes for several dht misbehaviours 20 years ago
orbiter 5716f8521d bug fixes for word ordering and dht index selection 20 years ago
orbiter f5259f29e8 word cache behaviour fix and other fixes 20 years ago
orbiter 2c234e1b82 better log output for search result 20 years ago
orbiter 248c24b60a intermission-feature usage in case of local and remote search 20 years ago
theli 865b9490a2 *) Making DHT Transfer while Crawling configurable 20 years ago
orbiter 2d8557cb10 minor changes 20 years ago
orbiter 91163db52e fix for more time-related problems in proxy 20 years ago
orbiter 40da910f41 bugfixes and automatic news-cleanup 20 years ago
theli 228b04b499 *) Bugfix for "wrong seed-upload timestamp" problem 20 years ago
theli 470839a16a *) Crawler/Session pool settings will now be stored properly into configfile 20 years ago
orbiter 1022fbeb65 many YaCyNews fixes 20 years ago
orbiter 13abd8b6e7 added news-creation at crawl start 20 years ago
orbiter cdbbfd50fb fixed bad remote crawl behavior 20 years ago
orbiter 81e564edb8 faster crawl profile list cleanup 20 years ago
orbiter ad90f0ad13 activated RWI distribution to DHT for senior peers (default redundancy 3), necessary now for network growth 20 years ago
orbiter b9d18d40cb configuration of proxy idle time in performance menue 20 years ago
orbiter c64970fa47 re-implemented proxy-busy-check and fixed some other things 20 years ago
orbiter b73557ed2d better assortment monitoring and enhanced profile menue 20 years ago
orbiter 9f505af7aa preparations for bulk remote crawls 20 years ago
orbiter 51962d55bf added 'PPM', page-per-minute statistics 20 years ago
orbiter 159f795f65 bugfix (null pointer exception in assortments) 20 years ago
orbiter 1d2155675b changed assortment memory cache flush 20 years ago
orbiter 19dbed7cc8 code clean-up 20 years ago
orbiter 40036ba69c fixed dht transmission; added url-blacklist blocking also for remote search 20 years ago
orbiter 311e627363 blocking of blacklisted urls in indexReceive and small changes 20 years ago
orbiter 277048501e bugfix 20 years ago
orbiter 8b89c46afe fixed problem with cache write 20 years ago
orbiter 419f8fb398 fixed bugs/missing code regarding new crawl stack 20 years ago
orbiter 858cd94299 replaced indexing ram-queue by file-based stack-queue 20 years ago
theli 0e2c33ee55 *) Network.html/Network.java: 20 years ago
orbiter eb74fa0c82 fixed a bug with snippet-length 20 years ago
orbiter 86f2aa8478 fixed seed-load date bug (evaluating server date for age computation) 20 years ago
orbiter 75ebdbc852 enhanced snippet-generation (case where snippet is too long) 20 years ago
orbiter 8a4f297324 fixed/enhanced snippet error-handling; suppression of results where no snippet exists 20 years ago
orbiter 712fe9ef18 bugfixed utf-8 decoding and parser 20 years ago
orbiter 3addf58046 enhanced snippet-loading with threads 20 years ago
orbiter 56d28a16f0 bugfixes 20 years ago
orbiter d6c85228a6 enhanced snippet computation 20 years ago
theli aae9a433a6 *) correcting usage of supportedFileExt-List 20 years ago
orbiter 1e7f062350 many bugfixes, memory leak fixes, performance enhancements; new kelondroHashtable; activated snippets 20 years ago
orbiter 68dc2b0c6b added kelondroArray, the basis for upcoming kelondroHash and some bug fixes 20 years ago
orbiter a19541e563 code-enhancements after analysis with AppPerfect 20 years ago
orbiter 85075269a6 extended fail-safe memory-managament. prevents too much allocation, too often GC and should help for the 100%CPU-bug 20 years ago
orbiter e3c92818db avoiding OutOfMemoryError routines 20 years ago
orbiter 3e8ee5a46d enhanced caching in kelondroRecords and added better synchronization/finalizer 20 years ago
orbiter 5d06ded005 enhanced html parser speed 20 years ago
orbiter 5a490aa065 fixed html parser 20 years ago
orbiter a25b5b4986 fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary 20 years ago
theli 9e47ba5ad6 *) adding missing calls for function close() to avoid "too many open file" bug 20 years ago
orbiter a1ffc27041 preparations for image/movie/music indexing 20 years ago
orbiter a5b40923b6 added word migration to assortments (start with 'java -classpath classes yacy -migratewords') 20 years ago
theli ee9e110366 *) removing old logging configuration properties from yacy.init 20 years ago
theli c1a4e0dc28 *) changing reference to logger 20 years ago
orbiter 4574fa4ce7 bugfixes 20 years ago
orbiter 33f9315e58 implemented multithreading of indexing 20 years ago
orbiter ca3b4ccaf4 added snippet-routines (not yet finished) 20 years ago
orbiter 594c591223 changes towards 0.38 20 years ago
orbiter d8fdc2526e added experimental snipplet-generation (to be disabled for 0.38) 20 years ago
orbiter 3771b10b89 implemented automated migration indexCache 0.37 -> indexAssortmentCluster 20 years ago
orbiter e89ded9e41 bugfixes 20 years ago
orbiter 3d8a2ff937 enhanced parallelization of local/global/remote crawling 20 years ago
orbiter 21110dcd5e fixed bugs with open files and caching 20 years ago
theli 74eb21f62e *) adding image tag into rss template 20 years ago
orbiter 5c6147a54c introduced assortment structure (generalization of singletons) 20 years ago
theli 73e297f30f *) adding proper default values for RealtimeParsableMimeTypes if something goes wrong with the configuration file 20 years ago
theli 361f05978d Multiple updates regarding the yacy seedUpload facility, 20 years ago
theli ddc5675781 *) Correcting typo 20 years ago
theli d2c4e9a55e *) Implementing yacy forum wishlist item: "Pause Crawling" 20 years ago
orbiter b4030e5023 implemented serverSwitchActions - action-hooks 20 years ago
orbiter 1d7fed87dc redesign of index caching - removed indexCache.db 20 years ago
rramthun 3f85978519 Fixed one spelling mistake, limited input for ICQ numbers to 9 digits and made ICQ number in peer profiles clickable. 20 years ago
theli 2aa5fe8f50 *) Import statements reorganized 20 years ago
orbiter 48650c082c fixed 100%-CPU-Bug in plasmaCondenser 20 years ago
orbiter 995673d795 several bugfixes 20 years ago
orbiter 2de90020ed fixed caching+synchronization+brute-force-denial 20 years ago
orbiter 9156fd53bc fixed bugs in last commit 20 years ago
orbiter e25f2354c2 removed synchronization and thread blockings 20 years ago
theli 58a65b60bd *) synchronized keyword removed from function processLocalCrawling to avoid deadlocks. 20 years ago
theli 65fc650109 *) plasmaCrawlLoader shutdown problem fixed (hopefully) 20 years ago
orbiter ba16da72b4 fixed not-working kelondroRecords-Cache 20 years ago
orbiter 7fb645b0ab enhanced crawling performance, changed memory settings, new performace options 20 years ago
theli 58b1a0ba40 *) adding an new package for extra content parsers 20 years ago
orbiter 8b31f9e202 enhanced shut-down behaviour & added experimental nio-wrapper for kelondroRA (not active yet) 20 years ago
orbiter 00f223cfc1 fixed post-parsing (a case when the bluelist is empty) 20 years ago
orbiter 97ec8d65e4 fixed makerelease & clean-up of dead code 20 years ago
orbiter b9203bdb50 bug fixes and code cleaning 20 years ago
orbiter c0807abd33 new crawl/proxy/cache design + fixes 20 years ago
orbiter e7d055b98e very experimental integration of the new generic parser and optional disabling of bluelist filtering in proxy. Does not yet work properly. To disable the disable-feature, the presence of a non-empty bluelist is necessary 20 years ago
orbiter a87a17a3c8 prepared generic text parser environment 20 years ago
orbiter 89eb9a2292 fixed bug with crawl profiles 20 years ago
orbiter 248077d3f0 initial load with yacy 0.36 20 years ago