boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
6562 Commits
1 Branch
20 Tags
1023 MiB
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '431852f0a7'
${ noResults }
Commit Graph

4357 Commits (431852f0a70195759cc6e310e273db72c7565d02)

Author SHA1 Message Date
orbiter 1a6c2f77b4 fix for NPE in statistic servlet
15 years ago
orbiter 64f29f990e a collection of performance hacks and code cleanup:
15 years ago
orbiter 8b8107b2a3 reduced IO-load and synchronization/blocking
15 years ago
orbiter ed07046870 flush only when > 3000 RWIs present + code cleanup
15 years ago
orbiter 3a50b5aa04 enhanced object hash computation
15 years ago
orbiter 1a8a134e0c continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790
15 years ago
orbiter dde394a977 - shifted some computation out of synchronization to allow more concurrency
15 years ago
orbiter f204076d25 removed usage of temporary files: causes too much IO
15 years ago
orbiter 48b9371735 changed balancer re-load counter. causes less blocking here doing intranet indexing.
15 years ago
orbiter 650be3599f added a time-out to the RWI cache to flush the cache if it has not been written for ten minutes. This additional dump criteria is necessary because some data sources repeat their vocabulary and may cause that the number of words in a RWI does not increase while the number of references in the RWI set increases. Now the RWI Buffer is flushed every 10 minutes or later if at that time already a dump is ongoing.
15 years ago
orbiter ff6cf24b80 replaced RowSetArray in ObjectIndexCache with RowSet to reduce complexity in MergeIterator. This complexity caused too much computing overhead when the RowSetArray had become very large.
15 years ago
orbiter 0d04ab1422 new access tracking data type strategy; previous data types may have caused deadlocks of httpd when performing statistic cleanups
15 years ago
low012 fc43f3028e *) hopefully fixing NPE issue introduced in r6797
15 years ago
orbiter 55d8e686ea performance hacks
15 years ago
orbiter 2f181d0027 introduced concurrency in HTCACHE storage compression
15 years ago
orbiter 2e26744f4e more concurrency when normalizing RWI entries + cleanup
15 years ago
orbiter 555b333041 fix for wrong count of server processes. may fix non-access problems in some cases
15 years ago
orbiter aa083fc45c try to get a fix for OOM problem in case that there is no real problem with missing memory.
15 years ago
orbiter 70e6222978 more concurrency during search requests
15 years ago
orbiter 4917f96729 fixes for some changes in SVN 6797 that caused NPEs when the bookmarks initialized
15 years ago
low012 dff660441a *) changes for better code readability
15 years ago
low012 15d9ea8375 *) changes for better code readability
15 years ago
low012 2bc459252e *) changes for better code readability
15 years ago
low012 dc93cec3a8 *) Java 1.5 compatibility (see http://forum.yacy-websuche.de/viewtopic.php?f=8&t=2764)
15 years ago
orbiter 67ec58d8e7 search performance enhancement
15 years ago
hermens 4ec0092677 more null == proxy fixes
15 years ago
hermens 2f90f0ad56 Remove asserts blocking proxy use cases
15 years ago
hermens ef467a0303 Another workaround for the second part of http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770
15 years ago
sixcooler eb2a4bb555 workaround(?) for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770&start=0&st=0&sk=t&sd=a&hilit=DefaultCharsetStringPart
15 years ago
orbiter 25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775
15 years ago
low012 b97ad0f380 *) some minor changes for better code readability
15 years ago
orbiter ba51d140e1 added more info in assert in balancer
15 years ago
orbiter a85c5bb8a7 added support for multiple (fail-over) network definition locations when http-locations are given. multiple locations can be given with a comma-separated list of urls pointing to the network definition file
15 years ago
orbiter 9b3840cb66 performance hacks for the template engine + cleanup
15 years ago
orbiter 5c10f8bc5f enhanced latest hack
15 years ago
orbiter b3238bec83 performance hack for httpd
15 years ago
orbiter 1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over:
15 years ago
orbiter 72d8e9897b removed unnecessary cache flush call in backend of BufferedRecords
15 years ago
orbiter 749ffbd642 - added another catch case for the index dump and index merge process that should cause non-blocking behavior in case that index dump and/or index merge caused any unexpected exception.
15 years ago
orbiter 9ddb8e4a43 set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles.
15 years ago
orbiter 312ca5d917 removed flush at end of every rwi entry since this reduces the write performance.
15 years ago
orbiter 0018163c07 moved table row/column matching method from front-end to back-end
15 years ago
orbiter e12f1fd821 - added setting of access rights for executable scripts after auto-installation
15 years ago
orbiter 31e29a8831 - removed synchronization during index dump and index cleaning
15 years ago
orbiter 95f31da8da increase dump cache queue length from 1 to 2
15 years ago
orbiter 6c093d6aed - enhanced domain navigator computation
15 years ago
orbiter bb63c5d075 using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets
15 years ago
orbiter e0da0a84b0 performance fix in http parser
15 years ago
orbiter 90dd197ae7 - no latency for local crawls
15 years ago
orbiter bfb518cd47 some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard
15 years ago
orbiter 36bd843ece for for RFC5322 comformance as suggested by Quix0r in http://forum.yacy-websuche.de/viewtopic.php?p=19585#p19585
15 years ago
orbiter c855fc48c6 only load robots.txt for http and http protocol
15 years ago
orbiter 748abfcffa added patches to prevent yacy-protocol DoS settings
15 years ago
orbiter e820ed061a avoiding excessive DNS lookups to determine localhost
15 years ago
orbiter 11983bc936 redesigned some parts of the parser entry point:
15 years ago
orbiter de88200e11 - added Byte Order Mark recognition to serverObjects
15 years ago
orbiter 89b4fff1c2 adopted ant script for new exif library
15 years ago
orbiter 24e5faee75 added exif parsing for jpg images
15 years ago
orbiter 82f76e1296 removed log line
15 years ago
orbiter 0f8004f9da enhanced html parser to recognize a href tags inside header tags
15 years ago
orbiter 3300930fc5 - (almost) fixed FTP crawler
15 years ago
orbiter 1198b9989d bugfixes, more sorttable
15 years ago
orbiter 9623d9e6d2 added a smb loader component for the YaCy crawler
15 years ago
orbiter ae2f3f000f better handling of table copy abandon .. prevent memory leak
15 years ago
orbiter 0769517129 added a robots.txt monitor in the crawler monitor submenu
15 years ago
orbiter 48995e71c4 added soft-auth to general authentication scheme
15 years ago
orbiter 72f00dee59 removed never-used server access account function
15 years ago
orbiter 57e1eae95e longer time-out for url fetching .. may help to show all that links that the statistic say for a search result
15 years ago
orbiter 9e639603e3 after frequent occurrences of 100% CPU usages and permanent blockings I try to disable a function in a method that may cause the problem when calling an external library (apache http client 3.x). The thread dump that shows the problem is attached here.
15 years ago
orbiter 4144927d94 show less errors
15 years ago
orbiter b88f5fbb4b slightly changed crawling policy
15 years ago
orbiter de01fe0e6d fix for bug in url parser
15 years ago
orbiter 7684a575c4 fix for deletion of error database each time when YaCy starts up
15 years ago
orbiter f561e340c6 show more results of single domains when not authorized fully (up to 100)
15 years ago
orbiter c4bdb1e7f2 added one more option in ViewFile to show an iframe like for the orginal web page content but using the cache than the direct link to the content in the web. Upgraded the very old and previously not any more used CacheResource_p servlet to a new and working version.
15 years ago
orbiter c09a995930 better logging of double occurrences of urls in the crawler
15 years ago
orbiter 1bbe14d23f SVN 6716 unfortunately contained parts of the unfinished SMB integration. To fix compile errors the remaining parts of the SMB implementation stub is added with this commit.
15 years ago
orbiter 884b262130 - added a new Wiki Namespace Navigator
15 years ago
orbiter 617dfbbd06 allo 'authorization by encoded password' also if requesting client is not from localhost but from the same host as yacy is running on.
15 years ago
orbiter 270fb38674 - fixed some bugs in Table viewer
15 years ago
orbiter 599c3766c4 added authentication to automated API call
15 years ago
orbiter 727dd9b193 - fixed a bug in robots.txt parser
15 years ago
orbiter 54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request
15 years ago
orbiter 46c4f8b68a better look-ahead into the crawl queue: show more on crawl monitor
15 years ago
lotus 7b546415dc added svn6695 for windows
15 years ago
orbiter f175f9a2d3 changed way how number of search requests are counted:
15 years ago
orbiter 84222e3b4f fix for auto-updater: delete old libraries before copy of new one
15 years ago
sixcooler cd6de83905 next try for for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703
15 years ago
sixcooler bfe4693e9a fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703
15 years ago
orbiter 93b7ddc27d fix for http://forum.yacy-websuche.de/viewtopic.php?p=19376#p19376
15 years ago
orbiter 8030ed3319 self-healing for lost crawl profile handles
15 years ago
orbiter e3e5e05ec2 fix for problem in ranking setting which was caused by the introduction of a toString() method in serverObjects
15 years ago
orbiter e3ccfb54aa fix for display problem in Firefox on MacOS X
15 years ago
orbiter 564927ce72 redesign of CrawlResult data structures because of OOM occurrences during URL deletion processes.
15 years ago
orbiter 30c8185139 fix for sid check
15 years ago
orbiter ef62d017e5 integrated session id filtering for crawler
15 years ago
orbiter d8d9984913 added framework for session id filtering (not ready yet)
15 years ago
orbiter 2bc36de336 - fix for bug in svn 6669
15 years ago
orbiter d378ca4604 better handling of concurrency in seed
15 years ago
orbiter 6538043d89 fix for http://forum.yacy-websuche.de/viewtopic.php?p=19189#p19189
15 years ago