boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
7305 Commits
1 Branch
20 Tags
1023 MiB
Commit Graph

4488 Commits (b2fe4b7b1aec5e0df2ef5ae6a65b255b1033aeaf)

Author SHA1 Message Date
orbiter 64f29f990e a collection of performance hacks and code cleanup: 15 years ago
orbiter 8b8107b2a3 reduced IO-load and synchronization/blocking 15 years ago
orbiter 3a50b5aa04 enhanced object hash computation 15 years ago
orbiter 1a8a134e0c continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790 15 years ago
orbiter dde394a977 - shifted some computation out of synchronization to allow more concurrency 15 years ago
orbiter f204076d25 removed usage of temporary files: causes too much IO 15 years ago
orbiter 48b9371735 changed balancer re-load counter. causes less blocking here doing intranet indexing. 15 years ago
orbiter 0d04ab1422 new access tracking data type strategy; previous data types may have caused deadlocks of httpd when performing statistic cleanups 15 years ago
low012 fc43f3028e *) hopefully fixing NPE issue introduced in r6797 15 years ago
orbiter 55d8e686ea performance hacks 15 years ago
orbiter 2e26744f4e more concurrency when normalizing RWI entries + cleanup 15 years ago
orbiter 555b333041 fix for wrong count of server processes. may fix non-access problems in some cases 15 years ago
orbiter 4917f96729 fixes for some changes in SVN 6797 that caused NPEs when the bookmarks initialized 15 years ago
low012 dff660441a *) changes for better code readability 15 years ago
low012 15d9ea8375 *) changes for better code readability 15 years ago
low012 2bc459252e *) changes for better code readability 15 years ago
orbiter 67ec58d8e7 search performance enhancement 15 years ago
hermens 4ec0092677 more null == proxy fixes 15 years ago
hermens 2f90f0ad56 Remove asserts blocking proxy use cases 15 years ago
sixcooler eb2a4bb555 workaround(?) for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770&start=0&st=0&sk=t&sd=a&hilit=DefaultCharsetStringPart 15 years ago
orbiter 25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 15 years ago
low012 b97ad0f380 *) some minor changes for better code readability 15 years ago
orbiter ba51d140e1 added more info in assert in balancer 15 years ago
orbiter a85c5bb8a7 added support for multiple (fail-over) network definition locations when http-locations are given. multiple locations can be given with a comma-separated list of urls pointing to the network definition file 15 years ago
orbiter 9b3840cb66 performance hacks for the template engine + cleanup 15 years ago
orbiter 5c10f8bc5f enhanced latest hack 15 years ago
orbiter b3238bec83 performance hack for httpd 15 years ago
orbiter 1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over: 15 years ago
orbiter 749ffbd642 - added another catch case for the index dump and index merge process that should cause non-blocking behavior in case that index dump and/or index merge caused any unexpected exception. 15 years ago
orbiter 9ddb8e4a43 set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles. 15 years ago
orbiter e12f1fd821 - added setting of access rights for executable scripts after auto-installation 15 years ago
orbiter 95f31da8da increase dump cache queue length from 1 to 2 15 years ago
orbiter 6c093d6aed - enhanced domain navigator computation 15 years ago
orbiter bb63c5d075 using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets 15 years ago
orbiter 90dd197ae7 - no latency for local crawls 15 years ago
orbiter bfb518cd47 some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard 15 years ago
orbiter c855fc48c6 only load robots.txt for http and http protocol 15 years ago
orbiter 748abfcffa added patches to prevent yacy-protocol DoS settings 15 years ago
orbiter e820ed061a avoiding excessive DNS lookups to determine localhost 15 years ago
orbiter de88200e11 - added Byte Order Mark recognition to serverObjects 15 years ago
orbiter 3300930fc5 - (almost) fixed FTP crawler 15 years ago
orbiter 9623d9e6d2 added a smb loader component for the YaCy crawler 15 years ago
orbiter 48995e71c4 added soft-auth to general authentication scheme 15 years ago
orbiter 72f00dee59 removed never-used server access account function 15 years ago
orbiter 57e1eae95e longer time-out for url fetching .. may help to show all that links that the statistic say for a search result 15 years ago
orbiter 9e639603e3 after frequent occurrences of 100% CPU usages and permanent blockings I try to disable a function in a method that may cause the problem when calling an external library (apache http client 3.x). The thread dump that shows the problem is attached here. 15 years ago
orbiter 4144927d94 show less errors 15 years ago
orbiter b88f5fbb4b slightly changed crawling policy 15 years ago
orbiter 7684a575c4 fix for deletion of error database each time when YaCy starts up 15 years ago
orbiter f561e340c6 show more results of single domains when not authorized fully (up to 100) 15 years ago
orbiter c4bdb1e7f2 added one more option in ViewFile to show an iframe like for the orginal web page content but using the cache than the direct link to the content in the web. Upgraded the very old and previously not any more used CacheResource_p servlet to a new and working version. 15 years ago
orbiter c09a995930 better logging of double occurrences of urls in the crawler 15 years ago
orbiter 884b262130 - added a new Wiki Namespace Navigator 15 years ago
orbiter 617dfbbd06 allo 'authorization by encoded password' also if requesting client is not from localhost but from the same host as yacy is running on. 15 years ago
orbiter 599c3766c4 added authentication to automated API call 15 years ago
orbiter 727dd9b193 - fixed a bug in robots.txt parser 15 years ago
orbiter 54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request 15 years ago
orbiter 46c4f8b68a better look-ahead into the crawl queue: show more on crawl monitor 15 years ago
lotus 7b546415dc added svn6695 for windows 15 years ago
orbiter f175f9a2d3 changed way how number of search requests are counted: 15 years ago
orbiter 84222e3b4f fix for auto-updater: delete old libraries before copy of new one 15 years ago
orbiter 93b7ddc27d fix for http://forum.yacy-websuche.de/viewtopic.php?p=19376#p19376 15 years ago
orbiter 8030ed3319 self-healing for lost crawl profile handles 15 years ago
orbiter e3e5e05ec2 fix for problem in ranking setting which was caused by the introduction of a toString() method in serverObjects 15 years ago
orbiter e3ccfb54aa fix for display problem in Firefox on MacOS X 15 years ago
orbiter 564927ce72 redesign of CrawlResult data structures because of OOM occurrences during URL deletion processes. 15 years ago
orbiter ef62d017e5 integrated session id filtering for crawler 15 years ago
orbiter d8d9984913 added framework for session id filtering (not ready yet) 15 years ago
orbiter 2bc36de336 - fix for bug in svn 6669 15 years ago
orbiter d378ca4604 better handling of concurrency in seed 15 years ago
orbiter 6538043d89 fix for http://forum.yacy-websuche.de/viewtopic.php?p=19189#p19189 15 years ago
lotus 945e0ba5a5 allow global search if res. observer disabled index transmission 15 years ago
lotus 8faeedd99a not a fix! for: 15 years ago
lotus 11188cd7eb resource observer now uses the Java 6 method to check for free space. thus, disk observing now needs Java 6 installed. 15 years ago
orbiter be18b5d8cd fix for 'cannot switch back to default language'-bug 15 years ago
orbiter 74e736c903 missing file for last commit 15 years ago
orbiter 308a973503 refactoring of tables data organisation 15 years ago
orbiter 8a76f38d26 Added a new steering servlet that can be used to repeat actions that had been made on the yacy interface. This can be used to: 15 years ago
orbiter 840527689b more simplification of bookmark class 15 years ago
orbiter d77782a8d5 removed bookmark tags file, tags are now stored only in RAM 15 years ago
orbiter ada0ce9de3 refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks 15 years ago
orbiter a131ebbcb5 one more fix for NPE, see 15 years ago
orbiter 24060885b6 - added Tables abstraction in data.Tables.java 15 years ago
orbiter 7fdf59a77f misc NPE check 15 years ago
orbiter a512aef6ad fix for http://forum.yacy-websuche.de/viewtopic.php?p=18918#p18918 15 years ago
orbiter 3889438db6 fix for bookmarks 15 years ago
orbiter 23bcca07a3 removed directly linked servlets that had been there to test memory failures that appeared in that servlets 15 years ago
orbiter 69c29acb6e no exception thread dump if parser cannot parse becuase that mime-type/extension is in the deny-set 15 years ago
orbiter 8ce936bcdd added an api recording function: it shall be possible to record 15 years ago
orbiter e80e060ca6 - increased thread priority for server threads 15 years ago
orbiter 234f733a3d - relocation of seed db is better for network switch than re-initialization because of the embedding of the peers object in other objects 15 years ago
orbiter 473b11033d fixed network switch process - crawling did not work after a switch before this fix 15 years ago
orbiter fd7b348973 some fixes for the network switch 15 years ago
orbiter f6731c6240 more logging etc. 15 years ago
orbiter d9169cc6c3 increased proxy load time-out from 30000 to 60000 milliseconds 15 years ago
orbiter 938e806182 tried to fix date problem that may have prevented that foreign peers stay in the network 15 years ago
orbiter bd05e57d3b fix for http://forum.yacy-websuche.de/viewtopic.php?p=18563#p18563 15 years ago
orbiter 5df628a2a4 - added BEncoder class 15 years ago
orbiter 82f57f79e5 more PMD enhancements 15 years ago
orbiter 5d930c96f0 more fixes to search result page navigation 15 years ago
orbiter 8c520f128d reverted a change in ranking process committed this afternoon 15 years ago
orbiter a06f7ddb33 more PMD recommendations 15 years ago
orbiter 18172451a0 better search computation: 15 years ago
orbiter d126d6c1b5 renamed the servlet WatchCrawler_p to Crawler_p 15 years ago
orbiter 66c0a8e849 more PMD recommendations 15 years ago
orbiter 909a4f91c7 added a logging output for crawl starts that shows the URL that can be used to start the crawl again 15 years ago
orbiter bc96d74813 - clean-up of robots.txt parser 15 years ago
orbiter dd459281c8 applied code changes that are recommended by PMD 15 years ago
lotus eac2daf2e8 * reenable DHT if yet enough memory is available 15 years ago
orbiter d77a8f3b3e added some modifications recommended by PMD for better performance 15 years ago
orbiter d1973bae2a code cleanup: removed unused code and unused methods 15 years ago
orbiter a3b8b7b5c5 some redesign of the main menu structure: 15 years ago
lotus ab3cf60dbe fix for npe 15 years ago
orbiter 7f20963b41 add-on to last commit 15 years ago
orbiter eeca2ded92 fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500 15 years ago
orbiter bb2e03761c - fix for deadlock with 100% CPU during search 15 years ago
orbiter dff4f95c78 some patches to get the torrent parser working 15 years ago
low012 82198acc06 *) minor changes 15 years ago
low012 b75547fc60 *) minor changes 15 years ago
orbiter 57d729e377 fix for negative numbers in network statistic 15 years ago
orbiter 4ac4fe952c patch for npe in bookmarks 15 years ago
orbiter d548bd41ad fix for a npe during search 15 years ago
orbiter 37245430c3 fix for NPE during DHT RWI selection 15 years ago
orbiter a37878b7d5 url parser regex performance hack 15 years ago
orbiter b527d2ebfa fix for media search 15 years ago
orbiter 362b7a929b added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function 15 years ago
orbiter 8281e29963 - more configuration for profiling graph (number of events) 15 years ago
f1ori 5f0f6b71b4 * revert last commit, something is more broken than before 15 years ago
f1ori 8c8b642eba * fix timezone problem 15 years ago
orbiter 4782d2c438 fix for search bug that appeared when looking at page 3 of results or further 15 years ago
orbiter 29fde9ed49 better control of ranking order in sort stack 15 years ago
orbiter 66923ebc6c - modified method in RequestHeader that delivers the host name of requester: no more reverse domain lookup (may have killed interface performance in some cases) 15 years ago
orbiter e34e63a039 preset of proper HashMap dimensions: should prevent re-hashing and increase performance 15 years ago
orbiter 4a5100789f replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration. 15 years ago
orbiter f4946eaf27 - better thread dump 15 years ago
orbiter 9743b70d1c disabled keep-alive of server, not really needed for speed but a cause for much trouble and memory occupancy 15 years ago
orbiter 491ba6a1ba - some refactoring in workflow 16 years ago
orbiter 969123385b added json and rss output for image search 16 years ago
orbiter d183f8d980 refactoring (moved code from ContentTransformer to TemplateEngine) 16 years ago
orbiter 23aef43786 - better synchronization in SortStack 16 years ago
orbiter 7b1f5b0430 - better media search ranking 16 years ago
orbiter 4df88a4e7a - fixes for missing or bad hashCode computation 16 years ago
orbiter dbdf2570ba added comparator and more fixes for SortStack/SortStore 16 years ago
orbiter 1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation. 16 years ago
orbiter fe41a84330 some enhancements in web caching: avoid double loading of response metadata and/or content 16 years ago
orbiter 06d0dcde20 more enhancements to image search 16 years ago
orbiter 4c6312d103 enhanced image search 16 years ago
orbiter 2d8f3ee301 some performance hacks 16 years ago
orbiter 94b2a664f3 - use a static DiskFileItemFactory (one instantiation is enough) 16 years ago
orbiter 013f337d3f - avoid unnecessary host name lookups for localhost 16 years ago
orbiter 20c5d78a5c fix for a ConcurrentModificationException 16 years ago
orbiter 7144d2df6e added crawlReceipt servlet as individual class to examine OOM problem as documented in 16 years ago
orbiter 29fe436e36 - fixed post-ranking including prefer mask 16 years ago
orbiter 5399d1e2bc refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets) 16 years ago
orbiter 1fa0ac26e9 better protection against NPEs during search/ranking 16 years ago
orbiter 4c99d4683d possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running. 16 years ago
orbiter 18b21eaffe small fixes to search default values and server logging 16 years ago
lotus 6edc168cfe option to disable dht by memory limit: 16 years ago
orbiter 4431b9767e added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/ 16 years ago
orbiter e3025ee691 - new icon for OAI-PMH loading action 16 years ago
orbiter f0b8db93f0 - more abstraction of serverCore thread access 16 years ago
orbiter 2889b9426e missing code for last commit 16 years ago
orbiter b6a8887ff5 better handling of running sessions without explicit hashtable 16 years ago
orbiter 1dc7ea986a added a dynamic keep-alive time-out for http server sessions: 16 years ago
orbiter b0b7a4f9a5 - added function to OAI-PMH reader that can pull all records from a server using an evaluation of the resumption token to get URL to retrieve remaining records 16 years ago
lotus 79251e6f60 configurable disk space hardlimit for dht 16 years ago
orbiter a0e891c63d - some redesign in UI menu structure to make room for new 'Content Integration' main menu containing import servlets for Wikimedia Dumps, phpbb3 forum imports and OAI-PMH imports 16 years ago
orbiter 30f108f97d added stub of oai-pmh importer (not working yet) 16 years ago
orbiter 77c99e500f added more control over memory allocation 16 years ago
orbiter 52470d0de4 - fix for xls parser 16 years ago
orbiter 5e8038ac4d - refactoring of blacklists 16 years ago
orbiter 26fafd85a5 - more refactoring 16 years ago
orbiter 3528b970d6 - refactoring 16 years ago
orbiter a8ce192f63 - shifted main classes to new package net.yacy 16 years ago
orbiter b79f4f062f refactoring of yacy documents and parsers: they depend now only on the kelondro classes 16 years ago
hermens 0fd9540866 Configuration of HTTPDProxyHandler logging 16 years ago
orbiter cee7a05ff2 - de-serialized the pdf parser 16 years ago
orbiter 9db928ce53 replaced fontbox 0.7.3 with fontbox 0.8.0 16 years ago
orbiter c2272785c7 - fix for xlsx and pptx parsing 16 years ago
orbiter c864901087 - moved httpd.mime to defaults path 16 years ago
low012 8829ec5f18 *) made sure that   is replaced with a space and not just deleted in CharacterCoding.java 16 years ago
orbiter 6c347a37eb more options for DocumentIndex 16 years ago
orbiter e7f18ba24b refactoring 16 years ago
orbiter ce8dc575ca refactoring 16 years ago
orbiter bea3b99aff moved table and util classes 16 years ago
orbiter bd876eb4b7 moved io classes 16 years ago
orbiter c0e0e1f422 moved blob classes 16 years ago
orbiter 1e4f8b56ed accumulated classes from different packages into the new rwi package 16 years ago
orbiter 194da25a2f moved kelondro index 16 years ago
orbiter 4446acc8cd moved kelondro order 16 years ago
orbiter f677d534b1 start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root 16 years ago
orbiter ea473e32b8 refactoring 16 years ago
orbiter 735e2737e3 * added index segments 16 years ago
orbiter 09de5da74a once again a performance hack 16 years ago
orbiter 2f6d88403e uä 16 years ago
orbiter d2615ea5a8 increased memory for scraper buffer to enhance parsing speed 16 years ago
orbiter 4bbbb74ec4 removed not necessary synchronization 16 years ago
hermens 67e5464cc2 Fix for SVN6380: x[] Arrays are unsuitable Keys for Maps without using a proper Comparator. 16 years ago
hermens aeab8c7917 Prevent failed DHT attemps from overwriting newer peer info 16 years ago
hermens 9324b5b6c5 Enhancements to DHT 16 years ago
hermens e49e2d75fe Limit the time Transmission.Chunks stay in the transmissionCloud by using a Map that iterates entires in insertion order. 16 years ago
orbiter 92db7c5d07 increased timeout for index retrieval 16 years ago
lotus 386b9f35f6 activated resource observer for windows 7 16 years ago
orbiter 6e0dc39a7d - some fixes to prevent blocking situations 16 years ago
orbiter 51f2bbf04b possible fix for problem in http://forum.yacy-websuche.de/viewtopic.php?p=17655#p17655 16 years ago
orbiter f8371707e5 - possibly better termination for SplitTable 16 years ago
orbiter 87780f2562 produce did-you-mean also for queries with more than one word 16 years ago
orbiter 04a548a1e3 - temporary integrated the transferURL servlet as static class instead as a class that is called using reflection to investigate the OOM problems in that class 16 years ago
orbiter ea427df944 fixed a worst case situation of the condenser which may cause a temporary full CPU load because of a bad data structure usage 16 years ago
orbiter 3e38035389 fix for interrupted thread during has() property check 16 years ago
orbiter 5bd1c1d205 just added some comments that had been produced to learn about OAI-PMH 16 years ago
orbiter 6aa474f529 - better logging for web cache access and fail reasons 16 years ago
orbiter 3671c37989 added experimental oai-pmh reader and integrated it with the existing dublin core parser 16 years ago
orbiter 58a00205d5 re-activated the emergency close when too many server connections exist 16 years ago
orbiter c57d2070e6 more logging 16 years ago
orbiter a995b95367 tried a fix for the httpd access bug (too many unclosed sessions) 16 years ago
orbiter e1fba41cad better logging 16 years ago
orbiter 2275f885a8 possible fix for concurrency problem 16 years ago
low012 a6a3090c3d *) blacklist cleaner supports usage of regular expressions now 16 years ago
orbiter 5a93807781 improved web cache speed: 16 years ago
orbiter 2e8b2867ff double performance of store method because it avoids one 'has' 16 years ago
orbiter afda5b1adc new join method for indexes (not yet used) 16 years ago
orbiter 65b66c2c18 better handling of array files of length 0 16 years ago
orbiter 1957b5797a fix for seed generation 16 years ago
orbiter 432154f725 new strategy for concurrent database index key retrieval 16 years ago
orbiter a11cd9f80f - removed reverse name lookup for http access logging (grr..) 16 years ago
orbiter 2e6bdce086 - added more logging to balancer 16 years ago
orbiter 1171a72006 fix for deadlock as seen in http://forum.yacy-websuche.de/viewtopic.php?p=17521#p17521 16 years ago
orbiter 031e6eefbd some updates to dublin core, metadata browsing, file indexing and parser stability 16 years ago
hermens 62a7341c4d Fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2204 16 years ago
low012 f65bfaa9af *) Removed base tag from errror page. This has been added by myself a long time ago as a workaround for some weird behavior of my router, but as it turns out, it does more bad than good in general: If HTTPS is used for communication with YaCy, entering a wrong passwort led to an errror page with a form which would send username and password unencrypted with the user possibly being unaware of this. 16 years ago
orbiter e4797ebcde fix for http://forum.yacy-websuche.de/viewtopic.php?p=17509#p17509 16 years ago
orbiter efa7fb34f0 better oom-awareness of miss-cache in cache 16 years ago
orbiter 3e9dcfc204 fix for http://forum.yacy-websuche.de/viewtopic.php?p=17504#p17504 16 years ago
orbiter c3a4aee255 some redesign with a possible fix for the ReferenceContainerCache. 16 years ago
orbiter aca8a78eb8 fix for shutdown of DocumentIndex objects 16 years ago
orbiter 23ab6fbca4 - navigation appear at correct position when opengeodb-results are also presented after a search 16 years ago
orbiter 4db34eea73 fix for OOM problem in kelondro Cache 16 years ago
orbiter 8ea1d7ab59 fix for wrong assert condition in search abstract generation 16 years ago
orbiter fbd77bd77c git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6328 6c8d7289-2bf4-0310-a012-ef5d649a1542 16 years ago
orbiter 54c7cbf1d9 - fast result for local search in case that less than 10 hits exists 16 years ago
orbiter 28d4b921b6 different approach for file search 16 years ago
orbiter f99f86c5c5 added concurrency to file indexing class 16 years ago
orbiter 902d16cf6c fixes to parser 16 years ago
orbiter 4a1c852435 fix in usage of RAM copy for Table objects and some cosmetics in asserts. 16 years ago
lotus dce450e2e0 possible fix for "hung" doc-documents 16 years ago
orbiter e627f75415 one more fix to badwords and stopwords 16 years ago
orbiter 721b88efbd - fixed a problem loading blacklists with new yacycore.jar 16 years ago
orbiter 80d5005044 fixed seed upload methods - replaced reflection with direct instantiation 16 years ago
orbiter 68465c37af added a convenience class to add files into a YaCy index 16 years ago
orbiter 2e41e10ffd - updates to yacyVersion parser (remove old targets) 16 years ago
orbiter 27d00285aa - added a new file reader cache that may serve as full-file-copy of blob database files. This is not yet used 16 years ago
orbiter fd6b9cb7dc refactoring of IO access classes 16 years ago
orbiter d64569aa39 reuturn only recommendations of words that have a greater count than the original word 16 years ago
orbiter 604c37927f used comparator for did-you-mean that uses index sizes for comparisment, but: 16 years ago
orbiter a58d9cae7d - show location name in geolocalization search result 16 years ago
orbiter 573d03c7d7 added configuration to enable ram table copy 16 years ago
orbiter 3be54e1891 fix to rule when to use a ram table copy 16 years ago
orbiter 700218846c disabled or removed sleep calls 16 years ago
orbiter 342c5d0fd4 fixed city name detection: finds now also substrings of city names 16 years ago
orbiter 18aa0609ca fix for caching of word hash computation 16 years ago
orbiter a10a6cce45 patch for http://forum.yacy-websuche.de/viewtopic.php?p=17289#p17289 16 years ago
low012 53bbdfd19a *) setting SVN keywords 16 years ago
low012 25f6145934 *) preventing null pointer exception in case empty search word or only one character is enterd or all search words are removed by filters 16 years ago
low012 248f3fd9b5 *) cleaned up code for better readability 16 years ago
orbiter eaddf2d464 - corrected layout of map preview 16 years ago
hermens 4b83875abd Small fixes for the heapCacheIterator in ReferenceContainerCache: 16 years ago
orbiter fd668f531b fixed map layout 16 years ago
orbiter 2740d9dd79 added integration of osm maps for search 16 years ago
orbiter af3a696fc4 added a fast-fail concept in search processes. The search now has better control if all the remote searches may bring any result. If all processes are finished, then all search tasks fail fast. 16 years ago
orbiter ce972ff4ef update to default ranking profile which has now some settings to deny some phpbb3 pages which are redundant in the index when crawling phpbb3. 16 years ago
orbiter 44579fa06d - fixed a problem loading images through yacy's document loader, 16 years ago
orbiter 67eddaec4b changed way to integrate dictionary files: 16 years ago
orbiter d656a94f55 fix for bad paths in dictionary processing 16 years ago
orbiter 3b9aaf9e9f - inserted new library tests inside DidYouMean 16 years ago
orbiter 8c35ffe34c fixes to the dymlib 16 years ago
orbiter bfa273bcc1 added a library provider which holds libraries in static objects, 16 years ago
orbiter 1762a7bcd6 - moved DidYouMean to the data package 16 years ago
orbiter bf8ed00e9e removed debugging code 16 years ago
orbiter ead48c4b25 fix for preparation of search result pages with offset > 10: 16 years ago
orbiter 39a311d608 better care to do not loose the merge/dump thread 16 years ago
orbiter 10d3e856b5 better concurrency, less blocking & performance hacks 16 years ago
orbiter 1a9cfd8718 some performance hacks (CPU only, not IO) 16 years ago
orbiter 92407009b2 cleanup 16 years ago
orbiter 0ba1beaf56 separated rwi constraint evaluation from rwi ranking and added concurrency 16 years ago
orbiter ce7924d712 better concurrency for rwi entry parsing during search processing 16 years ago
orbiter b0637600d5 enhanced url constraint computation: better position of constraint check during retrieval process 16 years ago
orbiter 61748285c3 more refactoring of search 16 years ago
orbiter 323a8e733d removed unused classes 16 years ago
orbiter 72e5407115 refactoring of snippet cache 16 years ago
orbiter 0e471ba33b - fixed a bug in fast digest computation 16 years ago
low012 93b2622503 *) repaired and added IM online status indicators 16 years ago
orbiter e7736d9c8d more refactoring: made all variables in SearchEvent private 16 years ago
orbiter 4b92d0b9b7 patch for possible problems with normalization of '/' in urls. This applies in rare cases when '/' appear in post-properties 16 years ago
orbiter d8ca6e6bf1 more refactoring for search 16 years ago
orbiter fe4a4e3f6b added missing class 16 years ago
orbiter 72ac5bd80f refactoring of search process. 16 years ago
hermens c4d0e22a77 Further speed upof concurrent DHT-receive 16 years ago
hermens 2fbc0696bf Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2334 16 years ago
f1ori d515bc11e2 added ooxmlparser 16 years ago
orbiter d9744b1b5d replaced old caching strategy control class with lightweight simplearc 16 years ago
orbiter 8e56c2ace6 fix for fixes from this afternoon 16 years ago
orbiter cf739edc2e fix for possible deadlock, see 16 years ago
orbiter 6354b5e447 removed possible deadlock, see 16 years ago
orbiter 5cc17ccf8a a better caching with less overhead and more appropriate 16 years ago
orbiter 92edd24e70 fixed problem with switching of networks 16 years ago
orbiter 0575f12838 fix for deadlock 16 years ago
orbiter fbfdaf063d - patch to omit IndexOutOfBoundsException when a b64-encoded key appears not to be well-formed. In that case the key is still accepted but rated higher than other regular keys to create a virtual ordering between well-formed and ill-formed keys 16 years ago
orbiter c0e17de2fb - fixes for some problems with the new crawling/caching strategies 16 years ago
orbiter 634a01a9a4 replaced wget-requests with caching requests 16 years ago
orbiter c6c97f23ad - added cache usage properties to crawl start 16 years ago
orbiter c4ae2cd03f fixed bug that caused deletion of crawl profiles at every application startup 16 years ago
orbiter 161d2fd2ef redesign of access to the HTCache (now http.client.Cache): 16 years ago
f1ori ba2e6de538 fix empty version string again 16 years ago
orbiter 51534df0cb fix for possible synchronization problem 16 years ago
orbiter 4da9042e8a code simplification 16 years ago
orbiter 1d8d51075c refactoring: 16 years ago
f1ori 67da20647f * add new odf parser based on sax-xml-parser 16 years ago
f1ori 6d0e6d591b * ops, fix compiler error :( 16 years ago
f1ori 3e5beb1654 * fix for empty version in seedlist 16 years ago
orbiter 5bb8074150 removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency. 16 years ago
orbiter eea4c17ef2 removed rpm parser 16 years ago
orbiter b332dfad67 - inserted request object into response object which carries this now instead generating new objects 16 years ago
orbiter ca72ed7526 -removed superfluous crawl cache 16 years ago
orbiter 13c63f4082 a set of small fixes to crawling behaviour 16 years ago
orbiter a564df3984 update to mime types in parsers and httpd.mime 16 years ago
orbiter 43c8defd79 enhanced parser with more extension + mime attributes 16 years ago
orbiter 49bbb9bd45 replaced tar library with integrated apache ant tar lib 16 years ago
orbiter b2263bc720 enhanced document type recognition 16 years ago
lotus aa38eb5a20 * maxfilesize -1 for infinite filesize 16 years ago
lotus 9cfe89c8fc * process content-length as soon as it is received 16 years ago
orbiter 50cf80056f removed jmimemagic library 16 years ago
orbiter 3f113f38a8 removed unused imports 16 years ago
lotus 9f083bb6b2 check filetype before loading (no more mp4 loading) 16 years ago
f1ori 076ae02c44 * added pl and py to extensions excepted by htmlParser 16 years ago
f1ori d5e51cfd09 * workaround for non-working build property replacements 16 years ago
f1ori f814e0fa81 enable warnings and fix most of it 16 years ago
f1ori 8931c8d6b4 improvments to debianpackage: 16 years ago
low012 fc1dc38b55 *) added spaces to make sure that no words are concatinated by accident 16 years ago
low012 f242e7d7bc *) using Apache POI library to parse Word documents now 16 years ago
orbiter caedd72400 - enhanced logging and exception details for parsers 16 years ago
orbiter 4b74ad0a46 fixed setting of parser configuration servlets 16 years ago
orbiter 57a88d435b redesign of parser mime type detection and parser steering 16 years ago
lotus e15d27bc63 avoiding double/wrong parser errors 16 years ago
orbiter 21b8704fb4 refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes 16 years ago
orbiter 8ca1f5d400 - some work to integrate the html parser the same way as the other parsers are integrated (not finished) 16 years ago
low012 1ee109761f *) added changes which were lost 16 years ago
orbiter 499723891d removed all non-http daemons; they had not been used and may be a potential security risk. 16 years ago
orbiter 0e8647d62f refactoring of search classes 16 years ago
orbiter dafffd0153 refactoring of parsers and document processing 16 years ago
low012 8041e91f56 *) Ooops! 16 years ago
low012 69551ff3d9 *) added several MIME types (derived from http://filext.com/), some of them might be rather uncommon 16 years ago
low012 11dfb2d54f minor changes: 16 years ago
orbiter 77d2a3782c removed strange debugging strings 16 years ago
lotus 4320f69574 universal handling for crashed parsers 16 years ago
orbiter 024744245c small refactoring to prepare for new queues 16 years ago
orbiter 16efcd0366 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2252&hilit=&p=16389#p16389 16 years ago
f1ori 0f3246e90a * fix debian package 16 years ago
f1ori 8544cfd5a6 * remove seperate build-files for parsers 16 years ago
orbiter 24cb6d68bc - renamed Stack to RecordStack to avoid name confusion with new classes 16 years ago
orbiter 995da28c73 all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path 16 years ago
orbiter aac89bf8ca trying to avoid "exceeding limit" message of server 16 years ago
f1ori 48d78166ed * fix double copy of libraries 16 years ago
lotus 7f868ca3c2 resource observer: support for yacyroot\DATA on an NTFS hardlink (Windows) 16 years ago
orbiter 409538e17a code cleanup and code simplifcation 16 years ago
orbiter 160031758d fix for problem with initializer 16 years ago
orbiter 302a02cec8 moved all libraries from libx to lib 16 years ago
orbiter 1f1399e5c5 extending visibility of objects and methods to avoid synthetic accessor methods and increase performance 16 years ago
orbiter 154bbc3364 code cleanup: call of static methods directly to the class 16 years ago
orbiter 222850414e simplification of the code: removed unused classes, methods and variables 16 years ago
orbiter 93dfb51fd4 problems with code style 16 years ago
orbiter adf01c676e reduce lookup time when merging a large number of BLOBs 16 years ago
orbiter 9a674d8047 - After the removal of the Tree class some code simplifications are possible. This affects mostly the Records class, which can be refactored and the result of the refactoring results in a reduced number of classes. 16 years ago
orbiter c5122d6836 completed migration of BLOBTree to BLOBHeaps: 16 years ago
orbiter d1083a6913 maybe we have less problems with open connections to the server if we don't do BF forced sleeps (just a test) 16 years ago
low012 ebe6c823ac *) changed svn properties agains (hopefully doing it right this time) 16 years ago
low012 a80ac3a415 *) fixed wrong parser descriptions 16 years ago
low012 457b6c0d6d *) updated Apache POI library to be able to parse Visio files 16 years ago
apfelmaennchen a10c8022d1 DidYouMean: 16 years ago
f1ori 7eb3bff5b3 * workaround for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2220&hilit=#p16128 16 years ago
orbiter 99fa265e1d fix for search bug caused by tenant patch 16 years ago
orbiter 79875782af be a bit more lazy when removing domain navigation entries 16 years ago
orbiter 57af311627 fix for wrong urls in navigator when a tenant is used 16 years ago
lotus 76b96337e2 just some chatty code 16 years ago
low012 91785d895c *) minor changes in comments 16 years ago
orbiter bdda140c02 fix for json output (no doubleqotes any more, doublequote quoting did not work) 16 years ago
orbiter 2f84736120 ignore signature files that cannot be downloaded because of failed encoding 16 years ago
orbiter 041d9c253e some refactoring and more error-awareness in LogalizeHandler 16 years ago
orbiter 6b307d6d59 more tolerance for corrupted index entries in exported row sets 16 years ago
orbiter 33aafa9b4b better logging when writing merged dumps 16 years ago
lotus db70badcf0 possibility to set remote host on upnp device 16 years ago
orbiter 4d29e90708 uaeh 16 years ago
orbiter 3c3e6499ae added more logging for merge operation 16 years ago
orbiter 15180fc95e - patch for future computation in SplitTable 16 years ago
orbiter 9a5ec20b3c avoid merge during startup 16 years ago
lotus bf6b92343c try to avoid stuck pdf parser 16 years ago
lotus c695c7f512 try to remove hung swf parser from queue 16 years ago
orbiter fc69a76197 update to web structure picture: 16 years ago
orbiter ae015e8e98 refactoring of blob package classes 16 years ago
orbiter 8b8877c233 moved image collector 16 years ago
orbiter be1c7ddc64 refactoring of search classes -- moved Ranking Profile to search package 16 years ago
orbiter fd31a3616a - more logging in server process 16 years ago
orbiter 5a7fd6b4c8 just some comment lines 16 years ago
orbiter 31f60a3b3e when doing searches, also apply a online caution to DHT transmission and stop transmissions while heavy load caused by searching. This omits the many requests to the URL database that are needed for DHT transfer and it avoids collisions with URL retrieval needed for search results. 16 years ago
orbiter 17dc6d4be5 small fix for new Logger 16 years ago
orbiter ce1adf9955 serialized all logging using concurrency: 16 years ago
lotus aec3e7995a autoconfig.pac can be used to browse .yacy-domains only 16 years ago
orbiter bc6dd8194b refactoring: moved search query class to new search package 16 years ago
orbiter a4805defdd added stub for new search process 16 years ago
orbiter b8e738a7be a collection of 16 years ago
apfelmaennchen 39779e4796 DidYouMean: as I moved to only 8 consumer and 4 producer threads, I removed poison pills as it does not make sense anymore - threads are interrupted directly. Having a consumer thread per test case just didn't make sense either (see svn 6070) due to the massive overhead. 16 years ago
apfelmaennchen c3c4dd0933 DidYouMean - changed to much simpler LinkedBlockingQueue 16 years ago
apfelmaennchen 01ac1b5d7e - blocking queue implementation of DidYouMean 16 years ago
orbiter b8bb1bb364 join with a timeout does not cause that the corresponding thread is stopped after the time-out. It does only cause that the waiting is stopped. Here we need additionally a signal to the thread to stop after we finished waiting. 16 years ago
orbiter b69f22e9ca mistake in last commit: computation of loops in ReversingTwoConsecutiveLetters 16 years ago
orbiter 3130334932 - start first with threads that run more loops 16 years ago
apfelmaennchen 6cde7ebf16 DidYouMean 16 years ago
orbiter f348190566 tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release. 16 years ago
orbiter 945777aa80 replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries. 16 years ago
orbiter 7c4d1d471c hand-over of more specific object 16 years ago
apfelmaennchen 09acfa66d1 - improved "did you mean" 16 years ago
apfelmaennchen da6ce37f7b - fixed encoding problem 16 years ago
apfelmaennchen 54a48b4184 - added "did you mean" to search page 16 years ago
orbiter 0fc1168554 - reduced time-out for socket-connection communication from 20 seconds to 5 seconds. This is a test to find out if the time-out was a cause for problems in metager environments 16 years ago
orbiter 28b86385cd patch for bad behaving swf parser 16 years ago
orbiter d58b395993 fix for http://forum.yacy-websuche.de/viewtopic.php?p=15693#p15693 16 years ago
orbiter 733385cdd7 enahnced database access times by removal of unnecessary synchronization. 16 years ago
orbiter 398e210fef removed synchronization in logging that causes deadlocks in high-performance environments 16 years ago
orbiter db3a06dd81 removed cookie handling in httpc: 16 years ago
orbiter 1c54ae4a63 some small changes in HandleMap Testing 16 years ago
orbiter 2c5554c912 small enhancements in search result computation speed 16 years ago
orbiter e0b3984805 added navigation keys for site and author facets to remote search interface 16 years ago
orbiter 27fa6a66ad - completed the author navigation 16 years ago
orbiter a9a8b8d161 - added display of author navigation (usage of that navigator not yet implemented 16 years ago
orbiter c879783008 added steering of navigator computation: 16 years ago
orbiter c079b18ee7 - refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing. 16 years ago
orbiter bead0006da replaced tmp file extensions by prt 16 years ago
orbiter 3189f9cd39 fixed problem with DCEntry initialization 16 years ago
orbiter a704d82280 patch for problem with digest 16 years ago
orbiter 3029ef6eb3 fixed a bug that was recently inserted which caused that no idx and gap files were written. 16 years ago
orbiter b6e274f211 omit most of forced crawl delays by using a separat delay table which flushes delayed URLs at the correct time 16 years ago
orbiter d50be59088 - added a automatic re-construction of the domain stack after 10 minutes. this includes then urls to the domain stack that were left over in case of stack size limitations when the domain stack was created the last time 16 years ago
orbiter 5fdba0fa51 - fixed a not working selection rule in balancer 16 years ago
orbiter f5602404d5 another speed boost for the balancer 16 years ago
orbiter 95e8cbd1c3 new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls 16 years ago
orbiter c062385552 fix for http://forum.yacy-websuche.de/viewtopic.php?p=15555#p15555 16 years ago
orbiter 42ae40b9f6 some bugfixes to database close() methods 16 years ago
orbiter a0c53abbe1 - wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521 16 years ago
orbiter 9bfd22f65d fix for http://forum.yacy-websuche.de/viewtopic.php?p=15523#p15523 16 years ago
orbiter 1c77db670f re-designed response format for navigation: 16 years ago