Commit Graph

207 Commits (610855e362bf20e5d3fcf00839a56ea09a554862)

Author SHA1 Message Date
orbiter 1a8a134e0c continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790
15 years ago
orbiter dde394a977 - shifted some computation out of synchronization to allow more concurrency
15 years ago
orbiter 55d8e686ea performance hacks
15 years ago
orbiter 2e26744f4e more concurrency when normalizing RWI entries + cleanup
15 years ago
orbiter 67ec58d8e7 search performance enhancement
15 years ago
hermens 2f90f0ad56 Remove asserts blocking proxy use cases
15 years ago
orbiter 25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775
15 years ago
orbiter a85c5bb8a7 added support for multiple (fail-over) network definition locations when http-locations are given. multiple locations can be given with a comma-separated list of urls pointing to the network definition file
15 years ago
orbiter 1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over:
15 years ago
orbiter 749ffbd642 - added another catch case for the index dump and index merge process that should cause non-blocking behavior in case that index dump and/or index merge caused any unexpected exception.
15 years ago
orbiter 95f31da8da increase dump cache queue length from 1 to 2
15 years ago
orbiter 6c093d6aed - enhanced domain navigator computation
15 years ago
orbiter bb63c5d075 using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets
15 years ago
orbiter bfb518cd47 some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard
15 years ago
orbiter 748abfcffa added patches to prevent yacy-protocol DoS settings
15 years ago
orbiter e820ed061a avoiding excessive DNS lookups to determine localhost
15 years ago
orbiter 3300930fc5 - (almost) fixed FTP crawler
15 years ago
orbiter 57e1eae95e longer time-out for url fetching .. may help to show all that links that the statistic say for a search result
15 years ago
orbiter f561e340c6 show more results of single domains when not authorized fully (up to 100)
15 years ago
orbiter 884b262130 - added a new Wiki Namespace Navigator
15 years ago
orbiter 617dfbbd06 allo 'authorization by encoded password' also if requesting client is not from localhost but from the same host as yacy is running on.
15 years ago
orbiter 727dd9b193 - fixed a bug in robots.txt parser
15 years ago
orbiter 54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request
15 years ago
orbiter f175f9a2d3 changed way how number of search requests are counted:
15 years ago
orbiter 8030ed3319 self-healing for lost crawl profile handles
15 years ago
orbiter e3e5e05ec2 fix for problem in ranking setting which was caused by the introduction of a toString() method in serverObjects
15 years ago
orbiter ef62d017e5 integrated session id filtering for crawler
15 years ago
orbiter d8d9984913 added framework for session id filtering (not ready yet)
15 years ago
lotus 945e0ba5a5 allow global search if res. observer disabled index transmission
15 years ago
orbiter 74e736c903 missing file for last commit
15 years ago
orbiter d77782a8d5 removed bookmark tags file, tags are now stored only in RAM
15 years ago
orbiter a131ebbcb5 one more fix for NPE, see
15 years ago
orbiter 24060885b6 - added Tables abstraction in data.Tables.java
15 years ago
orbiter 7fdf59a77f misc NPE check
15 years ago
orbiter a512aef6ad fix for http://forum.yacy-websuche.de/viewtopic.php?p=18918#p18918
15 years ago
orbiter 69c29acb6e no exception thread dump if parser cannot parse becuase that mime-type/extension is in the deny-set
15 years ago
orbiter 8ce936bcdd added an api recording function: it shall be possible to record
15 years ago
orbiter 234f733a3d - relocation of seed db is better for network switch than re-initialization because of the embedding of the peers object in other objects
15 years ago
orbiter 473b11033d fixed network switch process - crawling did not work after a switch before this fix
15 years ago
orbiter fd7b348973 some fixes for the network switch
15 years ago
orbiter f6731c6240 more logging etc.
15 years ago
orbiter 938e806182 tried to fix date problem that may have prevented that foreign peers stay in the network
15 years ago
orbiter 82f57f79e5 more PMD enhancements
15 years ago
orbiter 5d930c96f0 more fixes to search result page navigation
15 years ago
orbiter 8c520f128d reverted a change in ranking process committed this afternoon
15 years ago
orbiter a06f7ddb33 more PMD recommendations
15 years ago
orbiter 18172451a0 better search computation:
15 years ago
orbiter 66c0a8e849 more PMD recommendations
15 years ago
orbiter dd459281c8 applied code changes that are recommended by PMD
15 years ago
orbiter d77a8f3b3e added some modifications recommended by PMD for better performance
15 years ago
orbiter bb2e03761c - fix for deadlock with 100% CPU during search
15 years ago
orbiter dff4f95c78 some patches to get the torrent parser working
15 years ago
low012 82198acc06 *) minor changes
15 years ago
orbiter 57d729e377 fix for negative numbers in network statistic
15 years ago
orbiter d548bd41ad fix for a npe during search
15 years ago
orbiter a37878b7d5 url parser regex performance hack
15 years ago
orbiter b527d2ebfa fix for media search
15 years ago
orbiter 362b7a929b added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function
15 years ago
orbiter 8281e29963 - more configuration for profiling graph (number of events)
15 years ago
orbiter 4782d2c438 fix for search bug that appeared when looking at page 3 of results or further
15 years ago
orbiter 29fde9ed49 better control of ranking order in sort stack
15 years ago
orbiter e34e63a039 preset of proper HashMap dimensions: should prevent re-hashing and increase performance
15 years ago
orbiter 4a5100789f replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration.
15 years ago
orbiter 491ba6a1ba - some refactoring in workflow
15 years ago
orbiter 969123385b added json and rss output for image search
15 years ago
orbiter 23aef43786 - better synchronization in SortStack
15 years ago
orbiter 7b1f5b0430 - better media search ranking
15 years ago
orbiter 4df88a4e7a - fixes for missing or bad hashCode computation
15 years ago
orbiter dbdf2570ba added comparator and more fixes for SortStack/SortStore
15 years ago
orbiter 1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation.
15 years ago
orbiter fe41a84330 some enhancements in web caching: avoid double loading of response metadata and/or content
15 years ago
orbiter 06d0dcde20 more enhancements to image search
15 years ago
orbiter 4c6312d103 enhanced image search
15 years ago
orbiter 013f337d3f - avoid unnecessary host name lookups for localhost
15 years ago
orbiter 20c5d78a5c fix for a ConcurrentModificationException
15 years ago
orbiter 29fe436e36 - fixed post-ranking including prefer mask
15 years ago
orbiter 1fa0ac26e9 better protection against NPEs during search/ranking
15 years ago
orbiter 4c99d4683d possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running.
15 years ago
lotus 6edc168cfe option to disable dht by memory limit:
15 years ago
orbiter 4431b9767e added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/
15 years ago
orbiter b0b7a4f9a5 - added function to OAI-PMH reader that can pull all records from a server using an evaluation of the resumption token to get URL to retrieve remaining records
15 years ago
lotus 79251e6f60 configurable disk space hardlimit for dht
15 years ago
orbiter a0e891c63d - some redesign in UI menu structure to make room for new 'Content Integration' main menu containing import servlets for Wikimedia Dumps, phpbb3 forum imports and OAI-PMH imports
15 years ago
orbiter 52470d0de4 - fix for xls parser
15 years ago
orbiter 5e8038ac4d - refactoring of blacklists
15 years ago
orbiter 26fafd85a5 - more refactoring
15 years ago
orbiter 3528b970d6 - refactoring
15 years ago
orbiter a8ce192f63 - shifted main classes to new package net.yacy
15 years ago
orbiter b79f4f062f refactoring of yacy documents and parsers: they depend now only on the kelondro classes
15 years ago
orbiter cee7a05ff2 - de-serialized the pdf parser
15 years ago
orbiter c864901087 - moved httpd.mime to defaults path
15 years ago
orbiter 6c347a37eb more options for DocumentIndex
15 years ago
orbiter e7f18ba24b refactoring
15 years ago
orbiter ce8dc575ca refactoring
15 years ago
orbiter bea3b99aff moved table and util classes
15 years ago
orbiter 1e4f8b56ed accumulated classes from different packages into the new rwi package
15 years ago
orbiter 194da25a2f moved kelondro index
15 years ago
orbiter 4446acc8cd moved kelondro order
15 years ago
orbiter f677d534b1 start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root
15 years ago
orbiter ea473e32b8 refactoring
15 years ago