boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
7605 Commits
1 Branch
20 Tags
1023 MiB
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0819e1d397'
${ noResults }
Commit Graph

5093 Commits (0819e1d397b6d14aa1939b658a0ad284b0aa3e5e)

Author SHA1 Message Date
sixcooler 07f5954570 try better handling of corrupt blobs
14 years ago
orbiter f970670a7c - bugfix in ServerScannerList
14 years ago
orbiter 8e03b8ee8b better integration of server list in interactive search
14 years ago
orbiter 0a3ab7da1b do not sort concrrently the same array
14 years ago
orbiter 594d8f546a #cccamp11 maintenance fix: anons may find up to 1000 items in interactive search (was: 100)
14 years ago
sixcooler eb14111200 encapsulate potential expensive objects in TextSnippet to allow GC them asap
14 years ago
orbiter 0d33cf352b removed synchronization in DNS resolve (solves a problem when loading snippets but in the past concurrent dns requests also caused deadlocks. but this is many years ago and we will give it another try)
14 years ago
orbiter e3fc1efbef performance hack and ensuring termination in serverAccessTracker. cause:
14 years ago
orbiter 44d74f8f89 performance hacks for seed generation (because thread dumps showed multiple occurrences at these code points)
14 years ago
sixcooler 5cd07d7f84 early freeing resources on deleting index reference if search-verification fails (aka Switchboard.cleanupJob)
14 years ago
sixcooler a311596881 finishing up my commits (7855-7858) which could be helpful for
14 years ago
sixcooler 9170a434ed throwing an exception again in FileUtils.copy(reader, writer)
14 years ago
sixcooler c0caca57e3 stoping thread for fetching searchresults if running short on memory
14 years ago
sixcooler ce248cc8dd less byte-arrays of response-content, less byte-array <-> stream conversation
14 years ago
sixcooler 59b767eebd stop loading via http at defined maximum of bytes - even size is unknown before loading
14 years ago
sixcooler 916d79111e Runtime.maxMemory() DOES change @ runtime:
14 years ago
f1ori 3a5fa73008 * revert parts of previous commit, because it breaks the trickle-feature
14 years ago
f1ori 6e79675ff3 * use gzip-encoding in more cases
14 years ago
orbiter 299af4943c added another memory protection hack
14 years ago
orbiter 1f300217f8 more protection for the cleanup thread
14 years ago
orbiter d13103a0a7 changed the way how the index cache is flushed: do not flush when a put was made because that could cause that many put calls synchronize for a long time when the dump or a merge is performed. Instead a watchdog thread is doing the dump and therefore puts cannot block any more which is good when a put happens during a search result preparation.
14 years ago
orbiter b06faab9d3 do not allocate a StringBuilder object in case that there is not enough memory for that
14 years ago
orbiter 6a6f27eaf3 do not sort arrays again if arrays are already sorted
14 years ago
orbiter 3d043ce9d6 - refactoring
14 years ago
orbiter 48b78e9ff4 disabling concurrency in new sort since that is not working yet correctly
14 years ago
orbiter 62ac73a108 fixed bugs and deadlocks in core database indexing structures:
14 years ago
sixcooler aff875baef smaler ping-entry @ ProfilingGraph
14 years ago
orbiter 1912d0cccc changed handling of RowSet element retrieval: until today all elements had been copied from the underlying byte[] arrays into a new Entry object that again had a copy of a portion of that byte[] in its own bye[]. There was an option to just refer to the underlying byte[] with a pointer but that was almost never used. This commit now changes an interface to the Row class where it is now necessary to tell if a copy is always required. Fortunately the copy is only needed in very rare cases. That means that this change should cause much less memory allocation; it is expected that this happens especially during search situations.
14 years ago
orbiter bb8e3f8523 code cleanup
14 years ago
orbiter be15874be1 added request line in http which can support better debugging
14 years ago
orbiter 11dc653de3 added a visualization of peer pings to the performance graphic
14 years ago
orbiter 3a191cdf14 because newbies are scared about the memory consumption in the performance graph and arguments about high memory consumption according to bad knowledge about java garbage collection techniques, the memory display had been removed from the performance graph shown on the Status.html page. The memory graph can still be seen on the Performance page where the memory graph is just like it was.
14 years ago
cominch 09bb7a390c do not replace malformed or invalid URLs in urlproxy
14 years ago
orbiter 52d799e7c8 fix for solr auth
14 years ago
orbiter 9eb8e9acd9 no error message about missing browser in headless environments
14 years ago
orbiter d3c89b90ce temporary adding the old httpclient-3.1 again because the solrj classes need them. should be removed as soon solrj supports httpclient-4
14 years ago
orbiter bd99969758 fixed bad query
14 years ago
orbiter 768c59740c - replaced solrj 3.1 with solrj 3.3
14 years ago
low012 c7b95e8c81 *) Invalid crawl profiles (containing invalid mustmatch/mustnotmatch filters) will be moved from active crawls to invalid crawls (new file: DATA/INDEX/freeworld/QUEUES/crawlProfilesInvalid.heap). This file can not be edited yet, but it shoudl be easy to extend the CrawlProfileEditor accordingly.
14 years ago
orbiter 6d2e252bcf fix for:
14 years ago
orbiter 719777b2a7 replaced method to call getUsableSpace using reflection with direct call since we now use java 1.6
14 years ago
orbiter 2d4bb139d3 - added counting of links with noindex tag for solr index
14 years ago
orbiter 892caccdca added default configuration in ConfigurationSet in case of new values
14 years ago
orbiter bda3eec0ff added parsing of canonical link element to html parser
14 years ago
orbiter b6f09a475d - added an index profile editor in the /indexFederated_p.html servlet for solr indexes
14 years ago
orbiter b666a929e7 fixed Semaphore handling in case of interruptions
14 years ago
orbiter de7a054d77 added parser for such files like the new solr.key.list
14 years ago
f1ori a17351dcfe * navigation bar for filetype constraints
14 years ago
f1ori 96957375cc * fix url proxy for relative links and chromium
14 years ago
orbiter 9ebc75db4b fix for channel authorization
14 years ago
orbiter 267290a821 removed the semaphores from the cache dump process because I believe some of the semaphores may be lost somewhere which then causes that the cache is never flushed and then the peer dies from a OOM. The re-introduced synchronization may not be the best solution but should ensure that the caches are flushed.
14 years ago
orbiter 6d9e5865ee faster appearance of search result page (but complete search time is the same)
14 years ago
orbiter f7ca84cfc0 enhanced template engine
14 years ago
orbiter d8072d1866 added more info to DNS cache in /PerformanceMemory_p.html
14 years ago
orbiter f803da8aae code cleanup
14 years ago
orbiter 84c9658644 added a file type navigator
14 years ago
orbiter 31283ecd07 - added a search option to filter only specific network protocols. i.e. get only results from ftp servers. Just add '/ftp' to your search.
14 years ago
orbiter 4b425ffdd2 fix for http://bugs.yacy.net/view.php?id=41
14 years ago
orbiter 7db208c992 performance hacks: more pre-allocated StringBuilder
14 years ago
orbiter 87bd559c42 fixed warning
14 years ago
orbiter 07e89a7ae5 added @Deprecated
14 years ago
orbiter 9706fc55aa enhanced content scraper (should discover urls much faster in case of very large plain texts)
14 years ago
orbiter 996f0a8764 disabled assert in Base64Order which eats away too much performance during testing with -l
14 years ago
orbiter f667b9c289 enhanced identificator: using AtomicInteger for counter
14 years ago
orbiter 16327d1cbe unwrapping of call depth (one call less for UTF8.String)
14 years ago
orbiter f30d36b101 enhanced template engine
14 years ago
orbiter aa6c32d753 enhanced UTCDiffString
14 years ago
f1ori f87865a50b always shutdown log, fixes zombie processes in init stop script
14 years ago
orbiter 115abc8917 - more attributes for search progress bar
14 years ago
sixcooler 7bfa6bb4b6 prevent getting a yacySeed from zero-length-hash-string by chance
14 years ago
orbiter bce280a308 update on options for interface graphics
14 years ago
orbiter 77fe69395d added jempbox-1.5.0.jar which is required by pdfbox-1.5 as stated in http://pdfbox.apache.org/dependencies.html
14 years ago
sixcooler df1725ef43 re-enable POST over proxy, which didn't work since update to httpcore-4.1.1
14 years ago
orbiter 2683162ec5 - added more options to access grid picture, web structure picture and network graphics
14 years ago
orbiter 0c1b29f3c9 - applied many small performance hacks
14 years ago
f1ori 900dacbf97 * improve link rewriting in proxy-url
14 years ago
f1ori dc855d881b * further improve proxyurl
14 years ago
orbiter a7a6b392f5 code cleanup
14 years ago
orbiter fe0c08455b more concurrency (enhancement) hacks
14 years ago
orbiter 0e9a99cb05 another resource hack
14 years ago
orbiter 535b6b953c more hacks to omit superfluous string object allocation
14 years ago
orbiter 87082f407e less String object creation during search
14 years ago
orbiter ab5a16b957 lesse memory occupation during ranking and faster host navigator
14 years ago
orbiter 1489ebeedf one more hack to free ram for search events
14 years ago
orbiter 3c2b994bd6 write access/load time to solr index
14 years ago
orbiter a36fda991e hack to increase speed of url hash computation
14 years ago
f1ori ddcc333acc * fix negative result counts
14 years ago
orbiter fa734bdf9f better memory protection in search logger
14 years ago
orbiter dbea40d536 - changed snippet fetch strategy logic: do not check if entry is in cache. This should reduce IO load on the HTCACHE which is a showstopper during large number of search requests
14 years ago
orbiter 4bea3f9714 hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources:
14 years ago
orbiter 746e3c3b06 Replaced a widely-used Property Object in the httpd with HashMap<String, Object> which is not synchronized like Properties
14 years ago
f1ori 14e1666b21 * fix replacing regexes in url proxy
14 years ago
orbiter e28bd0d038 fix for some possible causes of memory leaks
14 years ago
orbiter 09ba6814c0 - non-blocking word hash computation with dynamic digest object generation (this was important!)
14 years ago
orbiter 10e2f588f8 - enhanced ybr ranking computation
14 years ago
orbiter bd55dcee50 - commented out experimental distributed ranking loading
14 years ago
orbiter 98c4d25185 fix for endless loop in FTP crawling, see http://bugs.yacy.net/view.php?id=32
14 years ago
orbiter d1dbbd956a always use a template method cache even if the template cache flag is set to false. This flag is only used to make dynamic updates to the template files, to not dynamic updates to the rewrite methods (which is not possible without recompiling). low memory usage is guaranteed by the usage of soft references which are dropped before an OOM is thrown
14 years ago
orbiter 0d040ff6bb fix for bug 0000036: no crawling of https pages
14 years ago
orbiter 3ed4a09368 small features, some bug fixes and performance hacks
14 years ago