Commit Graph

718 Commits (f7c4abfdd713659975550ca60ba91af79b2b702d)

Author SHA1 Message Date
orbiter 610b01e1c3 - added a 'add every media object linked in a html document as a new document' to the html parser. This causes that all image, app, video or audio file that is linked in a html file is added as document. In fact that means that parsing a single html document may cause that a number of documents is inserted into the search index.
13 years ago
orbiter 51cf697acd refactoring: moved all score-related classes to new ranking package
13 years ago
sixcooler 5cd07d7f84 early freeing resources on deleting index reference if search-verification fails (aka Switchboard.cleanupJob)
13 years ago
sixcooler 59b767eebd stop loading via http at defined maximum of bytes - even size is unknown before loading
13 years ago
orbiter 7db208c992 performance hacks: more pre-allocated StringBuilder
14 years ago
orbiter 115abc8917 - more attributes for search progress bar
14 years ago
orbiter 0c1b29f3c9 - applied many small performance hacks
14 years ago
orbiter 4bea3f9714 hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources:
14 years ago
orbiter 746e3c3b06 Replaced a widely-used Property Object in the httpd with HashMap<String, Object> which is not synchronized like Properties
14 years ago
orbiter 09ba6814c0 - non-blocking word hash computation with dynamic digest object generation (this was important!)
14 years ago
orbiter 10e2f588f8 - enhanced ybr ranking computation
14 years ago
orbiter 5b579e21a3 code cleanup
14 years ago
apfelmaennchen 8b8db2aaba YMarks: some small changes/fixes
14 years ago
apfelmaennchen 441035f1f4 YMarks: some improvements to flexigrid quick search on YMarks.html
14 years ago
orbiter 6fa439c82b - refactoring of robots
14 years ago
apfelmaennchen e7c2ea193b YMark:
14 years ago
orbiter b77b8cac0c - enhanced html parser: recognized much more details in the content
14 years ago
low012 bc84d2bc9d *) fixed typo in stop script
14 years ago
apfelmaennchen b2281f0b7d YMark: intermediate work towards flexigrid support
14 years ago
low012 06d50fd801 *) fixed stupid bug (introduced in r7663 by myself) which caused wrong parsing of Wiki pages
14 years ago
apfelmaennchen 60412d2bb3 YMark:
14 years ago
low012 2e9694c9e9 *) removed recursion which hopefully prevents exception
14 years ago
apfelmaennchen a2e86daae9 YMark: more bug fixes
14 years ago
apfelmaennchen 62855f9567 YMark: code clean up and some small fixes
14 years ago
apfelmaennchen 667e912b19 YMark:
14 years ago
apfelmaennchen a0e4960a4d YMark:
14 years ago
orbiter 958ff4778e enhanced location search:
14 years ago
orbiter c17d102bd8 enhanced speed for OrderedScoreMap inc method and size comparisment in concurrent environments
14 years ago
orbiter 01690eab86 fix for mediawiki importer and wikicode parser
14 years ago
orbiter 4c013d9088 more UTF8 getBytes() performance hacks
14 years ago
apfelmaennchen 78d6d6ca06 refactoring for ymarks
14 years ago
orbiter 17530ca7b5 fix for bug http://bugs.yacy.net/view.php?id=10
14 years ago
low012 1ff9947f91 *) added new user right: extended search right (allows to define users who can query more results than anonymous users)
14 years ago
low012 e7860b1239 *) <mode="Homer">D'oh!</Homer>
14 years ago
low012 82f1580a60 *) trying to fix ConcurrentModificationException
14 years ago
low012 9f0286b380 *) fixed potential "java.lang.IllegalArgumentException: Illegal group reference" which occured if special characters which are also used as metacharacters in regular expression were used inside of <pre>...</pre> (see: http://veerasundar.com/blog/2010/01/java-lang-illegalargumentexception-illegal-group-reference-in-string-replaceall/)
14 years ago
orbiter a50f28e6e7 - fixed missing save operation for peer name change
14 years ago
orbiter 8f11d3a5bb redesigned the ScoreMap classes:
14 years ago
orbiter dc0db3550e avoid string conversion
14 years ago
orbiter 694fa3a2a5 - replaced more direct string-based UTF-8 conversions by predefined UTF-8 conversion
14 years ago
orbiter 7962d35425 - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons:
14 years ago
orbiter e1b6916423 always try to guess the size of a StringBuilder to prevent too many memory re-allocations
14 years ago
low012 3b40b98256 *) set SVN properties
14 years ago
orbiter cb1f49d0f2 replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'.
14 years ago
orbiter 5e186e0122 continuing the fight against deadlocks during time formatting: better caching.
14 years ago
orbiter 5892fff51f introduction of dht-burst modes: this can expand the number of target peers in some cases where a better heuristic is needed. The problematic cases are either when a muti-word search is made (still a hard case for our term-oriented DHT) or when a network operator wants that all robinson peers are asked. We therefore introduced two new network steering values that switch on more peers during the peer selection. Because the number of peers can now be very large, the number of maximum httpc connections was also increased.
14 years ago
orbiter 4588b5a291 - fixed document number limitation for crawls that restrict the number of documents per domain
14 years ago
orbiter 88773e4daa changed the default port from 8080 to 8090
14 years ago
f1ori 4e29e9712a * create cleanupjob for cached failed urls
14 years ago
orbiter 10ae8d961b - cora package has now no dependencies to other yacy packages and becomes a 'base' package (refactoring)
14 years ago