Commit Graph

5461 Commits (125d47b3c178e018a650b1bcd5624da6a6627317)

Author SHA1 Message Date
Michael Peter Christen a58dc4a91f added autotagging to document condenser:
13 years ago
Michael Peter Christen 0d6176804b emergency disabling of GenerationMemoryStrategy because of non-working
13 years ago
Lotus 411aab02e3 Windows installer now detects reliably whether YaCy runs. A file lock on
13 years ago
Michael Peter Christen 87f0210480 enriched log output to find NPE in HeapReader
13 years ago
Michael Peter Christen 987b412491 updated solr scheme: generic declaration of solr schemes
13 years ago
Michael Peter Christen 254adea51c small fixes
13 years ago
Michael Peter Christen 49be60a7c8 WorkflowProcess is forced to make small pauses if shortMemoryStatus is
13 years ago
Michael Peter Christen b7bb84c0bb set a limit to CharBuffer object size to fight against bad/too large
13 years ago
Michael Peter Christen c602eaaf46 enhanced search process
13 years ago
Michael Peter Christen 087f97d4c0 less noise if a browser cannot be opened
13 years ago
Michael Christen eff966f396 fix for search process (it was aborted too early during remote search)
13 years ago
Michael Christen e6d51363ee Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Marek Otahal a231d0eeb9 Run from Java the whole app YACY
13 years ago
Marek Otahal 72adbeae90 !Important: move from Hashtable to HashMap
13 years ago
Marek Otahal c1af123ddd just a little faster toString
13 years ago
Marek Otahal 64e4bcee82 serverSwitch get(App/Data)Path() use common helper method
13 years ago
Marek Otahal 371fbb4deb just comment + shorter code in serverSwitch
13 years ago
Marek Otahal ed253b7aff update javadoc, does not throw IOException
13 years ago
Marek Otahal f40efb39af Blacklist loadList() remove duplicates by using Set
13 years ago
Marek Otahal f75b5e40e0 little fix in copy()
13 years ago
Marek Otahal 1dc5d9f0f3 make ConnectionInfo comparable and sort list of connections in Connections_p
13 years ago
Michael Christen fa8da7f89d vocabularies are now also used as source for a did-you-mean computation
13 years ago
Michael Christen eaec14ecc4 Dictionaries from words caches can now be used as autotagging vocabulary
13 years ago
Michael Peter Christen 91940fdf56 redesign of WordCache to be prepared to hold multiple
13 years ago
Michael Christen bd40a10230 added autotaggig stub .. only reading and parsing of vocabularies at
13 years ago
Michael Peter Christen 2ee8cbeb2c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 992dbdf4bb added noload statistic to servlets
13 years ago
Michael Christen 354b976110 fix for concurrency problem and endless loop in /suggest.json
13 years ago
Michael Christen c21966bb43 fix
13 years ago
Michael Christen 13b05f9c08 fix
13 years ago
Michael Christen e5d878c59e Merge branch 'master' of ssh://gitorious.org/yacy/rc1
13 years ago
Michael Christen ec26b2bea4 Merge commit 'fa08ed5ae5d72bddc3cc6a662b23103579e86109' into quix0r
13 years ago
Michael Christen eebc02f5c1 fix
13 years ago
Michael Christen 216a287a85 Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r
13 years ago
stbrumm d18095dc48 Patch fuer Issue 0000102
13 years ago
stbrumm 9f1b1b4604 Type for Robinson-Mode/Private Perr added
13 years ago
Michael Christen 20962a4ed7 added metadata node stub for metadata from blobs
13 years ago
Michael Christen 575dbbaa93 enhancements in Blob retrieval: try to use less CPU resources by testing
13 years ago
Michael Christen 585a8f3c44 fixed a bug in search sequence (caused emtpy results)
13 years ago
Michael Christen 361146dd7a better error handling for file loader
13 years ago
Roland 'Quix0r' Haeder 6d4e08ed06 Rewrote filesize() to (hopefully) avoid a NPE, rewrote Blacklist class to concurrent classes to avoid a CME
13 years ago
Roland 'Quix0r' Haeder 901f37d608 Also this ... :( #2
13 years ago
Roland 'Quix0r' Haeder a985717ed2 Also this ... :(
13 years ago
Roland 'Quix0r' Haeder 5f490de554 Fix for ported fix from my old days ...
13 years ago
Roland 'Quix0r' Haeder fa08ed5ae5 Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check
13 years ago
Roland Haeder 319fd1f4aa A concurrent access can happen on the blacklist (with latest introduced blacklist check in media snippet computation)
13 years ago
Roland 'Quix0r' Haeder a3083d13bf Blacklist checks are now always turned on, in media searches (e.g. image search) images matching blacklist entries are no longer shown to the user
13 years ago
Michael Christen 52184a1170 fix for search process
13 years ago
Michael Christen 85bd4cc8bc better lookup for peer names
13 years ago
Michael Christen 20e3084bd4 redesign of fining of peers by ip: more leightweight method to read the
13 years ago
Michael Christen 0797b0de99 new handling of remote search processes: looking for seeds will now not
13 years ago
Michael Christen ee9aae5cc0 more about CreativeCommons license vocabulary
13 years ago
Michael Christen ecd74fe34f less dramatic upnp failures
13 years ago
Michael Christen c75e1a3125 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Christen 13f5b5f80d the component part in the YaCy Metadata is filled using the Dubling Core
13 years ago
Michael Peter Christen 8d2cbfb685 more vocabularies and more semantics for lod data structures
13 years ago
Michael Christen 9cd36b4c44 added vocabulary for geolocalization as used in georss
13 years ago
Michael Christen 9e5894c784 Removed handling of components objects for URIMetadataRows.
13 years ago
Michael Christen 66ab51f89d added rdf vocabulary
13 years ago
Michael Christen c04bfaa51b refactoring
13 years ago
Michael Peter Christen 136b514f52 added a Triple Store based on Nodes that fit to the new storage classes.
13 years ago
Michael Peter Christen 613ab6a69d added BEncodedHeapBag and BEncodedHeapShard which are storage container
13 years ago
Michael Christen 6fecd0db88 one more performance hack to prevent costly md5 computation
13 years ago
Michael Christen e13441b069 better digest pool size (smaller by default but unlimited)
13 years ago
Michael Christen 1f4afb4dc0 performance hacks
13 years ago
Michael Christen 675d557e88 removed debug logging
13 years ago
Michael Christen e9dc99fe15 added rules to set specific RWIs as private RWIs which are not
13 years ago
Michael Peter Christen 4243ace863 added phonetic classes
13 years ago
Michael Peter Christen 0bcef2d156 added feature as requested in
13 years ago
Michael Christen 204c29f010 small bugfixes for search result display and cache display
13 years ago
Michael Christen 17f962fceb translator updates:
13 years ago
Michael Christen 752b092b8a Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Christen 078fcde0dd bad initialization
13 years ago
admin 391fc9bd57 Merge branch 'master' of gitorious.org:yacy/rc1
13 years ago
admin 23afee58fe Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
Michael Christen 14e45e90fd patch for a bug that I don't understand by now.
13 years ago
Michael Christen 3eccdca63c protection against too long running snippet fetch processes
13 years ago
Michael Christen 86b3385847 fixed a deadlock during secondary remote search
13 years ago
apfelmaennchen ff19fcdb28 bugfix for YMarks XBEL import and export; thanks to Dominic
13 years ago
Michael Christen c715d19c09 fixes for dependency on svn
13 years ago
Michael Christen 404758698a less io operations
13 years ago
Michael Christen 0bc5d76bee ups
13 years ago
Michael Christen 044f83feed added some pauses into the search process which shall produce
13 years ago
Michael Christen 943b670738 less terrible warning if uPnP fails
13 years ago
sixcooler 448656087a probably fix for http://bugs.yacy.net/view.php?id=94
13 years ago
Michael Christen f14faf503b better ranking because we wait a very little time during the search
13 years ago
Michael Christen 762e0ecfb6 fixed localization dictionaries, see
13 years ago
Michael Christen 6e66c9d7f1 fix for http://bugs.yacy.net/view.php?id=87
13 years ago
Michael Christen d35bdc2df6 removed npe
13 years ago
Michael Christen e7e429705a - less automatic indexing after a search (needs to reset the default
13 years ago
admin a4ac051029 Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
low012 7cfdc2c092 Improved CGI capabilities:
13 years ago
Michael Christen 9cd469e6d6 added pull request from als plus an NPE fix
13 years ago
admin 484c4ad339 Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
orbiter 402e9d71ef changed ording on release files: main criteria is not the svn any more; releases are now ordered by
13 years ago
admin 29f07fea33 Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
orbiter 11729061f2 added an option in the bookmark import process to put everything into the crawler
13 years ago
admin b9c5ce6cae Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
apfelmaennchen 70bcfc150a - small bug fix to ymarks html importer
13 years ago
admin 56ce8488e4 Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
orbiter 4b8ff84705 - search bugfixes (page counter and number of results per page; recognition of new search)
13 years ago
sixcooler aeeae75b8a the timeout of httpclient is not absolut, but till a connection is
13 years ago
hermens 2ac272cfbf Fix for PeerSelection.seedsByAge() for big networks (>1000 Peers)
13 years ago
apfelmaennchen b5d9f631e3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8128 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
orbiter 83335c3b09 fix for http://bugs.yacy.net/view.php?id=78
13 years ago
Al Sutton 39898cb94a Added try/finally protection to ensure streams are closed. Added initial size guess for the CharBuffer
13 years ago
Al Sutton 4c67a964a1 Added try/finally protection to ensure streams are closed. Added initial size guess for the CharBuffer
13 years ago
Al Sutton 3f9b9f953f Added close() to ensure buffer close actions are invoked
13 years ago
Al Sutton d73c84f9a0 Allow initial buffer size definition in TransformWriter, and use available() method to set it in htmlParser. In this situation a ByteArrayInputStream is used so the available() method gives a good size estimation and avoid the buffer needing to be continually grown
13 years ago
Al Sutton f02ea27b31 Added missing closure of ByteArrayInputSteam
13 years ago
orbiter 0796b54601 - some speed hacks for network image
13 years ago
orbiter f9216e388c - faster ping to clean up old peers faster
13 years ago
orbiter 35a9e8f307 - fixed network graphic
13 years ago
Al Sutton 8993cac4d8 Initial performance improvements
13 years ago
orbiter d9c066227a fix for npe
13 years ago
orbiter 8895d8c1cd removed unnecessary log entries
13 years ago
orbiter 550c881d80 remove more news (all older than one day) because they can be a performance problem if we have too many peers sending news
13 years ago
orbiter ebd840ebf6 - enhanced description on search front page
13 years ago
apfelmaennchen 77a080ced9 smaller fixes for YMarks
13 years ago
orbiter e22f8497c9 - tested the ARC methods
13 years ago
orbiter bc5df0eef5 updated ranking tables (fresh computation)
13 years ago
orbiter 5a55397f99 some last-minute performance hacks
13 years ago
apfelmaennchen dd1482aaf5 further update to YMarks
13 years ago
orbiter c9216d5adf fixed secondary remote search (the process that finds distributed join situations)
13 years ago
orbiter 64fd20b857 new default ranking profile
13 years ago
orbiter 0cf9ebc3b0 speed enhancements when parsing RWI rows (makes search slightly faster)
13 years ago
orbiter c9a0dbd25a added a security check
13 years ago
orbiter ee8b1d4de1 fixed unresolved pattern and unwanted local/global switch when using votes on search results
13 years ago
orbiter c584db991f creating a bookmark from the search results now works again .. with new YMarks
13 years ago
orbiter 1120f0c93c update to network graphics: slightly less crawling activity, slightly stronger color for query activity
13 years ago
orbiter 6cd27473f5 - better default values for caching and cache usage
13 years ago
orbiter 709013385a fix for language fix
13 years ago
orbiter 1019c36dad bug fixes and speed enhancements for search
13 years ago
orbiter 507c9d478d much better timing when search globally; less blocking; more results earlier!
13 years ago
orbiter 8e0b2c5832 fixed cluster search
13 years ago
orbiter c0c6e9e7a5 fix for bad language encoding
13 years ago
apfelmaennchen 564374d1fe - included YMarks in addition to old bookmarks in yacysearchitem.html; don't get confused by the old bookmark dialog, the ymark is automatically added silently beforehand.
13 years ago
orbiter 05f34a3fa7 added a full, complete, database insert, update and delete API for the tables.
13 years ago
lotus 3cc93325f0 temporary remove compare search from tray
13 years ago
orbiter c93f10417a add a bookmark automatically each time a new crawl is started
13 years ago
orbiter e4a82ddd8b produce a bookmark entry from every crawl start. these bookmarks are always private.
13 years ago
apfelmaennchen 6287c2b4a9 YMarks:
13 years ago
cominch 2236e01137 Minor correction to prevent useless comma at beginning of string, created from list
13 years ago
apfelmaennchen 5581be12fb YMarks:
13 years ago
orbiter 804e48888b smaller bug fixes for search behavior; should produce less unnecessary removals and an exact number of results as shown in counter
13 years ago
apfelmaennchen a3eebfdcba YMarks:
13 years ago
orbiter c50f8f9a06 code cleanup
13 years ago
orbiter 84c3fc9d97 local/global fixes in search, better abstraction
13 years ago
apfelmaennchen 4f95f72124 YMarks:
13 years ago
orbiter aa322bc6d0 fix
13 years ago
orbiter 97d1347adb added also a default accept field to robots.txt downloads
13 years ago
orbiter f183d3822c added a default accept header in http requests since some http fraud detection functions check that this header field exist
13 years ago
orbiter 06352b8d6b more logging
13 years ago
orbiter a99934226e more logging for debugging of robots.txt
13 years ago
orbiter 7a5841e061 fix for robot parser
13 years ago
orbiter 458c20ff72 fix for robot parser
13 years ago
orbiter 017a01714d - enhanced logging in robots.txt parser for remote debugging
13 years ago
apfelmaennchen a8dfe787ed - updated to jquery flexigrid 1.1
13 years ago
orbiter eb1c7c041d write info about robots.txt evaluation into getpageinfo_p.xml
13 years ago
apfelmaennchen abba31f02e - bugfix for correctly sorting ymarks
13 years ago
orbiter 3a15e58e28 - increased stability when opening the robots table
13 years ago
orbiter 775b44017e refactoring
13 years ago
orbiter e914a30099 fix for npe
13 years ago
apfelmaennchen 5f7dbe1c42 - some refactoring (ymarks)
13 years ago
orbiter 78ce3b13be typo
13 years ago
orbiter 85d6bf4ac4 fixed urls to media content during indexing
13 years ago
orbiter 0d858d48ec replaced String with StringBuilder in suggestion process
13 years ago
orbiter 3a807e10cf - added a cache for active crawl profiles to the crawl switchboard
13 years ago
orbiter 37e35f2741 normalization of url using urlencoding/decoding
13 years ago
orbiter e58438c01c - added a new retry connector for solr (for cases where solr responses are slow)
13 years ago
orbiter d8d9735b4f stability bugfix
13 years ago
orbiter c31564ef08 stability bugfixes
13 years ago
orbiter f121f4bb45 fix for link in Supporter and Suftipps page
13 years ago
orbiter 94eab08794 - updated opensearchdescription text and icon
13 years ago
orbiter 279482a76d fix for npe
13 years ago
orbiter 1b86d06d1e fix for http://bugs.yacy.net/view.php?id=62
13 years ago
orbiter 9e4875230f performance hacks
13 years ago
orbiter eb9c9edb01 enhanced table method (used by almost all yacy api interfaces)
13 years ago
orbiter 4ad9fc2bff new snippet strategy for search hits in metadata: show beginning of text instead of hit position
13 years ago
orbiter a9838f8b99 fix for http://bugs.yacy.net/view.php?id=59
13 years ago
hermens d3df03838a make sure myself-target is always inserted at its appropriate position
13 years ago
hermens c3e7efa846 added sender side prevention of rwi flooding as mentioned in SVN 7993
13 years ago
orbiter 5af9598bd1 enhanced exported row parsing during row import
13 years ago
orbiter 7598a9e26b fix for thread dump
13 years ago
orbiter 8eef8722d1 update to ThreadDump analysis: freerunner and thread state recognition
13 years ago
orbiter 1df43b137d another performance hack
13 years ago
orbiter 7df0643f0e performance hacks
13 years ago
orbiter a7df70221e refactoring
13 years ago
orbiter 1b45e33f04 added robots tag parser to solr scheme
13 years ago
orbiter cf4fd525ee added directDocByURL attribute in crawl profile
13 years ago
orbiter c61e4cfd78 - fix for incomplete clear() in balancer
13 years ago
orbiter 813f297a95 another performance hack: re-use of known host addresses for isLocal property; avoids look-up in local hash
13 years ago
orbiter 035ebfbf3b - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill)
13 years ago
orbiter b250e6466d implemented crawl restrictions for IP pattern and country lists
13 years ago
f1ori e207c41c8e * fix urlproxy for urls containing dolar signs
13 years ago
orbiter 57d5529a01 performance hacks
13 years ago
orbiter 5ad7f9612b added crawl settings for three new filters for each crawl:
13 years ago
orbiter 47a8c69745 added a new feature to MultiProtocolURIs to get the locale for each url:
13 years ago
orbiter 2c3161b4ac refactoring:
13 years ago
orbiter d2ea250d99 refactoring:
13 years ago