Commit Graph

3101 Commits (474e29ce4a831550d06a4fd3d6354fd41e32f121)

Author SHA1 Message Date
orbiter 415b92bb07 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1148&hilit=&p=7711#p7711
17 years ago
apfelmaennchen 2113672bf2 small fix on tag comporator functions
17 years ago
orbiter dd75b3cabc - patch for bad profiles
17 years ago
danielr 2080ff72b7 ftpc fix for npe
17 years ago
lotus e021278bf0 unescape link display in search results
17 years ago
danielr 3f1721b827 informational comment
17 years ago
danielr 04a51b775a changed .org/.net back to America
17 years ago
danielr 74b1a60043 fixed "java.lang.NoClassDefFoundError: org/a"
17 years ago
danielr f44d5d302b updated TLDs
17 years ago
hermens 5bfc02ccfb Repair publishThread
17 years ago
orbiter f42c8cf69c updated terminal and dynamic webstructure applet: can now change when crawl is running
17 years ago
orbiter 906c144799 - design update to new terminal and rssTerminal
17 years ago
orbiter 7ec01d444a fix for npe
17 years ago
orbiter ad0f905124 fix for npe in crawler
17 years ago
danielr ae03a54d23 pdfParser: updated lib, fixed ClassNotFoundException: CMSError
17 years ago
danielr 0d3808bd9e minor refactoring
17 years ago
orbiter 719f5defb1 updated some grafics at new terminal_p
17 years ago
lotus 9bc56a9edc xss protection
17 years ago
orbiter b32736762c enhanced rssTerminal
17 years ago
orbiter fbb712c669 refactoring:
17 years ago
orbiter 1689030ee8 refactoring: moved all crawler classes into their own package
17 years ago
orbiter d2ba1fd2ab major step forward to network switching (target is easy switch to intranet or other networks .. and back)
17 years ago
danielr d70a472460 added file for previous commit
17 years ago
danielr d32fe84472 added default User-Agent
17 years ago
danielr 8c5f062e0b corrected YaCy version in HTTP User-Agent
17 years ago
danielr d7b21bc90c re-added gzip POST for transferRWI/URL (HTTP/1.1 compliant)
17 years ago
low012 8d83febb95 *) BlacklistCleaner_p.java reports exception to log instead of System.err
17 years ago
danielr d4bce6affd refactoring (initialized static fields, removed empty if/else, serialized some fields in serializable classes)
17 years ago
danielr be2c9c07ff escape some unescaped characers in URLs (fixes problems with proxy)
17 years ago
orbiter d0678f7ab9 refactoring as result of
17 years ago
orbiter 483e9a2066 - shifted tld recognition methods from yacyURL to serverDomains
17 years ago
orbiter a3df23659c re-implementation of charset checking
17 years ago
orbiter 75a1702133 - fix for ConcurrentModificationException during shutdown
17 years ago
orbiter 32b5b057b9 - modified, simplified old kelondroHTCache object; I believe it should be replaced by something completely new
17 years ago
danielr ec84a52adb change for problem with NPE (seen as "PROXY Unknown Error while processing request")
17 years ago
orbiter 5813cc149f fix for bad rssTerminal behavior
17 years ago
orbiter 88216c1f1f fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1103&hilit=&p=7362#p7362
17 years ago
orbiter d0b893523e - protection against RAM overflow caused by new peer rss news
17 years ago
orbiter 685794e7e7 fix for parser/encoding Exception
17 years ago
orbiter cf042e6957 reverted change by mistake in yacyVersion
17 years ago
orbiter 9935e83c86 added new news window into the status page. At this moment it is just a test.
17 years ago
orbiter bac38cfa18 added very rudimentary peer news as rss feed. An example can be retrieved with
17 years ago
orbiter 6495227ad6 the class rssReader is replaced by RSSReader, RSSFeed and RSSMessage
17 years ago
orbiter 724bbdf9b2 refactoring of RSS reader
17 years ago
orbiter b9a2a2d287 more search performance hacks
17 years ago
orbiter ff755fb858 small corrections and enhancements after search timing profiling
17 years ago
orbiter e024e3b9cf added new default profiles to distinguish snippet fetch for local and global search
17 years ago
danielr 9b03310f8a bin jetzt wach :/
17 years ago
danielr 7bd8601f04 delete old releases compatible with java 1.5 ;)
17 years ago
danielr da386a1924 fixed deleteOldDownloads if there are no downloads
17 years ago
danielr 21418a22a3 removed DEBUG output
17 years ago
danielr 79a3edeeef deleting downloaded releases after x days (default 30)
17 years ago
danielr 763f9d4f5d serverCore: setting timeout for new connection before SSLDetect
17 years ago
orbiter 1995faef8d - refactoring of Colage back-end: move to plasma package
17 years ago
orbiter d7e89c2aca fixed near-deadlock situation when deleting crawl profiles
17 years ago
orbiter 5e3ce46339 - better logging when rejecting a url because it is not in declared domain
17 years ago
danielr 48ffd61e6a changed "patched wrong" to warning, so it goes to the logfile
17 years ago
orbiter 2f629d20a7 - tried to fix the '4217666-problem'
17 years ago
orbiter 512f48e7d6 - removed unused methods
17 years ago
orbiter 3c76342619 - added servlet to configure the search page greeting line
17 years ago
danielr d1ee231866 HTTPC close more unused connections
17 years ago
danielr 181796cffb - HTTPC ConnectionInfo entfernen bei Exceptions, unnötigen Code entfernt
17 years ago
orbiter 04c1226c80 added/fixed missing integrity-test else-case during deploy in case that we update with a tar file
17 years ago
orbiter 45ae3da7e7 another patch to prevent NPE in EcoTable
17 years ago
orbiter cb93ded5c6 applied configuration path patch
17 years ago
danielr 96e39b297a reduced StackTraces (by connect timed out)
17 years ago
orbiter 93376acdca fixed a bad chunkcache limit check which could have caused ArrayIndexOutOfBoundsExceptions
17 years ago
orbiter 1cab240198 patch for possible NPE in EcoTable iterator
17 years ago
orbiter 9a32a4c328 fixed concurrentModificationException during hello-process
17 years ago
danielr 64c33e717f catched ConcurrentModificationException in ConnectionInfo.cleanUp so cleanUp is not interrupted
17 years ago
danielr d8677ba611 fixed ConcurrentModificationException in HttpConnectionInfos
17 years ago
orbiter c7021c14bb patch for ArrayIndexOutOfBoundsException in BMP parser
17 years ago
orbiter 8dd35f74c8 fixed redirect problem (does not work for POST)
17 years ago
orbiter 8313d58ae7 - integrated the collage into the Web Visualization menu
17 years ago
danielr 2617f4dcdb Connections_p.html: better formatting and remove very old entries
17 years ago
orbiter 82bf9ac1c8 - added Collage servlet from datengrab and modified it:
17 years ago
danielr 959f448e5f - disabled redirects in proxy (so client sees real path)
17 years ago
orbiter 8fe39ebd74 -fixed file transmission with POST. The only usage was in ranking transmission, therefore:
17 years ago
orbiter 82a9861779 fix for last commit
17 years ago
orbiter 5d1fbb25e7 fix for bad deploy:
17 years ago
orbiter 202a3adb3e refactoring of HttpClient Writer processes
17 years ago
danielr 8aa9fd8f24 HTTPC with only 1 retry
17 years ago
orbiter 444dce7e81 more performance hacks
17 years ago
orbiter 2c2dcd12a2 - enhanced performance of Eco-Tables: less time-consuming size() - operations
17 years ago
orbiter e356625b22 - refacotring of stream copy handling to support time-consuming operations
17 years ago
danielr f01c50cf8d Proxy logging error (first step to resolution!?)
17 years ago
orbiter c3342e1178 - removed class with only one static method
17 years ago
orbiter f97971b63b fixed NPE problems doing a shutdown from command-line
17 years ago
danielr 7a35126e91 http timeouts von alten httpc wieder gesetzt
17 years ago
orbiter 2c1c3bb6eb - some refactoring (sorry Daniel, hab in deinem Code rumgewütet)
17 years ago
danielr d96e2badc7 - fixed POST in proxy
17 years ago
orbiter 14404d31a8 - enhanced performance graph (more info)
17 years ago
orbiter 696b8ee3f5 fix for http://forum.yacy-websuche.de/viewtopic.php?p=6806#p6806
17 years ago
danielr 94d3d3a86f fixed Proxy (for GET, POST still does not work!)
17 years ago
danielr 081ed1d3ec HTTPLoader: reduced stackTraces
17 years ago
danielr 8b2efb6f8c fixed garbage in HTCACHE
17 years ago
orbiter 225f9fd429 various fixes
17 years ago
orbiter 6e36c156e8 added more logging to EcoFS
17 years ago
danielr fb541f9162 HTTPC: default timeout half-hour
17 years ago
danielr a94f6cdca4 HTTPC: allowed self-signed certs
17 years ago
danielr ab330cfdca Network.html: removed ; from location
17 years ago
orbiter 319144f4b2 fix for outofbounds-excception in EcoFS chunk iterator
17 years ago
orbiter a9cf6cf2f4 generalization of index container-heap class.
17 years ago
orbiter f099061944 protection against bad dht-flush word selection
17 years ago
orbiter 5e4fddc1e6 more logging for new EcoFS.ChunkIterator to find bug for
17 years ago
orbiter 117ae78001 speed enhancement for reading of eco-table indexes
17 years ago
danielr 7c149a4ee8 - undo less 'binary data found'
17 years ago
danielr 96cce8bed9 reduced 'Binary data found' errors
17 years ago
danielr 2aef1414f5 removed test (in yacy.init)
17 years ago
danielr 5c3c1fdf41 replaced httpc with Apache Jakarta Commons HttpClient (includes some refactoring ;)
17 years ago
orbiter daa04f5db9 added additional check in file handler to prevent that url attacks are hidden in url path encodings
17 years ago
orbiter 783a4c9edb strong speed enhancements for the index cache dump and restore:
17 years ago
orbiter 442204a1c8 fix for concurrentModificationException
17 years ago
orbiter d2f4926951 - more logging for balancer to get a hint where the problem is
17 years ago
orbiter 20dadba426 - added a deadlock prevention function in cache flushing
17 years ago
orbiter 764a40e37d speed enhancements for crawler and url retrieval (affects also search speed)
17 years ago
orbiter 3ce3a4a3a1 added stub for new index container heap data structure (purpose: index folding)
17 years ago
orbiter 2c34038912 addition/correction to last commit: usage of concurrent-classes
17 years ago
orbiter b2150057d2 removed unnecessary cleanup method
17 years ago
lulabad c4c0d54b22 * added regex extended blacklistengine
17 years ago
orbiter 368593e449 enhanced the concurrency handling of indexing process (better queue size control, better data concept, better shutdown behavior)
17 years ago
orbiter be58135b3e possible fix for deadlock in search execution
17 years ago
orbiter 0241d070bc added concurrency to indexing process:
17 years ago
lulabad 9fb5d661f2 added my Blacklistengines
17 years ago
orbiter bca87f1e38 - refactoring of serverThreads: renaming to distinguish busy-threads and blocking-threads
17 years ago
orbiter 968c775025 - preparation of parsing/indexing queue for concurrent execution
17 years ago
orbiter 9b0e20fb06 next refactoring step in document indexing to prepare concurrency environment for document parsing
17 years ago
orbiter 7f9f639d20 - refactoring and abstraction of index reference (urls) handling: blacklisting is part of reference filtering
17 years ago
orbiter d6050b9ffb - separated the LURL data storage and Crawl result stack for process supervision.
17 years ago
orbiter 8d6a13bc07 refactoring of parsing-condensing-indexing process:
17 years ago
orbiter d3b06913ec protection against seed-db failure during enumeration
17 years ago
orbiter 5aa96dbc36 fix for shutdown configuration
17 years ago
orbiter 93633abed8 - removed some debugging code from search process - should speed up now
17 years ago
orbiter fba46c51d7 fixed non-termination bug in qsort
17 years ago
orbiter 541b817502 refactoring of switchboard queueing
17 years ago
orbiter fc94fbe224 another improvement to the collection sorting
17 years ago
orbiter 11270d450e better quicksort-pivot computation: 30% faster (measured with test program)
17 years ago
orbiter 3e44293f07 - fixed a problem with thread pools in row collection
17 years ago
danielr e43051b125 - fixed Threaddump output (html-escaped ie. <init>)
17 years ago
orbiter 433ff855f7 - fixed another concurrency problem in collection sorting
17 years ago
orbiter 19286fa2d1 tried to fix seed2.old.db-problem
17 years ago
orbiter f3996e63b8 tried to fix more deadlocks:
17 years ago
danielr 7008a218b3 avoid ConcurrentModificationException in plasmaCrawlerQueues
17 years ago
orbiter 7150b463ff changed handling of default values and database paths:
17 years ago
lulabad 25f5035f23 typo
17 years ago
orbiter 7fd094fcbe small bug in ftpc: did cot compile in Java 1.5
17 years ago
danielr f51bad8ae5 FTP:
17 years ago
danielr 820641938e ftpc: fixed date parsing, some refactoring
17 years ago
orbiter 4c584dff87 disabled soLinger to prevent that too many connections stay open (it's a TEST!)
17 years ago
orbiter 9c989fe5f7 fixed deadlock
17 years ago