Commit Graph

120 Commits (effbf35f57fc6ae10d46eaabdcdff861b5ecc31f)

Author SHA1 Message Date
orbiter 416c126815 fix for a profile = null problem and new monitor in crawl queue
20 years ago
theli 7fe8784231 *) URLs pointing to a server having a private ip addess will not be indexed anymore
20 years ago
theli f8ad65eae1 *) First trial implementation of robots.txt support
20 years ago
borg-0300 8cd6a52dd0 Convention
20 years ago
borg-0300 da9c6857fb *) changed a misunderstand, no BUG ;)
20 years ago
theli 578f36ae18 *) Speedup of indexer. Proxy files will not be enqueued by the cachemanager
20 years ago
theli 1219ef99f0 *) Bugfix for NullpointerException in yacyDebugMode Init
20 years ago
theli 6c722706b7 *) Moving yacyDebugMode intialization to switchboard
20 years ago
theli 4e07828807 *) httpdProxyHandler.java
20 years ago
theli a47f9238fe *) Blacklist is now also used by the crawler
20 years ago
theli 732a107160 *) Bugfix for "-UNRESOLVED_PATTERN-" Bug on IndexCreateWWWLocalQueue_p.html and "urlEntry.url() == null" Bug
20 years ago
theli 0471019606 *) IndexCreateIndexingQueue_p.html now also shows indexing jobs that are currently in process
20 years ago
theli 48aaf703cc *) Adding additional logging output to detect crawling problems
20 years ago
theli 4fd5b95b1f *) Renaming Logger function names to reflect the proper Java Logging API Loglevels
20 years ago
theli 6adf8a4bde *) Renaming Logger function names to reflect the proper Java Logging API Loglevels
20 years ago
borg-0300 bf14e6def5 *) proxyCache, proxyCacheSize can be changed under 'Proxy Indexing'
20 years ago
theli 2a081c9ee5 *) Adding additional logging message for "NURL.entry() == null" Bug
20 years ago
theli b70de495a0 *) Remembering Crawler-isPaused setting
20 years ago
theli e569a84dc0 *) Using the same configuration settings for all indexing threads on server Startup
20 years ago
theli 17be77a468 *) Bugfix for "Crawler data will not be removed from htcache if content parsing failed"
20 years ago
allo eb6365c069 local Bootstrapping bug.
20 years ago
theli 330eae7cf3 *) Normalizing CrawlerStartURL now before crawling is started
20 years ago
theli d4a045d7b1 *) Trying to solve "de.anomic.plasma.plasmaSwitchboard.deQueue': null" Bug
20 years ago
orbiter 25f632dbd9 more DHT bugfixes and better logging of DHT effects
20 years ago
orbiter 5cb00889d9 enhancements to dht selection, search and search presentation
20 years ago
orbiter ba0a486328 moved printStackTrace() to logging
20 years ago
orbiter cd10370992 several bugfixes and dht selection / logging improvement
20 years ago
orbiter c8a7a85ce2 fix for http://www.yacy-forum.de/viewtopic.php?p=7384#7384
20 years ago
orbiter 7db543a9fa fixes for several dht misbehaviours
20 years ago
orbiter 5716f8521d bug fixes for word ordering and dht index selection
20 years ago
orbiter f5259f29e8 word cache behaviour fix and other fixes
20 years ago
orbiter 2c234e1b82 better log output for search result
20 years ago
orbiter 248c24b60a intermission-feature usage in case of local and remote search
20 years ago
theli 865b9490a2 *) Making DHT Transfer while Crawling configurable
20 years ago
orbiter 2d8557cb10 minor changes
20 years ago
orbiter 91163db52e fix for more time-related problems in proxy
20 years ago
orbiter 40da910f41 bugfixes and automatic news-cleanup
20 years ago
theli 228b04b499 *) Bugfix for "wrong seed-upload timestamp" problem
20 years ago
theli 470839a16a *) Crawler/Session pool settings will now be stored properly into configfile
20 years ago
orbiter 1022fbeb65 many YaCyNews fixes
20 years ago
orbiter 13abd8b6e7 added news-creation at crawl start
20 years ago
orbiter cdbbfd50fb fixed bad remote crawl behavior
20 years ago
orbiter 81e564edb8 faster crawl profile list cleanup
20 years ago
orbiter ad90f0ad13 activated RWI distribution to DHT for senior peers (default redundancy 3), necessary now for network growth
20 years ago
orbiter b9d18d40cb configuration of proxy idle time in performance menue
20 years ago
orbiter c64970fa47 re-implemented proxy-busy-check and fixed some other things
20 years ago
orbiter b73557ed2d better assortment monitoring and enhanced profile menue
20 years ago
orbiter 9f505af7aa preparations for bulk remote crawls
20 years ago
orbiter 51962d55bf added 'PPM', page-per-minute statistics
20 years ago
orbiter 159f795f65 bugfix (null pointer exception in assortments)
20 years ago