Commit Graph

4679 Commits (fe72d6e5490d2f148010e2b5ae36968bdb5a598e)

Author SHA1 Message Date
sixcooler fe72d6e549 Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
apfelmaennchen a3eebfdcba YMarks:
14 years ago
orbiter c50f8f9a06 code cleanup
14 years ago
sixcooler b51c3fade4 Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
apfelmaennchen 4f95f72124 YMarks:
14 years ago
sixcooler 29c2289b5c Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
sixcooler 605bc4c10e Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
orbiter aa322bc6d0 fix
14 years ago
orbiter 97d1347adb added also a default accept field to robots.txt downloads
14 years ago
orbiter f183d3822c added a default accept header in http requests since some http fraud detection functions check that this header field exist
14 years ago
orbiter 06352b8d6b more logging
14 years ago
orbiter a99934226e more logging for debugging of robots.txt
14 years ago
orbiter 7a5841e061 fix for robot parser
14 years ago
orbiter 458c20ff72 fix for robot parser
14 years ago
sixcooler e7dedc56f2 Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
orbiter 017a01714d - enhanced logging in robots.txt parser for remote debugging
14 years ago
sixcooler 7545822db5 Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
apfelmaennchen a8dfe787ed - updated to jquery flexigrid 1.1
14 years ago
sixcooler 0aa5e134ea Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
orbiter eb1c7c041d write info about robots.txt evaluation into getpageinfo_p.xml
14 years ago
apfelmaennchen abba31f02e - bugfix for correctly sorting ymarks
14 years ago
sixcooler 3b70ff7046 Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
orbiter 775b44017e refactoring
14 years ago
sixcooler db5ef90b0f Merge branch 'master' of https://git.gitorious.org/yacy/rc1.git
14 years ago
sixcooler f522f61af0 clean offline copy of URL Tables
14 years ago
apfelmaennchen 5f7dbe1c42 - some refactoring (ymarks)
14 years ago
orbiter 78ce3b13be typo
14 years ago
orbiter 85d6bf4ac4 fixed urls to media content during indexing
14 years ago
orbiter 0d858d48ec replaced String with StringBuilder in suggestion process
14 years ago
orbiter 3a807e10cf - added a cache for active crawl profiles to the crawl switchboard
14 years ago
orbiter 37e35f2741 normalization of url using urlencoding/decoding
14 years ago
orbiter 1b86d06d1e fix for http://bugs.yacy.net/view.php?id=62
14 years ago
orbiter 9e4875230f performance hacks
14 years ago
orbiter a9838f8b99 fix for http://bugs.yacy.net/view.php?id=59
14 years ago
orbiter a7df70221e refactoring
14 years ago
orbiter cf4fd525ee added directDocByURL attribute in crawl profile
14 years ago
orbiter c61e4cfd78 - fix for incomplete clear() in balancer
14 years ago
orbiter 813f297a95 another performance hack: re-use of known host addresses for isLocal property; avoids look-up in local hash
14 years ago
orbiter 035ebfbf3b - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill)
14 years ago
orbiter b250e6466d implemented crawl restrictions for IP pattern and country lists
14 years ago
f1ori e207c41c8e * fix urlproxy for urls containing dolar signs
14 years ago
orbiter 5ad7f9612b added crawl settings for three new filters for each crawl:
14 years ago
orbiter d2ea250d99 refactoring:
14 years ago
low012 42b5f09f68 *) this should fix a bug in snippet creation (also cleaned up a little bit)
14 years ago
orbiter 6b22865dbc - removed some warinings
14 years ago
orbiter 0c6d95e57b - more tolerance against failure of table opening
14 years ago
orbiter 4f31869c5a enhanced search result timing
14 years ago
orbiter 6b02b696b0 - add number of search results to end of rss and json output to reflect latest status of retrieval
14 years ago
f1ori 87e6abd168 * fix urls containing a port number in urlproxy
14 years ago
f1ori 97045022fa * pass cookies to Server Side Includes
14 years ago