Commit Graph

18 Commits (0f3a3e34e10781426b5d491c509d901584ae7981)

Author SHA1 Message Date
orbiter 3f93a0cc8f redesign of remote proxy settings
15 years ago
orbiter 2126c03a62 - removed download-limit that can be given for the crawler for non-crawler download tasks. This was necessary because the same procedure was used for other downloads like for the download of dictionary files where a limit is not useful. The limit still stays for the indexer
15 years ago
orbiter 3300930fc5 - (almost) fixed FTP crawler
15 years ago
orbiter 26fafd85a5 - more refactoring
15 years ago
orbiter b79f4f062f refactoring of yacy documents and parsers: they depend now only on the kelondro classes
15 years ago
orbiter 5841ee83d3 refactoring
15 years ago
orbiter ce8dc575ca refactoring
15 years ago
orbiter c0e17de2fb - fixes for some problems with the new crawling/caching strategies
16 years ago
orbiter 634a01a9a4 replaced wget-requests with caching requests
16 years ago
orbiter 1d8d51075c refactoring:
16 years ago
orbiter 5bb8074150 removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency.
16 years ago
orbiter ca72ed7526 -removed superfluous crawl cache
16 years ago
orbiter dafffd0153 refactoring of parsers and document processing
16 years ago
orbiter 154bbc3364 code cleanup: call of static methods directly to the class
16 years ago
f1ori dd6b5005ff * fix missing charset handling in getpageinfo_p
16 years ago
orbiter c12bb8a6d0 - refactoring of the http client
16 years ago
orbiter 94110df85a moved logging partially to kelondro
16 years ago
orbiter b423d0a036 moved all servlets from htroot/xml to htroot/api
16 years ago