orbiter
b79f4f062f
refactoring of yacy documents and parsers: they depend now only on the kelondro classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
f1ori
34c71b22e8
fix and enable parser unit tests (tested with eclipse)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6419 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ce8dc575ca
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6398 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bea3b99aff
moved table and util classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ce7924d712
better concurrency for rwi entry parsing during search processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6273 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
72ac5bd80f
refactoring of search process.
...
this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6260 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
f1ori
d515bc11e2
added ooxmlparser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6256 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
f1ori
8c1b02af04
* fix warning in testcase
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6255 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
f1ori
67da20647f
* add new odf parser based on sax-xml-parser
...
* remove odf_utils-jar
* test metadata in ParserTest
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6231 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
06557485f5
* added parser unittest!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6229 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
69dfd03985
reactivate unittests
...
* fix old tests
* add buildtarget "ant test"
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6228 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
daf0f74361
joined anomic.net.URL, plasmaURL and url hash computation:
...
search profiling showed, that a major amount of time is wasted by computing url hashes. The computation does an intranet-check, which needs a DNS lookup. This caused that each urlhash computation needed 100-200 milliseconds, which caused remote searches to delay at least 1 second more that necessary. The solution to this problem is to attach a URL hash to the URL data structure, because that means that the url hash value can be filled after retrieval of the URL from the database. The redesign of the url/urlhash management caused a major redesign of many parts of the software. Since some parts had been decided to be given up they had been removed during this change to avoid unnecessary maintenance of unused code.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4074 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
theli
2399ed817c
*) robots.txt parser now extracts the sitemap-URL (will be used later)
...
*) some javadoc added
*) junit testclass for robots.txt parser added
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3602 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
1b7fda12ee
*) SOAP: separate function to get the active/passive/potential peer list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3526 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
a1d68fe092
- use .class rather than Class.forName for classes in class-path
...
- added Bost's patch for Diff.findDiagonale() from: http://www.yacy-forum.de//files/patch_685.txt
- fixed minor bugs in Blog
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3416 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d25caa07bf
redesigned some parts of http authentication
...
added another access check for peer hops
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3340 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
eb20ec3837
*) soap-service: adding function to check if a specific url is blacklisted
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3014 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
5c0669429e
*) soap: adding function to query the peer list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2968 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
203f2bde9a
*) adding function to query the pause/resume state of the crawling queues
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2958 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
6d3a130878
*) bugfix needed because of db refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2957 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
892b9f2fc4
*) additional soap function to query peer status
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2920 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
bd3710a974
*) new xml template to view peer profile as xml
...
*) bugfix for wrong profile display (some fields where displayed twice)
*) new soap functions to get and set peer profile
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2919 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
d1afe1ce6b
*) adding xml template to get the message list as xml
...
*) Bugfix in client stub jar generation (too many files where added)
*) new soap service to manage peer messages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2918 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
f37e2041e8
*) adding soap function to import yacy bookmarks from xml or html (transfered via soap attachments)
...
*) soapHandler: code cleanup for service deployment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2915 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
4a3ec63e34
*) new soap service to manage yacy bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2906 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
5e57e0814d
*) new soap function to display log
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2902 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
c7bea4addb
*) soap api
...
- adding function to get and set message forwarding
- adding new testclass
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2878 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
532c23b5c7
*) soap handler
...
- better errorhandling
- adding support for outgoing transfer- and content-encoding
- avoid holding outgoing messages into memory before sending them
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2872 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
7299dc30e3
*) new soap service to manage the yacy file-share
...
- upload / download files (as soap attachment)
- create directory
- receive directory listing
- delete files / directories
- change file comment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2857 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
9e8942a064
*) adding method to implement blacklist from file
...
- file transfer is done via soap attachments (see BlaclistSerivceTest for details)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2855 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
d38ef0493d
*) be more tolerant against missing ports in url
...
"http://yacy.net:/ " is now interpreted as "http://yacy.net/ "
See: http://www.yacy-forum.de/viewtopic.php?p=27102
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2852 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
cfe54fedc7
*) Bugfix for resolveBackpath problem with tailing /..
...
*) Junit testclass for resolveBackpath testing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2850 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
ac13fa763a
*) bugfix for blacklist remove (blacklist was not informed about remove)
...
*) adding new soap service class for blacklist management
*) new junit class to test soap blacklist service
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2841 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
3e0516446b
*) new soap function to get the current queue status
...
*) new junit testclass to test soap statusService
*) refactoring of admin service (usage of constants instead of strings)
*) libraries upgraded to newer version + adding missing dependency
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2836 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
92f774edd1
*) Better charset encoding detection
...
*) New testclass for charset encoding detection tests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2808 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
eedb898c45
*) adding date parsing test routine to determine if we have a date-parsing bug
...
See: http://www.yacy-forum.de/viewtopic.php?t=3007
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2806 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago