orbiter
08a12e9bb5
- removed dashed line from default skin (looks much better!)
...
- better timing when displaying results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4428 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
89169d54fd
fixed search result preparation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4427 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
acf771d5e1
- fixed bug with too much RAM in crawler queue
...
- fixed dir bug
- better calculation of TF for join
- better waiting-on-result logic
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4424 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a8a5df4a51
- more dublin core naming of page metadata
...
- better presentation of result counters in search results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4420 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
fa3b8f0ae1
fixed bug in remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4419 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7d875290b2
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4417 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9d693ee635
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4415 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0f5c4abaca
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4414 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
974fea7933
added term-frequency ranking
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4413 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1a296af6ff
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4412 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4a80902081
- added ViewProfile as rdf in foaf syntax
...
- added link to rdf and vCard version on html page
- can be seen on http://localhost:8080/ViewProfile.html?hash=localhash
- more generics
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4411 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
da8c850a25
disabled IO path optimization (seems to block other methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4405 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
d177ceb3b3
Fix for growing responseHeader[12].db when using proxyCacheLayout = hash
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4404 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
b1fae9b5af
fixed import Netscape Bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4401 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2485681002
added termination control for RotateIterator
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4399 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e2e7f065e9
minor fixes, some generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4398 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
15397298dc
- refactoring of indexControlRWIs: moved statics to own class; better Dublin Core naming
...
- fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=759&hilit=&p=4866#p4866
- some bugfixes in EcoTable according remove method
- switched more tables to Eco: crawl Profiles, htcache, seeddb, newsdb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4397 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
f3a9e9c542
added getFolderList() to bookmarksDB
...
added cleanTagsString() to bookmarksDB
added getFoldersString() to Bookmark
modified getTagsString() to exclude folderTags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4383 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
db25425893
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4382 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9e7cd4fdbb
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4380 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4e70dff8cf
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4379 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6dc679785f
- fixed bad sort behavior of kelondroRowSet, in this case: no sort at all!
...
see http://forum.yacy-websuche.de/viewtopic.php?p=4841#p4841
- some memory calculation enhancements in kelondroFlex and a little bit more logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4378 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0b4205eb5a
- fix double-deletion in eco tables
...
- changed behaviour of sort moment (not during a get)
- added some asserts in snippet cache for debugging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4375 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4ce6fab428
added special handling for doubles in eco tables after initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4370 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
002a109c4d
patch for http://forum.yacy-websuche.de/viewtopic.php?p=4597#p4597
...
(urls that have no protocol but start with www will be treated as http://www ...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4369 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
634430c48a
- more logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4368 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d372a78aef
some fixes to bring back lulabads peer..
...
see also: http://forum.yacy-websuche.de/viewtopic.php?p=4772#p4772
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4366 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
f4799c2334
*) removed since I decided to turn this into a project of it's own using Perl to gather n-gram data which YaCy will be able to use
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4365 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4ffbcd54a4
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=754
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4358 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e81bced2bd
reorganized the code and adjusted getTagIterator() to suit folders
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4357 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
85dc62c16f
refactoring: more dublin core - compliant naming
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4354 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
efd0b8371a
- added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser
...
- refactoring of plasmaParserDocument to use Dublin Core - compatible property names
- redesign of url handling in parser and condenser (less String-to-yacyURL conversion)
- more generics
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4352 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
cfd4fecd12
*) blanks in paths for restart and update script are replaced by backslash+blank now (see http://forum.yacy-websuche.de/viewtopic.php?t=745 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4351 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f945ee21d2
some security additions, keep maximum byte[] size to 2^27
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4350 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2f3b2f3481
- extended dbtest for comparisment tests
...
- added initial space option for eco tables
- used initial space value in initialization of collectionIndex, this should avoid OOM failures" /Volumes/Magneto/dev/workspace/trunk/source/dbtest.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroCollectionIndex.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroDyn.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroEcoTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroRow.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroSplitTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlBalancer.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlStacker.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlZURL.java
- added index consistency check (checks for double-occurrences of primary keys in file)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4349 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9eb746863d
interface enhancements for eco records memory statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4348 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9abc927645
to fix inconsistencies in collection index, a double reference reporting mechanism has been implemented
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4347 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
58a1f518f8
fixed some problems with eco tables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4346 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d4d07802ac
better RAM protection using eco tables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4345 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f4e9ff6ce9
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4343 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
cbefc651ac
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4342 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
45339c3db5
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4341 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
94f21d9403
activated new kelondroEcoTable file structure.
...
This data structure replaces almost all files in the PLASMA directory
also the collection.index and the LURL-db will be created as Eco-DB, if it does not exist before
existing Flex-databases will be used as they are (the is no data lost)
If you want to force the creation of a Eco-collection.index, simply delete the old index.
The Eco file system will only be used if there is enough memory.
The collection.index RAM limit is 200MB, if you have less, a flex-Table is createt.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4340 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a0f7f2faad
some more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4338 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dc26d6262b
- removed write buffer from kelondroCache (was never used because buggy; will now be replaced by new EcoBuffer)
...
- added new data structure 'eco' for an index file that should use only 50% of write-IO compared to kelondroFlex
The new eco index is not used yet, but already successfully tested with the collectionIndex
The main purpose is to replace the kelondroFlex at every point when enough RAM is available.
Othervise, the kelondroFlex stays as option in case of low memory (which then can even use a file-index)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4337 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dbdec0f4d3
another fix for the "too many processes in loader queue, dismissed" - problem:
...
this was probably caused by http-forward cases; which are cases when urls from the loader queue change
and it was not possible to remove the old urls from the queue because they had been based on url hashes.
The queue is now again stored using the entry.hashCode, which does not change ieven if the url changes.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4332 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b806a6af8b
renamed kelondroEcoRecords to kelondroFullRecords (the "Eco"-name will be used for something else)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4331 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
065ba2d60f
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=719&hilit=
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4330 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f3f02b08ec
no distinction between standard and pro releases in auto-updater
...
this did not work in 0.56 main (but should).
Therefore it will be necessery to provide a hand-made 'virtual pro' (just renaming) release 0.57
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4326 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
3cab85158c
update for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4325 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago