orbiter
8fd5e52f04
added basket icons and experimental gif animation class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4485 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
94e256e13b
* removed single Blogview, now links direct to BlogComments.html
...
* some other small changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4483 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ff5969901c
modified dir servlet to cooperate with intranet indexing from the own HTDOCS repository:
...
- removed md5 file generation (spoils the won repository)
- removed comments in file share (was never used)
- moved dir list comparator to other place (maybe solves problem, lets see)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4481 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
00f5f917de
- more refactoring to blog
...
- fixed moderate comment bug. see http://forum.yacy-websuche.de/viewtopic.php?f=9&t=860
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4478 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f890b039ee
experiments wit openstreetmaps
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4477 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7f445f34a6
bitte die Java 5 - typischen Warnings einschalten!
...
(unboxed-Fehler wies auf Programmfehler hin und Typangabe fehlte)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4476 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
c1b9a03304
* some refactoring to Blog
...
* changed default sort order to reverse (newest first)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4475 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
766a04bc06
fixed sort problem in Blog. see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=639
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4474 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
bfe171e693
Small change (generics)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4473 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
2589290ded
better ping
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4472 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
dae9053b21
BUGFIX
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4464 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
77ba446332
seedDB helpers update/cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4461 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
dd215e7f6b
NPE fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4460 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bd63999801
- faster search: using different data structures that avoid multiplr calculations
...
- no more table copy for error-eco table
- optional table copy for lurl-entries
- more abstractions (less single constant strings)
- better logging (using host names instead of ips)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4459 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
8358652fa9
some small changes to blog
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4457 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
159aaf8889
re-introduced global search limitation when index receive is switched off
...
this was necessary because othervise robinson peers did also global searches, which cannot be a wanted effect
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4456 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
a9c4e9c309
Small change (ping)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4453 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
9ab6ad8b73
more seedDB helpers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4452 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
6a85764e1a
Second bugfix for numberbug in Blog.
...
This update fix automatic existing blogentrys.
A backup is not needed but almost a good idea ;)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4451 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
efd5807a7c
- some renaming of variables to support DC
...
- initial 120mb RAM for fresh peers
- release 0.57
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4445 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lulabad
40a0591942
Fixed numberbug in Blog, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=639 . This wont fix existing Blogentrys (comes later).
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4443 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
141db7ba48
there is less RAM needed for eco table (its just a security-plus for RAM check)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4442 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
249d61759a
fix for false RAM table activation in EcoTables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4441 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ff6b69b37e
fix for NPE in access tracker
...
fix for NPE in word index
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4439 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3c7b94c119
- fix for online caution delay settings, see
...
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=738&p=4723#p4723
- removed remote search limitation for non-dht-peers according to discussion in
http://forum.yacy-websuche.de/viewtopic.php?f=15&t=793&hilit=&p=5277#p5277
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4438 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f35a3794e0
auto-healing (deletion) of bad peer addresses during start-up
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4437 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
42c1e11f2b
added another link double-check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4434 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a5d388bfff
fix for HTCache organisation that may have caused unlimited grow of the cache
...
appeared only for tree-caches
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4433 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
96c5e6acc7
added a double-check for search results
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4432 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a1e9e6e2e6
fix for search result page navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4431 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7404256997
- no more search time-out!
...
- fixed a bug with last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4430 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
cd3e0d6f03
tried to fix another eco bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4429 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
08a12e9bb5
- removed dashed line from default skin (looks much better!)
...
- better timing when displaying results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4428 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
89169d54fd
fixed search result preparation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4427 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
acf771d5e1
- fixed bug with too much RAM in crawler queue
...
- fixed dir bug
- better calculation of TF for join
- better waiting-on-result logic
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4424 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a8a5df4a51
- more dublin core naming of page metadata
...
- better presentation of result counters in search results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4420 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
fa3b8f0ae1
fixed bug in remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4419 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7d875290b2
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4417 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9d693ee635
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4415 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0f5c4abaca
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4414 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
974fea7933
added term-frequency ranking
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4413 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1a296af6ff
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4412 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4a80902081
- added ViewProfile as rdf in foaf syntax
...
- added link to rdf and vCard version on html page
- can be seen on http://localhost:8080/ViewProfile.html?hash=localhash
- more generics
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4411 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
da8c850a25
disabled IO path optimization (seems to block other methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4405 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
d177ceb3b3
Fix for growing responseHeader[12].db when using proxyCacheLayout = hash
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4404 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
b1fae9b5af
fixed import Netscape Bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4401 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2485681002
added termination control for RotateIterator
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4399 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e2e7f065e9
minor fixes, some generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4398 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
15397298dc
- refactoring of indexControlRWIs: moved statics to own class; better Dublin Core naming
...
- fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=759&hilit=&p=4866#p4866
- some bugfixes in EcoTable according remove method
- switched more tables to Eco: crawl Profiles, htcache, seeddb, newsdb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4397 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
f3a9e9c542
added getFolderList() to bookmarksDB
...
added cleanTagsString() to bookmarksDB
added getFoldersString() to Bookmark
modified getTagsString() to exclude folderTags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4383 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
db25425893
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4382 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9e7cd4fdbb
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4380 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4e70dff8cf
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4379 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6dc679785f
- fixed bad sort behavior of kelondroRowSet, in this case: no sort at all!
...
see http://forum.yacy-websuche.de/viewtopic.php?p=4841#p4841
- some memory calculation enhancements in kelondroFlex and a little bit more logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4378 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0b4205eb5a
- fix double-deletion in eco tables
...
- changed behaviour of sort moment (not during a get)
- added some asserts in snippet cache for debugging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4375 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4ce6fab428
added special handling for doubles in eco tables after initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4370 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
002a109c4d
patch for http://forum.yacy-websuche.de/viewtopic.php?p=4597#p4597
...
(urls that have no protocol but start with www will be treated as http://www ...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4369 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
634430c48a
- more logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4368 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d372a78aef
some fixes to bring back lulabads peer..
...
see also: http://forum.yacy-websuche.de/viewtopic.php?p=4772#p4772
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4366 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
f4799c2334
*) removed since I decided to turn this into a project of it's own using Perl to gather n-gram data which YaCy will be able to use
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4365 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4ffbcd54a4
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=754
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4358 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e81bced2bd
reorganized the code and adjusted getTagIterator() to suit folders
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4357 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
85dc62c16f
refactoring: more dublin core - compliant naming
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4354 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
efd0b8371a
- added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser
...
- refactoring of plasmaParserDocument to use Dublin Core - compatible property names
- redesign of url handling in parser and condenser (less String-to-yacyURL conversion)
- more generics
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4352 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
cfd4fecd12
*) blanks in paths for restart and update script are replaced by backslash+blank now (see http://forum.yacy-websuche.de/viewtopic.php?t=745 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4351 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f945ee21d2
some security additions, keep maximum byte[] size to 2^27
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4350 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2f3b2f3481
- extended dbtest for comparisment tests
...
- added initial space option for eco tables
- used initial space value in initialization of collectionIndex, this should avoid OOM failures" /Volumes/Magneto/dev/workspace/trunk/source/dbtest.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroCollectionIndex.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroDyn.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroEcoTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroRow.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroSplitTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlBalancer.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlStacker.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlZURL.java
- added index consistency check (checks for double-occurrences of primary keys in file)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4349 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9eb746863d
interface enhancements for eco records memory statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4348 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9abc927645
to fix inconsistencies in collection index, a double reference reporting mechanism has been implemented
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4347 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
58a1f518f8
fixed some problems with eco tables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4346 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d4d07802ac
better RAM protection using eco tables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4345 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f4e9ff6ce9
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4343 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
cbefc651ac
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4342 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
45339c3db5
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4341 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
94f21d9403
activated new kelondroEcoTable file structure.
...
This data structure replaces almost all files in the PLASMA directory
also the collection.index and the LURL-db will be created as Eco-DB, if it does not exist before
existing Flex-databases will be used as they are (the is no data lost)
If you want to force the creation of a Eco-collection.index, simply delete the old index.
The Eco file system will only be used if there is enough memory.
The collection.index RAM limit is 200MB, if you have less, a flex-Table is createt.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4340 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a0f7f2faad
some more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4338 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dc26d6262b
- removed write buffer from kelondroCache (was never used because buggy; will now be replaced by new EcoBuffer)
...
- added new data structure 'eco' for an index file that should use only 50% of write-IO compared to kelondroFlex
The new eco index is not used yet, but already successfully tested with the collectionIndex
The main purpose is to replace the kelondroFlex at every point when enough RAM is available.
Othervise, the kelondroFlex stays as option in case of low memory (which then can even use a file-index)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4337 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dbdec0f4d3
another fix for the "too many processes in loader queue, dismissed" - problem:
...
this was probably caused by http-forward cases; which are cases when urls from the loader queue change
and it was not possible to remove the old urls from the queue because they had been based on url hashes.
The queue is now again stored using the entry.hashCode, which does not change ieven if the url changes.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4332 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b806a6af8b
renamed kelondroEcoRecords to kelondroFullRecords (the "Eco"-name will be used for something else)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4331 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
065ba2d60f
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=719&hilit=
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4330 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f3f02b08ec
no distinction between standard and pro releases in auto-updater
...
this did not work in 0.56 main (but should).
Therefore it will be necessery to provide a hand-made 'virtual pro' (just renaming) release 0.57
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4326 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
3cab85158c
update for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4325 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
53367d941a
more information (BASE64)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4324 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b3636f5ba8
re-implemented file index in kelondroFlex
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4323 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a6ca3b51be
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4322 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
a5054c038d
- added large number of generics
...
- redesign of ordering structures in kelondro (old did not work with strict generics)
- 50% IO reduction during read access on kelondroFlex (ommiting of read on index table)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4320 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
71bcf02d3a
- removed pro-version (is the same as standard version, use the standard instead)
...
- changed yacy logo
- removed crawlOrder protocol (unused)
- removed file index in kelondroFlex (will not work, it takes too long to maintain)
- fixed remoted crawl for clusters (now denies remote crawls from peers outside cluster)
- 0.562
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4317 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ce7257483d
fix for bad fix with random access files (no performace enhancement)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4314 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
704de4dee8
Neue Funktion angelegt - notwendig für Einschränkung der Tagwolke
...
public Iterator getTagIterator(String tagName, boolean priv)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4313 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
016fc594af
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4311 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ecd7f8ba4e
- added NEAR operator (must be written in UPPERCASE in search query)
...
- more generics
- removed unused commons classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4310 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3e3d2e39a4
- some refactoring and redesign of kelondroBytesIntMap (created new class kelondroRAMIndex)
...
- more generics
- preparation to extend the balancer for flexible forced delay times
- set different random-access type, should now omit update of metadata in file and could be a bit faster (lets see)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4309 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
03e7782269
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4305 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f7c5ccedc7
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4301 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
7af60fb24d
*) fixed bug in update script
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4299 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
ae6d07bdb8
*) "Did you mean:" will only be displayed if the list of suggested URLs is not empty.
...
*) Removed <hr /> to make the "404 Unknown Host" error pag look like the other 404 error pages.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4298 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
408cb7a29b
*) added check if archive for update is OK, install if OK, else just restart ( http://forum.yacy-websuche.de/viewtopic.php?t=663 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4297 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
df2a7a8ac8
more generics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4295 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9d8b17188a
more generics, bugfixes for wrong cast
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4294 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
b08f877e97
*) tried to get rid of warnings when compiling parsers ( http://forum.yacy-websuche.de/viewtopic.php?t=660 )
...
lots of warnings are gone, new one in htmlFilterContentScraper
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4293 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
4dc438f7e7
moved to Java 1.5:
...
- changed build script to use java 1.5 compiler
- first stept to resolve missing generics definition (about 400 from over 4100 'missing'-warnings)
- added key-iterator to kelondro databases (for rapid from-memory enumerations, will be used for domain name collection, not used yet)
please set your development environment to use java 1.5!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4292 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
db0d3d5e54
release 0.56 (and some last fixes)
...
- fixed bad peer hash computation in case no peer list is avaiable upon first startup
- security minimum waiting time in search result preparation
- removed dead superseed link
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4290 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
d517e96714
last cleanup bits to serverDate before the release. only safe refactoring (method renaming) changes outside of serverDate.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4289 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hermens
4748d5c1ab
Some enhancements to time management:
...
- remove unnecessary generation of Calendar and Date objects
- synchronized SimpleDateFormat objects in blog-, message- and wikiBoard
- correct use of TimeZones and SimpleDateFormats
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4288 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
52dd015218
new release strategy: the standard release is now built the same way as the pro release
...
a new release type was added: 'embedded' which is the same as the current standard release was
this will not have any effect to the next release 0.56, which will still a pro-release on public download
the transition the the new release strategy must be done now to enable automatic update by the updated in future releases
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4287 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
1cb6e431a6
Replace the ISO8601 aka W3C datetime parser by one that supports every representation allowed by this standard, see http://www.w3.org/TR/NOTE-datetime
...
- useful expecially for sitemaps parsing, where this date format is used
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4286 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
33ee6745f6
more cleanup in serverDate
...
- remove direct accesses to SimpleDateFormat fields in serverDate and use the static parse... methods instead
- remove nowDate() as a Date doesn't store timezone information and a new Date() is always faster
- default formatter methods use a GMT timezone by default now, this is important for interchangability as some date formats we use don't include a timezone offset.
- continued renaming and rearanging (formatter) methods. all should follow the general naming scheme formatWHAT(...)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4285 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
3c30c2da75
more cleanup and API consistency changes, more to come...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4284 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
f41172f850
Merge httpDate into serverDate as suggested. Removed some unnecessary code and fixed a possible synchronization problem.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4283 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
a52681dd49
add buffering for the performance graph to avoid ConcurrentModificationException
...
closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=628
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4281 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
814aff60bd
- (re-)activated ftp protocol. see discussion here: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=623&hilit=&p=3875#p3875
...
- set default-flushsize for pro to 500
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4280 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
6fbda9ef4f
*) cleaned up code
...
*) added new filter 'FILTER_INVERT' and new method ymageMatrix.invert() to use it (does not work where characters have been written with ymageToolPrint.print(), haven't found the reason yet)
*) fixed a possible arrayOutOfBoundsExceptions in filter() if y-value of area to use filter in was larger than height of image filter is used with
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4279 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
21f7e13fa1
fix stupid tiny bug introduced in rev 4276 that broke request URL parsing almost completely
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4277 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
5d406d0094
- fixed url "file extension" parsing when there is no extension (like http://yacy.net/ would have extracted .net/)
...
- removed unecessary code + minor cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4276 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
21b8d1b918
small cosmetic change for static fields in serverCore (special protocol ASCII entities) to improve readability
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4275 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
270d016d89
fix for missing anonymization in search profiling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4274 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e3e4f06be4
enhanced search result preparation in the case that no result is found (fast abandon of search)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4273 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
1bd02762de
Improve HTTP/ICAP header processing.
...
- workaround for illegal line endings (LF only), closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=595
- fixed bug where we didn't break the processing immediately on EOS (the loop was run until the buffer was completely filled with -1)
- further performance improvements (one simple loop, avoid double processing of every byte and unnecessary temporary buffers)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4270 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
01554f4012
fixed bug with double-check in crawler
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4269 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b1e08d354c
repaired indexing after search snippet loading
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4268 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
48138952ff
added memory measurement for index recreation to avoid OOM during index RAM space extension
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4267 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9e23acf2d6
introduced new 'authority' ranking property
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4265 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
a3bfd668aa
opening of array files at startup time, not when first time the web index is accessed
...
this speeds up the first search after startup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4263 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ca488e03f5
fixed authorization case
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4262 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
6a3a292015
- smoothed ymage font
...
- changed position of status banner
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4261 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
7397152e04
*) quick hack for antialiasing, works only on borders now => less blurry image
...
*) code is not finished, needs refactoring, still thinking about how to do it best
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4260 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2954f96fae
- removed public peer info box on status page, this info can now be seen in the status banner
...
- added peer count to banner
- added some values to protected status box
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4257 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
4eb40c4f61
*) added 2 filters: blur and antialiasing (which in fact is nothing more than a mild blur) to ymageMatrix
...
*) antialiasing is used for logo in banner
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4256 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
aeb1cf83a6
- corrected banner link (relative now)
...
- changed color mode (replace) for banner
- changed default color (fits to default skin) of banner in status
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4255 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e22014dc83
some memory enhancements when generating and displaying ymage objects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4253 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f243e338cf
implemented online caution also for local and remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4252 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c57eb76b13
removed CMY color model from ymage classes and re-introduced RGB color model
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4249 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b46bcaa5d8
changed method of profiling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4248 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
76cd6ed6f6
*) New methods to insert bitmaps that feature transparencies.
...
*) Logo background is transparent now. (Using pixel at (0,0) to determine which color is transparent. Too dirty?)
*) Logo is loaded through filesystem instead of HTTP now.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4247 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
be214e594f
- generalized ymage initialization options
...
- auto-adoption of performance memory graph to needed dimension
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4246 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
ee8a177c26
*) Logo is in the middle of free space now.
...
*) Fixed bugs in insertBitmap()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4245 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
72698fcd36
*) Banner features a logo now. It does not look nice, but at least it works. Banner is not finished yet.
...
Which path do I have to set for IMAGE (htroot/env/grafics/yacy.gi) if I want to load it through the file system and not via HTTP?
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4244 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
39d0f10ca1
Fix parsing oof dates in HTTP headers.
...
RFC 2616 requires a client to support RFC 1123 (default), RFC 1036 and ANSI C formatted date strings (we only supported 1123 before).
Closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=525 (and probably others). There are servers which break the standards, please report those "DATE ERROR" messages if they contain a "sane" date string.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4243 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
aefb3f7765
added memory graph picture to PerformanceMemory_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4241 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9b0ae4b989
added referrer to remote crawl url list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4236 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
18e516317d
Fix problem with buggy HTTP-Servers which send illegal control characters in HTTP-Headers, they are ignored now.
...
Thx to celle for the patch and see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=560 for more information.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4235 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7d5544e9b1
added some security checks to new remote crawl pull method to prevent that indexer is overloaded
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4234 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
89b9b2b02a
redesigned remote crawl process:
...
- instead of pushing urls to other peers, the urls are actively pulled
by the peer that wants to do a remote crawl
- the remote crawl push process had been removed
- a process that adds urls from remote peers had been added
- the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version
- the list-interface had been removed
- servlets using the list-interface had been removed (this implementation did not properly manage double-check)
- changes in configuration file to support new pull-process
- fixed a bug in crawl balancer (status was not saved/closed properly)
- the yacy/urls-protocol was extended to support different networks/clusters
- many interface-adoptions to new stack counters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4232 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
fuchsi
69521d92e5
Add another external dependency from PDFBox package ("Bouncy Castle"). This is necessary for parsing of some encrypted PDF files.
...
bcprov-jdk14-132.jar is the binary jar as it is provided in the PDFBox-0.7.3 package (same as our FontBox, PDFBox packages).
Closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=453
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4231 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
90a02990d2
NPE fix, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3383#p3383
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4230 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2fcd18a972
- fixed bad behaviour of search event worker processes
...
- fixed export of url lists in xml
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4229 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
445c0b5333
added domain list extraction and html export format
...
to URL administration menu http://localhost:8080/IndexControlURLs_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4228 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d8d77fc4b2
fix for NPE, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3368#p3368
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4227 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bf6952abe7
- added url export to http://localhost:8080/IndexControlURLs_p.html
...
- removed command-line option to export urls
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4226 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
af10f729df
fixed image search and favicon loading
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4225 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago