hermens
b48327904a
Don't disconnect peers that report 'busy' during index transfer.
...
These peers are already being marked as not accepting remote index transmissions by yacyClient.transferIndex. That should by enough to prevent further transfer attempts until newer seed information is received.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2142 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4d8f8ba384
added cache-performance analysis for node caches
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2140 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bd057b44dd
- automatic setting of peer-does-not-accept-remote-crawl
...
- increased percentage of object cache to node cache to 30%
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2136 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
81e79f2caf
fixed new cache behaviour changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2134 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
cda087f43b
- integrated cache miss storage into object cache
...
- removed cache-miss handling from indexURL
todo: new Monitoring in PerformanceMemory_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2132 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
757ec28430
refactoring: better data capsulation for indexURL
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2131 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
61078b3885
*) adding support for delayed shutdown
...
- needed by Ismael to receive the Steering page properly on shutdown
- now the steering page should always be displayed properly in the web browser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2129 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
90d569d70f
refactoring of index management:
...
url storage is part of index management; moved plasmaURL to indexURL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2122 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a930be4ba3
refactoring of index management:
...
generalized the index entry
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2121 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
df7e1d9df3
Changes to plasmaURL and subclasses:
...
- Improve performance of plasmaURL.exists() by remembering URL-hashes that are not present
- Use a more realistic estimation of memory usage by the existsIndex cache
- Routine cleanup of the existsIndex to limit its memory usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2113 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a474669338
start with refactoring of index management
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2110 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
f08e33680c
Added Blog-news-symbol as requested.
...
I think I will change the character distance a little bit later.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2101 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f331def5d8
*) Bugfix for distribution. Incorrect behavior if peerCount == selectedCount
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2098 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
55ea4cbfe6
*)reverted patch for memory-display issue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2095 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5048b05bc6
*) Index Transfer should only restart at the beginning if the delete
...
option is configured. Otherwise we have an endless loop
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2092 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
53d9ab6db7
*)fixed bug in PerformanceMemory_p.java which caused negative memory-values on big peers
...
see http://www.yacy-forum.de/viewtopic.php?t=2370
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2091 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
ddfe0f0e27
*) don't try to parse referer string if it's null
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2090 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bcc950c533
*) Bugfix for Index Transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2088 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
015d044c25
tried to fix some problems with latest changes to httpc
...
very experimental!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2078 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3e31820c3d
- corrections to PerformanceMemory display of object cache
...
- configuration of object cache size in kelondroTree initializer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2075 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
461548698c
configuration of index transfer chunk size
...
see http://www.yacy-forum.de/viewtopic.php?p=20951#20951
new properties in yacy.init:
indexDistribution.minChunkSize = 5
indexDistribution.maxChunkSize = 1000
indexDistribution.startChunkSize = 50
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2073 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
29b1b0823c
added monitoring of new object cache to performanceMemory page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2072 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9104001e7c
*) Better error handling for assortment import
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2067 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
51e3bb576f
Don't increase dhtTransferIndexCount when the last transferred index was smaller
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2064 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
a0ca4c5fb8
Remove a possible race condition between DHT transfer and deQueue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2059 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
0cfba8950f
Removing unnecessary and possibly dangerous synchronization of the wordIndex
...
when deleting transferred indexes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2058 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d6213f8a85
quickfix for http://www.yacy-forum.de/viewtopic.php?p=19482#19482
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2042 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b0036249c1
added some attributes to network picture
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2032 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
cbcf7418ef
Cleanup synchronization in plasmaWordIndex
...
- only synchronize when changing data in more than one database
see: http://www.yacy-forum.de/viewtopic.php?t=2167
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2031 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
60e5aff9fc
some enhancements to the remote crawl trigger
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2030 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
dbe96e6541
added hand-over of search filter and prefer ranking to yacy protocol
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2029 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
0604203bce
Updated and corrected German language file
...
Changed Italian language file for an Italian/English interface and not Italian/German
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2024 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
00a5d435e2
- fixed some bugs with domain filter
...
- added new ranking filter "prefermask": urls that match the filter are ranked better
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2022 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
14d6e476c9
tried to solve some problems with new picture viewer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2019 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9324425165
fix for remote crawl reject
...
see http://www.yacy-forum.de/viewtopic.php?p=20075#20075
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2017 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
30e4fc39a5
HTCache extended
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2015 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d0dd8b14d2
fixed picture tag and presentation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2014 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
da6a8bafa2
rename currCacheSize -> curCacheSize;
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2010 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
92110aea32
nullpointer fix for profile(); other minor change;
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2009 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f0833b0328
introduced simple search interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2007 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
47b541b2d1
added better option handling in yacysearch
...
added depth option for image presentation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2001 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c9e16bfd48
first try to insert image search (does not work yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2000 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f77775220b
fixed parser error
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1999 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
22de954a57
added some log output to parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1996 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
83e0e765ec
redesigned some parts of the html scanner & parser
...
to better support image tags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1995 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ac114d69c0
tried to fix some problems with time-outs during search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1994 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e2e8d0c188
some kind of refactoring of yacysearch:
...
made 'room' for new picture search result presentation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1993 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6b63e26cbb
- removed search function from index.html/java, only imput left
...
- added media fetcher/crawler class (not ready yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1992 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bc3e80fe42
quickfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1990 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d8d0ac29c3
added image-viewer servlet that can do:
...
- each image that is requested is stored in the cache
- the image is taken from the cache if exists there
- the image can be scaled
The purpose of creation a scaled image is because of copyright problems
In a further stept the retrieval of not-shrinked images is restricted
to either access from localhost or with given authentication
This servlet can be used for image-preview purpose after an image search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1989 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago