karlchenofhell
d114a0136e
- crawl profile: don't add null-values
...
- added some settings and statistics for url-fetcher 'server'-mode
- added own stack for fetchable URLs
- added possibility to fill stack via shift from peer's queues, via POST (addurls=$count and url$num=$url) or via file-upload
- added "htroot" to classpath of linux start-script
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3370 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
b2a9d32f29
why do I always forget some lines? sorry...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3368 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
e1edb23689
*) Bugfix for IllegalMonitorStateException
...
See: http://www.yacy-forum.de/viewtopic.php?t=3522
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3358 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bf69a721cb
more protection against mis-use of YaCyHop interface:
...
- target must not be at port 80
- target access not more than every 3 seconds
- requester may not access more than every 10 seconds
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3357 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
a15963ff98
better balancing: if element from top would force a busy waiting,
...
an element from the bottom of the stack is used instead.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3356 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
dda24fcb85
ups
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3355 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
8c1d2e0227
protection against crawl balancer failure:
...
a minimum of 500 milliseconds distance between two acesses
to the same domain is now ensured
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3354 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
1f1f398bfa
enhanced speed of RAM cache flush by factor 20 (twenty times faster)
...
- the speed was doubled by avoiding read access during the dump
- the speed was dramatically increased at least by factor 10
by using a temporary ram-file where the structures are flushed to
before it is dumped then as a whole byte-chunk to the file system.
The speed enhancements also affects some other parts of the database.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3353 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
30d79d69a6
fix for wrong display of search statistics
...
see http://www.yacy-forum.de/viewtopic.php?p=31242#31242
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3352 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
daf2e15f59
some storage process enhancements (write without preceding read)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3348 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9c2101a852
small enhancement to cache dump
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3346 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c464157a6e
replaced some toString()
...
see http://www.yacy-forum.de/viewtopic.php?p=31151#31151
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3345 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7673f0869b
minor enhancements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3344 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b4aa195c27
added user-agent check for yacy-hop proxy authentication
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3343 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2d7f7da7ce
fix for null pointer exception
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3342 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d25caa07bf
redesigned some parts of http authentication
...
added another access check for peer hops
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3340 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
9184113284
*) fixed News deletion. News are now removed if they are no longer in a news-stack. This does not effect News-entires in the news-db that have no stack-entries.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3336 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
e218940293
The copyright sign "\u00A9" is already replaced by "©". String "(C)" is not a unicode sequence!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3334 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
1bc4d8d470
*) If there is more than one pair of patterns in a line, all of them (and not only one pair) will be replaced.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3333 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
ea7a8cf7aa
*) <hr> and <br> tags are XHTML compliant now.
...
*) Avoid superflous trailing blank in non-proportional sections.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3332 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d03cd41266
fix for http://www.yacy-forum.de/viewtopic.php?t=3411
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3331 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
f2e6f19b90
- added versioning to Wiki
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3327 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
10ebcb5e1e
fixed bug in disconnection time
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3326 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
02a73dce87
- added Diff-class for wiki-versioning (forthcoming, first need suitable serverObjects.put() for it)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3325 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
2401e748a3
- fixed wrong replacement of POST-parameters in httpd ('<' and '>' are still replaced, don't know why): http://www.yacy-forum.de/viewtopic.php?t=3466
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3324 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b2f4087400
redesign of last-seen fieln inside seed:
...
the field contains now a time in UDC-0 (instead relative to local UDC offset)
this fixes a bug in peer selection, where an iteration over all seeds
ordered by lastseen did not work correctly.
Problems may occur because the new meaning of this field may mix with
the different meaning of that field in older peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3322 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
e68cdeeeb3
- reverted parseArg(String) to use a byte-array to handle correct UTF-8 parsing
...
- arguments aren't passed html-escaped to the servlets anymore, bug-fix for http://www.yacy-forum.de/viewtopic.php?p=30573
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3321 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e00e850a98
removed constants (no connection with yacySeed.dna identifier)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3320 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
fcc11391a8
some redesign attempts because sorting of lastseen does not work correctly
...
not finished yet
target: better selection of peer-ping targets, which should enhance stabilization of the net
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3319 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c2d6edf21d
integrated number of remote targets as 'partitions' into remote search protocol
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3317 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f696d3c1eb
added double computation to kelondroMapObjects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3316 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
819ff21c92
fixed QPM output
...
QPM is temporarily called QPH (until more search requests are present?)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3313 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
auron_x
89e7af037a
*) used more switchboard-vars instead of config-vars
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3310 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
306c50ac40
QPM (queries per minute) statistic stub
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3308 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7598e1243e
removed unused variables/imports
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3306 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
47ab83a7c0
added flag for YaCyHop - proxy access for all paths that start with /yacy/
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3304 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
9f74b128dd
- added many more commented constants (please use constants rather than i.e. config-setting strings directly)
...
- not all constants may be located correctly in the switchboard. Please relocate if you know the appropriate place for them
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3303 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e4910f03d1
tag storage fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3302 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
991182b29b
more space for bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3299 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
88fa764b64
implemented new kelondroObjects into bookmarkDB
...
- Bookmark-Objects are stored inside the kelondroObjects cache
- removed superfluous classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3298 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9c05e2a820
re-design ob kelondroMap
...
- this class is replaced by an object that can hold any type of object
- this object must be defined as a class that implements kelondroObjectsEntry
- the kelodroMap is now implemented as kelondroMapObjects
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3297 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
669c21db05
first version of abstracted kelondroMap Cache.
...
get returns a kelondroCachedObject(or in most cases a subclass of it),
or a map, which can be used to construct a kelondroCachedObject.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3295 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
98cb777e18
abstract wikiCode in putWiki
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3293 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f25c0e98d1
- replaced String by StringBuffer in condenser
...
- added CamelCase parser in condenser
- added option to switch on or off indexing for proxy
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3292 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
14f2068daf
some more bookmark changes towards multiuser bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3291 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
25c7d4e25e
fix for form (cookie) login
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3284 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
ff79c52fc0
bookmark users can now edit bookmarks.
...
TO COME: tag bookmarks with username, list bookmarks of a special user, filter private bookmarks for users.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3274 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
49972b6193
*) SOAP: getPeerHash returns the proper value now
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3273 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
15f0334cd3
- fixed IllegalThreadStateException in LogParser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3265 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
f40169fcd7
preparing multiuser bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3256 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago