orbiter
474659a71f
- modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order
...
- added option to set minimum crawl delta for domains in balancer
- added default values to crawl deltas in yacy.init
- added configuration for these deltas in performance queues
- enhanced performance setting computation (more time for indexing queue for a faster flush
- remote crawling is now enabled during local crawling if indexer has space and time for more links
- added database stub for new distributed file system
- refactoring of time computation to get an abstraction level that will be used by a TTL rule in new distributed file system
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4966 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
1afc36d9a8
* update to JRE 6u6
...
* fix for proper translation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4965 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
8e179f6588
removed option to do a re-crawl with a period of minutes. Such a short time does not make sense and it may cause endless indexing loops. The removing of the option will ensure that a misuse is prevented.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4964 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
080cda97ef
added another peer selection rule:
...
- select also non-robinson (dht-) peers if their peer tags match with search words
- the peer tag '*' can now act as catch-all rule: shall be selected always
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4963 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d37fd064f9
changed peer selection for search targets:
...
- less dht targets are selected
- more other peers are selected: all robinson peers with more than one million urls
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4962 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
69aac0d74c
modified the diskUsage class regarding the following two aspects:
...
1. The usage and dependency of the plasmaSwitchboad was used many times in the past but this was
a bad mistake. The classes should be independent from the switchboard to support a better abstraction. Therefore the object was removed. The parameters from the switchboard are computed outside and then handed over.
2. the class is considered as a tightly connected to hardware resources. Classes which handle data that cannot be replicated because it would need to replicate hadware should not support dynamic object allocation, but should be coded as collection of private static methods. Therefore all class objects had been transformed into static private objects.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4961 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
da917cf4b1
undo reduced menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4960 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
0c1dc703e4
- set staticIP at startUp
...
- added setting for reduced menu (simpleMenu)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4959 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f6b92456c0
confirm bookmark delete
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4958 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
f7f9ceb967
diskUsage: replaced blocking sleep with semaphore
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4957 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
4a53649ee7
fixed dht-urls and ranking distribution log statistics
...
* NOTE: please have in mind that there can be whitespaces in pathnames
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4956 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
8d83185cb4
fixed dht-chunks/protocol log statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4955 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
63eadfdf84
fixed unlimited FileSizeLimit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4954 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
2dc7c00c1c
fixed indexing log statistics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4953 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
dba7ba079e
fixed NPE seen with queues_p.xml (serverClassLoader finds already loaded classes)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4952 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
c5e5c554d2
corrected unresolved dings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4951 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
273fb01142
revert last fix; was wrong
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4950 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
b6f50851fa
fix memory requirement calculation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4949 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
ac85c52bae
better readability for MIN_FREE_DISK_SPACE
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4948 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
54a73b58cf
fixed restart on Windows when directory had spaces in it's name
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4947 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f30f976f2d
installer downloads Java to shared documents and creates a shortcut on the desktop in case user is no admin
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4946 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
0a956d1b11
*) removes focus on text field on search result page ( http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1257 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4945 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
2a53551423
removed unused code from installer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4944 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
609aaf0df3
rework of the windows part
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4943 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
1a4f26ba30
exclude HTDOCS from recursiv scan
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4942 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
6c07e894d9
add needed sleep
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4941 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
d742cc080c
Fix for RAMCache not flushing
...
see: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1255
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4940 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
6b7e873962
resourceObserver refactoring and some synchronisation for console output
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4939 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
01e9c7320e
PerformanceProfile: don't change peerPing, peerSeed and cleanUp
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4938 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6bdd99e065
- more asserts to solve the ooB-problem
...
- better caching (?), lets see how it behaves
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4937 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
24fb2d0922
distribute searchtest
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4936 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b928ae492a
some code-cleanup and possible speed enhancements in different core methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4935 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
6a9cc29cdd
workaround for IndexOutOfBoundsException in ResultURLs.getExecutorHash() seen @ CrawlResults.html?process=4
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4934 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c998dc6556
- added security functions to flush url and search caches in case that memory is full
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4933 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f4ae8082c3
- better error analysis for ooRange Exception in kelondroBase64Ordering
...
- quadcore support for kelondroRowSet array ordering
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4932 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f8af00d7e8
added some experimental jvm startup properties which helped the metager peer to get more stable.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4931 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
84cbe75005
more asserts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4930 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e269c12710
small changes in partition routine
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4929 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
31efb8fbee
- fix for LOG path generation when the DATA/LOG does not exists (fix for bug introduced in SVN 4923)
...
- some more/better asserts
- slight performance enhancements in remove method in index management. Works for all who do not run using asserts (the majority)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4928 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
877299cc74
better installer on Windows Vista
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4927 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
726218dd4a
fixed logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4926 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
21c87c36e3
added a log line
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4925 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
5ea4fcd946
eindeutschung der Performance-Profile
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4924 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
68c38c2d34
- WatchCrawler shows status without JavaScript
...
- Performance can be scaled + DHT-profile
- names for pool-threads
- some small refactorings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4923 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
da4b1b5c0d
added a little translation for the new searchinterface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4922 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
d4fa7834f7
added flori and me
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4921 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fc79f013c4
better solution to update shortcut
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4920 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
c0dfe49743
also exclude collection.0028.commons and RANKING at startup check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4919 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
cc40cde9e7
update of readme: use Java 1.5 instead of 1.4.2
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4918 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
det
11656741f1
exclude LOCALE and RELEASE at startup check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4917 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago