orbiter
4a5b5515b5
added a lowercase to url normal form generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1966 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
44d72f06c4
more Caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1965 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
918445a2f4
Bugfix for last commit.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1964 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
219d847aaa
fix for appearance of radio buttons
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1963 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
8cac4a1e73
syntax fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1962 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
7310d2de1f
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1961 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0c9b61820e
enhanced re-crawl settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1960 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
708cc6c8d9
fixed some bugs for auto-filter and added monitor in profile list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1959 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1f32ad8fd9
very small enhancements to indexCreate page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1958 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
5938b93782
replaced a nick with real name
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1957 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
c58789177f
bookmarkCache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1956 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
250864406f
...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1955 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
157d01cc97
Checked in old version, sorry.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1954 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
42b0b10a95
-Adding Windows Media to types which are not sended compressed
...
-Renaming writeandzip to writeandgzip to avoid confusion about type of compression
-Adding new startup message to windows script
-The usual language "enhancements" ;-)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1953 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
2ed4fa96b7
tagCache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1952 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
330eb9c74f
bookmarkDB cleanup
...
(preparation for tagCache)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1951 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e82899ba57
fixed missing urls map initializer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1950 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
63f39ac7b5
added 3 new crawling steering options:
...
- re-crawl by age of page (enter in minutes)
- auto-domain-filter
- maximum number of pages per domain
NOT YET TESTED!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1949 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1fc3b34be6
some pre-work (without function yet) to implement:
...
- re-crawl (by age of last crawl)
- auto-crawl-filter by crawl depth (to be explained..)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1948 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0a4c2e89ed
remote crawl orders are now only accepted if sum over all
...
queues is less than 100 (the indexing queue was not measured before)
see also: http://www.yacy-forum.de/viewtopic.php?p=19374#19374
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1947 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c9e6b5e391
*) check size of indexing-queue and crawler pool before processing remote triggered crawl jobs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1946 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1509314ea6
set tighter control during DHT index and peer selection
...
see http://www.yacy-forum.de/viewtopic.php?p=19329#19329
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1945 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
fcc0683200
*) undoing last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1944 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
9411961eec
*) another little fix for DHT-Transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1943 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
77f3237de3
adapted for isListed()
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1942 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
8b14a0c833
*) little fix for DHT-Transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1941 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1f4412a146
adopted isListed to discussed new behavior as discussed (url, getFile)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1940 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
59fc55ea1e
added checks to protect peers from wrong seeds
...
see also: http://www.yacy-forum.de/viewtopic.php?p=19249#19249
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1939 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
eba22e7f89
include the defaultskins in the Release.
...
at least skins/default.css is needed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1938 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
cded4923ff
changed start-up message and Mac startup method
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1937 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
063ef4660a
bug?
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1936 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
82358677a9
added another shiftK2W to flushCacheSome
...
this should fix the bug that the DHT cache is not flushed if there is no indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1935 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
cb23fc3d83
keywords added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1934 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
1258df8133
no "[0-9]+" in topwords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1933 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
128e4ab199
- in serverSystem: maxPathLength is now a variable, not a method
...
- upon startup the calculated maximum path length is shown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1932 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a37b09e303
implemented automatic adoption of chunk-read-ahead in kelondroTree to needed chunk size
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1931 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
30e3e3a0fd
adopted MAXPATHLENGTH to host system capabilities
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1930 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
d808765087
something Javadoc
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1929 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
85bb8e32a1
Bugfix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1928 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
3fe402069f
try to fix
...
see: http://www.yacy-forum.de/viewtopic.php?p=19175#19175
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1927 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f16f1f15cd
bugfix for 100% CPU bug; thanks to Matthias for analysis
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1926 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
254a13efd9
MAXPATHLENGTH used
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1925 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
2635244489
Bugfix from cato
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1924 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
8865948e4e
Cleanup;
...
Methode replaceRegex added;
Constant MAXPATHLENGTH added;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1923 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
9c85820d35
added MIME-type for wmv and rm
...
removed double copyright at startup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1922 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6c70f4a0cf
renamed wordHashes for a word hash set generation to wordHashSet
...
This was done because the wordHashes iterator will get another integer
parameter and then conflicts with the wordHashes set generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1921 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d5f8f40c31
removed correcting iterator
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1920 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
526407f32e
adoptions, fixes for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1919 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
488a0ed580
replaced old keyIterator and rowIterator by buffered iterators
...
that are synchronized with database access
Main change is done in kelondroTree, other classes are only adoptions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1918 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
4e9a8f41fd
rwiDBCleaner + dbImporter: Iterate over small excerpts of
...
word hashes instead of the whole DB especially while changing
the DB in the process.
see http://www.yacy-forum.de/viewtopic.php?p=19136#19136
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1917 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago