orbiter
91dca2cd8d
fixed a bug in last commit: LURL entries cannot be written,
...
because a stored property was not set to false (but true)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1906 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3286b1f498
re-organisation of lurl-creation and -stacking
...
this was necessary to prevent useless write to the database
in case of blacklist appearance of the url
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1905 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
289da326e5
*) Bugfix: remove blacklisted URL from loadedURL, when received via DHT transfer
...
see: http://www.yacy-forum.de/viewtopic.php?p=18976#18976
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1904 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
9b94b374cc
Bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1903 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
60995a3a67
changed behaviour of "search again" appearance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1902 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9cca36a107
no more strict comparator checking in map exclude method if not needed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1901 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0b903c5317
removed usage of kelondroNaturalOrder from plasmaCondenser to experimental
...
exclude cause of a 100% bug.
see http://www.yacy-forum.de/viewtopic.php?p=19076#19076
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1900 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4239db0d1c
fixed new ordering for backup iterator TreeSet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1899 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
399538b7de
Bugfix: wrongly compared
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1898 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
0b056f5cc7
a irssi (www.irssi.org) Script for YaCy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1897 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
33eba5ecb8
temporary disabling last change, does not work (cannot debug right now)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1896 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f0464042fc
fix for latest iterator-replacement-fix:
...
iterator generated TreeSet which did not resprect rotations
this has now be implemented using kelondroOrder Objects
and by adding this rotation-rules to the ording
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1895 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3881f9ccde
fixed total number in network list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1894 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
ec21c585cb
try to fix path too long
...
see http://www.yacy-forum.de/viewtopic.php?p=19079
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1893 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a6a3f4b694
fix for svn 1888
...
this is a redesign of the no-iterator solution
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1892 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
1fc494858d
*)added blog to the nav-bar
...
*)added some padding to blog entries (looks nicer)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1891 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
8da13088e9
*)removed multiple DHT_Distribution_Threads
...
*)boosted DHT_Distribution sending chunk parallel to multiple peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1890 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
daburna
7605ddd450
just testing my new svn-client with linux - daburna - if login is possible...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1889 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
283a7181c6
try to fix new 100% cpu bug, possibly caused by iterator method
...
see http://www.yacy-forum.de/viewtopic.php?p=18900#18900
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1888 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
(no author)
5ddc32f49a
just testing my new svn-client with linux - daburna
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1887 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
34c075c1c7
testcommit with subversive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1886 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f588c0724f
removed cache flush in case of DHT receive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1885 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e94b374d56
update to cache flush method
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1884 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
9f979d4fa5
Domain-lists gzip-compressable and sendable via cr-send/receive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1883 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
1a13c8b78e
right wordCachesize after orbiters commit.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1882 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
6b056610e3
updated watchcrawler for the recent changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1881 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bcd99fe83e
introduced a second RAM cache for DHT transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1880 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
360a460da8
*)URL-Cleaner: moved logging-statement to correct position
...
*)plasmaURLPattern: host is now added to the hashset in lowercase
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1879 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
02f9765013
quickfix for time problem during cache restore
...
see http://www.yacy-forum.de/viewtopic.php?p=18810#18810
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1878 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
5d4f144f3f
some de polish...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1877 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
69d26045eb
quick-fix for http://www.yacy-forum.de/viewtopic.php?p=18805#18805
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1876 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
b3df5ce2b8
another local iprange
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1875 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
ad119f06af
*) Don't overwrite new entries with older ones
...
see: http://www.yacy-forum.de/viewtopic.php?t=2015
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1874 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
be88687d8c
fixed some problems with new cache flush karenz
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1873 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d3da7c9a08
*) Adding support for robots Allow directive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1872 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
7fe0658f8b
*) updated german language
...
- cleaning of most <a>-Tags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1871 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
734d18f283
*) more correct robots.txt validation
...
- isDisallowed now uses getFile instead of getPath
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1870 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
f046e1814a
*fix or last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1869 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
c55c51e2a8
*)added keywords to IndexCleaner_p.java
...
*)updated Logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1868 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f0ad0d2b2b
*) better robots.txt support
...
- previously rules for all crawlers and special rules for yacy where
combined using AND. Now the general rule will be ignored if there is
a special rule for yacy (according to rfc)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1867 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ddbeda738e
added minimum age of word in cache to performance menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1866 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f188611fc6
apply blacklist on rwis during dht receive
...
very experimental!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1865 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
915812f597
*) Undoing robots parser policy changes from svn rev. 1421
...
- crawling is not allowed if server returned a 403 statuscode (according to rfc)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1864 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0ec28b8f8e
added DBCleaner from Hydrox
...
see http://www.yacy-forum.de/viewtopic.php?p=18093#18093
The servlet is now named IndexCleaner_p.
See http://localhost:8080/IndexCleaner_p.html
The Servlet was adopted to fit in the overall architecture
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1863 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
74ffb9b842
added more info/examples
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1862 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5ee0125046
*) adding possibility to configure the server port for seed uploading via scp.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1861 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
2065138a41
reverted last Commit.
...
@Orbiter: kannst du dich dann mal drum kümmern, wenn ich versuche die ganze Sache ans Laufen zu bringen, hast du jedesmal was dagegen. Dann mach du es bitte, du wirst ja wissen was du willst...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1860 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
7afa5c1b8e
staticIP fix
...
tried to solve http://www.yacy-forum.de/viewtopic.php?p=18663#18663
D 2006/03/08 07:08:20 YACY yacyClient.publishMySeed mySeed error - not proper: IP is not proper: -UNRESOLVED_PATTERN-
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1859 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
f6452879d5
prevent nullpointer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1858 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
04ab5da350
fixing server & proxy access settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1857 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago