hydrox
88669ce008
*) cleaned up Settings_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1049 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
93cadb47b9
*) More tolerant robots parser for robots-files which missing empty lines between rule blocks
...
See: http://www.yacy-forum.de/viewtopic.php?p=12471
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1048 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
02f8013013
auto-delete of corrupted word files during word-migration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1047 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d2731418bf
added creation of global ranking files and changed url normal form usage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1046 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6f9f8ed8f8
*) Automatic Reset of Stack Crawler DB on startup errors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1432
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1045 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
fb766413d1
*) Changes on httpc dns caching
...
- Bugfix: old dns cache did not handle case insensitive hostnames correctly.
- adding a possibility to set domain name patterns defining hostnames that should not be cached by the httpc dns cache
e.g. borg-300.dyndns.org
This can be done by setting the new httpc.nameCacheNoCachingPatterns property
- using httpc.dnsResolve wherever possible within the sourcecode
[httpd.java,plasmaCrawlStacker.java]
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1044 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
89a4cca4df
max. num of Entries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1043 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
8d8d866494
Bugfix for catch up late Peer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1042 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bc420c62f6
fixed htcache path generation (never change a running system)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1041 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
795f488222
new urlNormalform version
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1040 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c86d801b0f
removed dyndns domains from dns caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1039 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6dc42a2392
detecting of loops in kelondroTree during last/first-Node search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1038 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
17d2830394
see: http://www.yacy-forum.de/viewtopic.php?t=1416
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1037 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
dd24f0252f
*) Searchword highlighting for info page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1036 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f9fb284fb7
*) Better handling of robots.txt files with incorrect keywords
...
See: http://www.yacy-forum.de/viewtopic.php?p=12292#12292
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1035 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
a1406f4617
urlNormalform: no logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1034 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
72cde1d894
getCachePath: no logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1033 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
1fbd72f9e0
rename "index.html" to "ndx"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1032 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
26c3f4aa5b
link update as requested by domain owner
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1031 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
cd1107d85e
added support for URLs with '?&'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1030 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
5fb2b017cb
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1029 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
60e869f236
bugfix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1028 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
544e4ea90e
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1027 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
00ab4d8723
cleaned, small change, Properties
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1026 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
440e6ed747
see http://www.yacy-forum.de/viewtopic.php?t=1416
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1025 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b8ceb1ffde
*) Adding better https support for crawler
...
- solving problems with unkown certificates by implementing a dummy trust Manager
- adding https support to robots-parser
- Seed File can now be downloaded from https resources
- adapting plasmaHTCache.java to support https URLs properly
*) URL Normalization
- sub URLs are now normalized properly during indexing
- pointing urlNormalForm function of plasmaParser to htmlFilterContentScraper function
- normalizing URLs which were received by a crawlOrder request
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1024 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
d2507c6081
rename setJunior()... to orJunior()...,
...
added javadoc,
added getPeerType(), setIP(), setPort(String port)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1023 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
e3179a6394
added getOwnSeedFile()
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1022 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
a803a509ae
bugfix: port handling in HTCache
...
grogram flow, cleared up
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1021 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
2c5999ae00
*)fixed UNRESOLVED PATTERN in ViewLog_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1020 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f871408729
*) sharedBlacklist_p.java
...
- Setting Pragma: no-cache
- increasing timeout to 12 sec.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1019 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3d0dfd4df4
*) Using StringBuffer instead of String concatenation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1018 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
452db479cd
*) bugfix: "21" was displayed as "21" in yacyWiki
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1017 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
cb69047b91
*)cleanup access static methods and fields
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1016 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
56b9f34411
*)removed unused imports
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1015 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
62b6c2b9e7
*)added news count to News.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1014 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
5f68b6886b
introduced new url-hashes for better ranking computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1013 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
aadace1285
fixed network image in search performance monitor
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1012 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bb369c98de
fixed search result ordering by date
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1011 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
295aff52a3
*)added offline-browsing-support (onlineMode=0)
...
*)online-mode now can be changed in Status.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1010 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4d1e56e4d9
fixed intermission-bug (removed 'break for intermission' of httpd-thread)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1009 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b058ecf0bc
refactoring of image-generation; added experimental PNG encoder (not active now)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1008 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
7a9fc653e9
fixed 100% cpu bug in wikiCode
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1007 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
fce616db23
security limits for network picture
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1006 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1aa707d357
fix for http://www.yacy-forum.de/viewtopic.php?p=11866#11866
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1005 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d42531e1b2
added auto-reset for NURL-DBs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1004 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
749f44d144
LiveUpdate.
...
(this does not work correctly at the Moment :-/)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1003 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
5526d2ff73
Basic WatchCrawler.html, which loades the Queues using AJAX.
...
ajax.js: generic Ajax-Functions
xml/queues/indexing_p*: the Backend
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1002 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
92c49b406b
adminAuth with userDB and adminAuthenticated (fix for statuspage)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1001 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
27f180f24b
Update of YaWoStat to 0.2.
...
Now does not try to make 400000! operations to load a 4MB textfile :-/
Program is not finished yet.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1000 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago