orbiter
341f7cb327
steps to enhance remote search performance:
...
- added a file size limitation, that disallows parsing of large documents during (offline-) remote search
- added profiling information to search result computation, visible at search access tracker. this info shows used time for URL fetch and snippet computation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4112 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2f1ff048ba
some fixes to socket connection time-out
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4111 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3c74014004
automatic deletion of dead client connections
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4110 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
49f1c58d64
restoring alternative update location
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4109 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
11b4f80bde
- fixed non-closing client connections
...
- added client connection tracker in connections servelet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4108 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d352853f2d
fix for non-closing client sessions
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4107 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1488769e1f
cleanup of unmaintained and outdated performance methods:
...
removed object pools in httpc. Object pooling is not recommended,
if the creation of the object is not time-intensive. Object pools are only useful,
if there is much computation necessary to create some basic data that is stored
in the object pool and can be re-used. This does not apply to object pools in YaCy.
Object pooling of client sessions would make sense if they would allow re-use of
living connections to other yacy clients. But every connection is closed after usage
of an object in the client pool, therefore the YaCy server client objects are not such
that hold hardware/network-allocated entities.
See:
http://www.javaperformancetuning.com/news/qotm033.shtml
http://java.sun.com/docs/hotspot/HotSpotFAQ.html#gc_pooling
http://docs.sun.com/source/816-7159-10/pt_chap5.html
http://www.microjava.com/articles/techtalk/recylcle2
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4106 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3cb9cdc9be
try to fix connection problem, possible cause for wrong junior status and non-passive passive peers:
...
the YaCy client treats disconnections during data transmissions as error and discards all data transmitted so far
this did not happen so far until I removed a delay time at the end of the daemon session which prevented this case.
To fix this problem, disconnections during transmissions are not treated as error now, which means that end-of-transmissions
with sudden disconnections are not a cause for peer diconnections any more. To be nice to non-updated peers, the sleep time
at the end of server sessions is also re-enabled.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4105 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
00dab81077
simpler solution to last commit + works with and without navigation collumn on the left
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4104 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
eb16a99e94
avoid floating of long page titles around the favicon in search results
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4103 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
9524b9c16a
second try of rev 4100 :). Tested in Iceweasel/Firefox 2.0.6, Konqueror 3.5.7, Opera 9.23 (all linux) and IE6-SP1 (wine)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4102 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
6b8faaadb6
undo last commit for further evaluation, a progressbar element is used on other pages as well...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4101 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
1880bba420
A few changes to the progress bar and search result statistics layout influenced by the discussion in < http://forum.yacy-websuche.de/viewtopic.php?f=5&t=268 > with the idea of saving vertical space. Please check in every available browser and comment wether it's better than before. ;)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4100 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
404ebf1474
# update of de.lng
...
- NO unused strings anymore!!!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4099 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
041922652a
# update of de.lng
...
- removed or updated unsused strings
- updated some files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4098 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
ba59de773f
again and again junior - test
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4097 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
9fa75ef4d1
Limit the percentage of the progress indicator to reasonable values
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4096 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4275727d69
fix for peer ping problem (implemented a 3-time re-ping); cause for 'Connection reset' still unknown
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4095 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
e78098be9b
According to HTML-Specs "name" and "id" attributes share the same namespace. So we can't have one element with name="offset" and another one with id="offset". Additionally IE6's getElementById() returns elements with matching names as well and Opera is mimicing this behaviour.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4094 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
07d1e98909
fixed round-robin method of peer-ping order (the successfully pinged peer was not updated to current last-seed date)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4093 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
a1dcd065ad
some tweaks to the search results layout
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4092 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
76e4c2d69e
fix for peer-ping in case that remote peer does not respond with valid values
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4091 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
e192f99134
fix small bug introduced in r4089 that appeared when we tried to remove "gzip" encoding from Accept-Encodings header
...
closes http://forum.yacy-websuche.de/viewtopic.php?f=6&t=336
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4090 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ae4b9308ef
Fix problems with some web servers which couldn't handle the way yacy was sending requests. Thx to celle for the patch.
...
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=320
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4089 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
6601e37512
clear caches after changing blacklists, closes http://forum.yacy-websuche.de/viewtopic.php?f=6&t=241&p=1964#p1964
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4088 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
5b0c1449e1
various fixes and cleanups for blacklist handling:
...
1. avoid adding duplicate file name entries in config properties for lists,
2. correctly merge all path masks from all list files for the same host masks,
3. rewrite helper methods standard java methods for Collection transformations,
4. merged various methods with identical functionality for different Collection implementations into one,
5. minor refactoring to improve code readability.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4087 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e27aeb7fdc
patch for bad crawl filter at crawl start
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4086 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
841cf71022
fix for NPE in DHT transfer selection, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=327
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4085 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3047ae2cd9
fixed some more old links to new hompage location
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4084 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dbd1eeead5
fix for missing object miss-cache flush value:
...
the value is alway zero because there is no miss-cache flush
see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=288
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4083 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f2a3434407
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=238&p=1341#p1341
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4082 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
229ca2ba48
fixed/re-implemented rss-version of search result page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4081 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f4a5c287fe
re-implemented post-ranking of search results
...
(should enhanced search result quality)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4080 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
d962200d11
added time format for better understanding
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4079 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
8ff5e2c283
- fixed/re-implemented media search
...
- fixed search tipps (topwords, now appearing at the bottom of the page)
- added search consequences execution (deletion of bad referenced some time after the search happened)
- added some formatting at network table
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4078 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6c3bcadc1c
- re-implemented image search
...
- generalized search result status bar, is now also visible during text search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4077 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6c819a6fd9
added cache to favicon display
...
added better synchronization for simultanous search requests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4076 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
borg-0300
d69013f66a
added patch from Fuchs - http://forum.yacy-websuche.de/viewtopic.php?f=6&t=241
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4075 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
daf0f74361
joined anomic.net.URL, plasmaURL and url hash computation:
...
search profiling showed, that a major amount of time is wasted by computing url hashes. The computation does an intranet-check, which needs a DNS lookup. This caused that each urlhash computation needed 100-200 milliseconds, which caused remote searches to delay at least 1 second more that necessary. The solution to this problem is to attach a URL hash to the URL data structure, because that means that the url hash value can be filled after retrieval of the URL from the database. The redesign of the url/urlhash management caused a major redesign of many parts of the software. Since some parts had been decided to be given up they had been removed during this change to avoid unnecessary maintenance of unused code.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4074 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
66905b7c97
update cause of the new searchpage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4073 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e90afa9483
fixed search access tracker
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4072 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4779f314fe
first version of next-generation search interface:
...
- snippets are not fetched by browser using ajax, they are now fetched internally
- YaCy-internat threads control existence of snippets and sort out bad results
- search results are prepared using SSI includes
- the search result page is visible right after the search request, the results drop in when they are detected
- no more time-out strategy during search processes, results are shifted within queues when they arrive from remote peers
- added result page switching! after the first 10 results, the next page can be retrieved
- number of remote results is updated online on the result page as they drop in
- removed old snippet servelet (which had been also a security leak btw)
- media search is broken now, will be redesigned and fixed in another step
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4071 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
34858be5ef
added option to simple crawl start: complete domain crawl
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4070 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
michitux
d2360eaf68
- removed enctype="multipart/form-data" from the get-form of the peer-selection in Messages_p.html (in Konqueror this didn't work and multipart/form-data is only for post)
...
- removed name="searchresults" from the searchresults (seems to be no longer needed and is invalid)
- moved the favicons in the searchresults to the left side as requested in http://forum.yacy-websuche.de/viewtopic.php?f=5&t=268
- added alt-attributes for the favicons (images must have alt-attributes to be valid)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4069 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
0e27febe47
*) fixed more links
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4068 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
01ac8c8f6a
*) fixed dead link
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4067 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
a493bd88b6
*) updated a few links
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4066 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6d759ad0a7
- new bot address
...
- removed unused skins
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4065 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
71e5d24f4a
fix for watch crawler, see http://forum.yacy-websuche.de/viewtopic.php?p=1771#p1771
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4064 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f9e6cf6a3d
more refactoring of search:
...
integrated first version of ssi-using search interface,
but the function is currently disabled
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4063 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago