orbiter
b69f22e9ca
mistake in last commit: computation of loops in ReversingTwoConsecutiveLetters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6068 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3130334932
- start first with threads that run more loops
...
- join first with threads that run less loops
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6067 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
6cde7ebf16
DidYouMean
...
- without I/O intensive sorting by count
- but with multiple threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6066 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f348190566
tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6065 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
945777aa80
replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6064 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
303ccda69f
small fix for "did you mean"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6063 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7c4d1d471c
hand-over of more specific object
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6062 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9150bc0f7d
- don't show empty "did you mean"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6061 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
6c116be536
- set default &meanCount=5
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6060 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
09acfa66d1
- improved "did you mean"
...
- added &meanCount= to query string
- &meanCount=0 ==> no suggestion, no performance loss
- sorting suggestions by sb.indexSegment.termIndex().count()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6059 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
da6ce37f7b
- fixed encoding problem
...
- added limit to 10 suggestions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6058 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
54a48b4184
- added "did you mean" to search page
...
- currently works for single word queries only!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6057 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
31360ba40c
- Updated ConfigLiveSearch.html
...
- added documentation for load_js and load_css
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6056 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
ab09d8ebb3
- small noscript fix
...
- noscript is now functionall but ugly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6055 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
55ef9ae12a
small fix for last post
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6054 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
36dc9b09ac
- partial update to jquery-1.3.2
...
- partial update to jquery-ui-1.7.2
- yacyportalsearch fixed sidebar for navigators
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6053 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
550312ac85
added new command script to do a auto-Update from command line. this will make it easy to do mass-auto-updates in private yacy clusters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6052 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0fc1168554
- reduced time-out for socket-connection communication from 20 seconds to 5 seconds. This is a test to find out if the time-out was a cause for problems in metager environments
...
- turned a fine log entry in case of rejected connections on the server socket into a warning. (look for 'exceeding limit')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6051 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
28b86385cd
patch for bad behaving swf parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6050 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d58b395993
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15693#p15693
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6049 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
cffef67dc5
added a short info line about the latency monitor
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6048 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
733385cdd7
enahnced database access times by removal of unnecessary synchronization.
...
added also more hacks that resulted from high-volum query testing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6047 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
5a7dec880e
- some improvements for: http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15668
...
- portalsearch: introduced yconf.load_js and yconf.load_css
- yacysearch.html still having problems with focus after sidebar is loaded
- yacysearchtrailer.json seems not to be valid json for ?nav=all
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6046 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5d7045387b
added more word lists and a multi-access search test tool for high-performance query testing:
...
run searchtestmulti.sh; then 10 concurrent processes fire 1000 requests each to the local peer.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6045 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
398e210fef
removed synchronization in logging that causes deadlocks in high-performance environments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6044 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
db3a06dd81
removed cookie handling in httpc:
...
- no need to do cookie handling in proxy, this was switched off so far
- no need for cookies in crawler, this was switched on (by mistake)
This fix was needed for a case where a web server flooded the crawler with cookies and caused a complete blocking of the httpc.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6043 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c54ae4a63
some small changes in HandleMap Testing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6042 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b21e9149f5
another fix for navigation results, the json result format and searches with yacyinteractive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6041 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15c5406b9c
fixed yacyinteractive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6040 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
2c5554c912
small enhancements in search result computation speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6039 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e0b3984805
added navigation keys for site and author facets to remote search interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6038 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27fa6a66ad
- completed the author navigation
...
- removed some unused variables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6037 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a9a8b8d161
- added display of author navigation (usage of that navigator not yet implemented
...
- added a synchronization in pdf parser which should help to avoid deadlocks that occur when displaying several search results pointing to pdf sources
- fixed smaller bugs in navigation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6036 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c879783008
added steering of navigator computation:
...
- by default the navigator computation if off for servlet yacysearch.html, but:
- the servlet is called by default with a option to switch navigator results on
this will prevent that metasearch users will get slow results that are caused by unnecessary computations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6035 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c079b18ee7
- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.
...
- added a analysis method that counts bytes that could be saved in case the new HandleMap can be applied in the most efficient way. Look for the log messages beginning with "HeapReader saturation": in most cases we could save about 30% RAM!
- removed the old FlexTable database structure. It was not used any more.
- removed memory statistics in PerformanceMemory about flex tables and node caches (node caches were used by Tree Tables, which are also not used any more)
- add a stub for a steering of navigation functions. That should help to switch off naviagtion computation in cases where it is not demanded by a client
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6034 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bead0006da
replaced tmp file extensions by prt
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6033 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3189f9cd39
fixed problem with DCEntry initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6032 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a704d82280
patch for problem with digest
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6031 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3029ef6eb3
fixed a bug that was recently inserted which caused that no idx and gap files were written.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6030 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b6e274f211
omit most of forced crawl delays by using a separat delay table which flushes delayed URLs at the correct time
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6029 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d50be59088
- added a automatic re-construction of the domain stack after 10 minutes. this includes then urls to the domain stack that were left over in case of stack size limitations when the domain stack was created the last time
...
- changed the busy sleep time for the crawl thread to 30 millisecons. This is sufficient to crawl with 2000 PPM.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6028 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fdba0fa51
- fixed a not working selection rule in balancer
...
- more security about crawl-delay, be more fail-save
- better logging in case of long forced crawl-delays
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6027 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f5602404d5
another speed boost for the balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6026 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
95e8cbd1c3
new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6025 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c062385552
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15555#p15555
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6024 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
42ae40b9f6
some bugfixes to database close() methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6023 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a0c53abbe1
- wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521
...
- show only x+1 pages in page navigator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6022 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
94f3d90af2
added a hint about regular expressions in crawl start
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6021 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9bfd22f65d
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15523#p15523
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6020 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c77db670f
re-designed response format for navigation:
...
- changed json and rss response templates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6019 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago