orbiter
bd5f4c78d8
- added default profile for surrogate indexing
...
- integrated surrogate indexing into indexing queue process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5810 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ad78e3a59f
- less lines in rssTerminal
...
- crawl more documents: if remote crawling is enabled, a remote crawl list is also loaded if a local crawl is running in case that the indexer is idle
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5809 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
12d81e98eb
- fixed bad search results when searching for empty string
...
- simplified result handling and page composition in case that nothing was searched
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5807 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8a24350036
- fix for join method with new generalized RWI data structure (caused by latest commit)
...
- added more functions to mediawiki parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5806 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
89ec3acb3e
- full abstraction of index content type: the kelondro full text index may now also contain indexes about other content than text, i.e. navigation indexes or reverse linking indexes.
...
- during index joins all word positions are maintained: better ranking for word distance possible; exact phrase match can be implemented soundly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5804 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dc2af61bc9
allow up to 50 results from remote peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5802 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
6504b21cea
*) fix for http://forum.yacy-websuche.de/viewtopic.php?t=1976
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5800 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
601d63ef48
removed comment tag (no use at this point)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5797 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
0c8fd811dc
*) first and very limited version of XML import, does not use benefits provided by XML yet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5795 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c08f9b36a4
refactoring of wiki parser.
...
This was done to prepare the wiki parser as parser for wikipedia dumps, which will be used for performance test (to omit crawling)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5785 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
faeff21012
- fix for display of automatic ReCrawls in surftips
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5784 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
44e01afa5b
- refactoring
...
- a little bit more abstraction
- new interfaces for index abstraction
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5783 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c2359f20dd
refactoring: better abstraction of reference and metadata prototypes.
...
This is a preparation to introduce other index tables as used now only for reverse text indexes. Next application of the reverse index is a citation index.
Moved to version 0.74
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5777 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a29a11e526
added evaluation of incoming links in webstructure api
...
the api hash changed, new XML schema.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5774 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f21a8c9e9c
a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5771 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7ba078daa1
- added fast site-operator
...
- refactoring merge into BLOBArray
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5770 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bd409fb7ba
added web structure analysis for a special domain that can be requested from the api.
...
Example:
http://localhost:8080/api/webstructure.xml?about=www.yacy.net
returns a xml with the following content:
<?xml version="1.0"?>
<webstructure>
<domains reference="reverse" count="1" maxref="300">
<domain host="www.yacy.net" id="FXg39Q" date="20090401">
<citation host="java.sun.com" id="o-R3yY" count="1" />
<citation host="yacy-suche.de" id="-KCLaB" count="1" />
<citation host="suma-ev.de" id="VRAHIA" count="1" />
<citation host="www.kit.edu" id="EMaLDQ" count="1" />
<citation host="yacy.net" id="Fh1hyQ" count="1" />
<citation host="www.fzk.de" id="V2Kl-A" count="1" />
<citation host="en.wikipedia.org" id="rwtdfR" count="3" />
<citation host="vimeo.com" id="MmdQDY" count="3" />
<citation host="liebel.fzk.de" id="sX4ozA" count="6" />
</domain>
</domains>
</webstructure>
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5766 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
587838bd09
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5758 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
96eaecda3e
- added migration class to go from index collections to the index cell data structure.
...
- added better control over file deletion, because this sometimes fails, especially on windows
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5756 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9ab009b16b
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1890#p13476
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5755 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
eb65990f85
small fix for opera in yacyui-portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5751 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
695c420bcd
small fix for yacyui-portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5750 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
95885a263a
- added default properies to yacyui-portalsearch
...
- see http://localhost.8080/yacy/ui/yacyui-portaltest.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5749 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
c001a020a9
- small modifications to yacyui-portalsearch
...
- see http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1762&p=13459#p13459
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5748 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
34a825f90d
small fix for yacyui-portaltest.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5746 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9f9d7f875d
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5745 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
453f3aaa94
RichClient: further clean-up
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5744 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
e888c9a934
RichClient:
...
- renamed base theme to start theme
- reoved all but start theme
- additional themes can be downloaded from http://jquery-ui.googlecode.com/files/jquery-ui-themes-1.7.zip
- or a custom theme can be generated at http://jqueryui.com/themeroller/
- themes are installed into DATA/LOCALE/htroot/yacy/ui/css/themes
- update for RichClient theme selection will follow soon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5743 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
42c5f930c8
reverted an accidental commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5742 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b5e6232f8d
small correction of font-size for portal search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5741 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
7425c6c3ca
added an ajax loading graph to portal search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5740 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a975ae4a7e
Added YaCy portal search: http://localhost:8080/yacy/ui/yacyui-portaltest.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5739 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b57a1820bd
small fix for jquery-faviconize-1.0.js to handle https properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5738 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
075b58a0a9
minor fixes for RichClient search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5737 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
8c494afcfe
svn attributes added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5734 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
67aaffc0a2
- added Latency control to the crawler:
...
because of the strongly enhanced indexing speed when using the new IndexCell RWI data structures (> 2000PPM on my notebook), it is now necessary to control the crawling speed depending on the response time of the target server (which is also YaCy in case of some intranet indexing use cases).
The latency factor in crawl delay times is derived from the time that a target hosts takes to answer on http requests. For internet domains, the crawl delay is a minimum of twice the response time, in intranet cases the delay time is now a halve of the response time.
- added API to monitor the latency times of the crawler:
a new api at /api/latency_p.xml returns the current response times of domains, the time when the domain was accessed by the crawler the last time and many more attributes.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5733 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
61f9dbf0cc
- fixed a display problem in watch crawler
...
- another small enhancement in balancer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5729 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b3f75e48fa
- enhanced balancer: auto-solving of waiting-deadlocks
...
- removed deprecated cache-init size value
- more debug lines for IndexCell cache dump merge
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5728 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
83792d9233
more refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5722 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
474aac65af
more refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5719 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
209f25f5f5
refactoring to integrate indexCell data structures
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5718 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7dff1cba62
removed option to use different primary keys in kelondro tables
...
this option was never used and there is also no use to set other columns but the first as the primary key. as a result, access methods to the key do not need to compute key positions, and they work faster.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5711 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7f67238f8b
refactoring of plasmaWordIndex: less methods in the class, separated the index to CachedIndexCollection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5710 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
14a1c33823
refactoring of wordIndex class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5709 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e2e7949feb
replaced old PPM computation with a better one that simply sums up events that had been stored in the profiling table.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5706 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
d612430fce
copy paste mistake
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5689 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
acbdac1b67
corrected examples
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5688 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
396a4451be
increased timeout in ViewFile
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5685 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9b6fac4a82
RichClient: better handling for small screens/windows
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5681 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
8ee946bf1d
show upnp status
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5679 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago