borg-0300
ec21c585cb
try to fix path too long
...
see http://www.yacy-forum.de/viewtopic.php?p=19079
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1893 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a6a3f4b694
fix for svn 1888
...
this is a redesign of the no-iterator solution
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1892 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
8da13088e9
*)removed multiple DHT_Distribution_Threads
...
*)boosted DHT_Distribution sending chunk parallel to multiple peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1890 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
283a7181c6
try to fix new 100% cpu bug, possibly caused by iterator method
...
see http://www.yacy-forum.de/viewtopic.php?p=18900#18900
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1888 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f588c0724f
removed cache flush in case of DHT receive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1885 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e94b374d56
update to cache flush method
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1884 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bcd99fe83e
introduced a second RAM cache for DHT transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1880 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
360a460da8
*)URL-Cleaner: moved logging-statement to correct position
...
*)plasmaURLPattern: host is now added to the hashset in lowercase
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1879 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
02f9765013
quickfix for time problem during cache restore
...
see http://www.yacy-forum.de/viewtopic.php?p=18810#18810
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1878 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
ad119f06af
*) Don't overwrite new entries with older ones
...
see: http://www.yacy-forum.de/viewtopic.php?t=2015
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1874 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
be88687d8c
fixed some problems with new cache flush karenz
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1873 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d3da7c9a08
*) Adding support for robots Allow directive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1872 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
f046e1814a
*fix or last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1869 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
c55c51e2a8
*)added keywords to IndexCleaner_p.java
...
*)updated Logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1868 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ddbeda738e
added minimum age of word in cache to performance menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1866 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f188611fc6
apply blacklist on rwis during dht receive
...
very experimental!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1865 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0ec28b8f8e
added DBCleaner from Hydrox
...
see http://www.yacy-forum.de/viewtopic.php?p=18093#18093
The servlet is now named IndexCleaner_p.
See http://localhost:8080/IndexCleaner_p.html
The Servlet was adopted to fit in the overall architecture
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1863 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
fb4100d47b
*) undoing last commit.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1856 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a84cc71218
*) removing getTotalRuntime
...
- not needed anymore
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1855 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
dce08771d1
*) Fix for wrong estimated and elapsed times when import was paused
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1850 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
b34713324a
DBImport: remove words from source index even if nothing has been added to home index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1849 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
520b60f15b
fix for http://www.yacy-forum.de/viewtopic.php?p=18610#18610
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1841 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bae3783d38
added a snippet marking
...
(search words are now bold in snippets)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1823 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f0a38873eb
* added yacysearch page with better view on search results
...
the old search page is obsolete and will be removed
* ConfigBasic.html is now the default page instead of index.html
as long as no password is set
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1815 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f0041d504d
remove of several results from a single domain is stopped if the result set is smaller than the wanted number of results
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1811 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
89286478e7
*) removing thread pool eviction for now. Not needed at the moment
...
See: http://www.yacy-forum.de/viewtopic.php?p=18290#18290
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1801 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
759800f543
*) Bugfix for storeHTCache problem
...
- content was not indexed if storeHTCache was off
See: http://www.yacy-forum.de/viewtopic.php?p=18269
See: http://www.yacy-forum.de/viewtopic.php?t=1882
See: http://www.yacy-forum.de/viewtopic.php?t=241
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1800 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a8548c0484
* several bugfixes regarding basic configuration
...
* extended number of search target peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1794 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1b9b8922d9
* fixed problems with new basic 1-2-3 configuration (now authentication required)
...
* fixed graphics problem
* fixed some other problems with default values
* 1-2-3 config now appears automatically on start-up if no password is set
* added new config menu
* moved profile to new config menu
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1792 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
8c6f38fe70
*) added Blog to YaCy (atm not reachable through interface) -> Blog.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1790 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ce5274c194
yacybot user agent
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1786 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
351bd0a678
*) dbImport: convert cacheSize to kb when creating plasma* objects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1773 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
eaffcfefe2
* added more ranking attributes (without function; this will be added later)
...
* added ranking coefficient transmission to remote peer (without evaluation on server side, will be added later)
* changed ranking coefficients slightly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1770 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
87e90b9d8c
refinements in ram cache flush procedure and default timing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1768 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d31a4e0b4f
some small enhancements with cache flushing parameters and data structures
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1767 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3703f76866
- fixed re-search bug: after a search with several words, a second search could not
...
find the same words as before. This was caused because indexContaines stored the url references
with a hashtable. A tree was needed to work with the index conjunction-by-numeration
- added permanent ram cache flush (again)
- removed direct flush of ram cache after a large container is added.
this happens especially during DHT transmission and therefore this fix should
speed up DHT transmission on server side.
- removed unused and out-dated methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1765 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
fbbbf5f411
*) remote trigger for proxy-crawl
...
- remote crawling can now be enabled for the proxy crawling profile
See: http://www.yacy-forum.de/viewtopic.php?p=17753#17753
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1758 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
dc9174c809
*) Implementing snippet fetching via ajax
...
Snippets that are not available on page load time will be fetched using ajax requests.
see: http://www.yacy-forum.de/viewtopic.php?p=16479
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1748 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1d8ca6e082
serialized dhtChunk deletion with indexing
...
The dht selection, transmission and deletion is now completely serialized with indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1731 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
2336f0f013
*) allow pausing/resuming of crawlJob Threads separately
...
- pausing/resuming localCrawls
- pausing/resuming remoteTriggeredCrawls
- pausing/resuming globalCrawlTrigger
See: http://www.yacy-forum.de/viewtopic.php?t=1591
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1723 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
60dac4325e
serialized indexing with dht selection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1719 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a840755964
moved parts of index transfer logic back to switchboard
...
this is needed to merge the dht selection with the indexing thread
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1718 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
134253a603
fixed bug with cache flush
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1717 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c2d863855d
different flush limit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1713 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
64441b1f78
ADDED: yacy.badwords list to filter the topwords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1711 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f9063e2040
added some synchronization to avoid that several tasks can trigger a cache flush simultanously
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1708 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
2c4e4ae6a2
further refactoring of dht selection, transfer and flushing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1707 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
73dad68cf1
outsourced thelis DHT flush class into own file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1706 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
aa4b04e3dd
reverted last change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1705 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
4b0dae8fcf
added a possiblity to get the ranking values for an url.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1703 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago