karlchenofhell
b873ad51ab
- fix for http://www.yacy-forum.de/viewtopic.php?t=3369
...
- merged netBude's alternative for tables in yacysearch.html & search results valid
- added statistic info to index.html as proposed here: http://www.yacy-forum.de/viewtopic.php?p=29762#29762
- fixed error-log in httpTemplate
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3189 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
auron_x
7fb30de8a4
*) made combined2prettyversion less restrictive and made it again easier to read
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3187 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
3acd90033c
*) added functions to get results from log-parsers (not documented yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3186 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
1aa74bbd2b
update for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3185 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
23e613b2ab
CPU & IO reduce (Index Distribution)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3184 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
c67d22116e
added exists-check based only on RAM index lookup:
...
- faster double-check during crawling
- less IO
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3179 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
37e53b4a6a
replaced tree database structure for seed db by flex data structure
...
I don't know if this helps, we will find out...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3177 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
340dc52a9d
- ConfigProfile_p.html now transmits usable encoding for other than 7-bit ASCII charset, see TODO in httpd.parseArg(String)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3174 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
35fb671721
- updated DetailedSearch and ViewFile
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3173 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
00aa9472d6
- added decode of HTML-entities in request lines
...
- removed Bookmark symbol on search pages and surftips if not authenticated
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3172 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
d157201e08
*) IfesL for "Unexpected end of ZLIB" error message
...
See: http://www.yacy-forum.de/viewtopic.php?t=3327
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3169 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
0336480a3e
- the maxMemory-fix for the Sun JVM 1.4.2 wrongly also applied to 1.6, thx to NN
...
- added logging of reducing word-cache (log-level fine)
- disabled memprereq field in PerformanceQueues_p.html, because it is now set by the collections db
- minor changes to ConfigSkins / -Language
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3165 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
735c1f87b1
restart re-activated (windows)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3164 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
2c01508ada
*) fix for http://www.yacy-forum.de/viewtopic.php?p=29575#29575
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3162 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
6ff8359b98
possibility to use anonther bindPort than the externally reachable port.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3161 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
d2be3c674d
wrong cache values fixed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3159 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
df6281ba1f
- removed JS from DetailedSearch => valid
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3151 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
fb1d8b91af
*) changed Startpoints of IndexCleaner and IndexTransfer from ------------ to AAAAAAAAAAAA.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3150 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
d6eb699e8e
- fix for last commit (didn't know that the paragraph sign has an UTF-8-specific location)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3134 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
41bc31d2c2
- ConfigAdvanced_p => XHTML (no invalid IDs)
...
- removed unmappable characters from code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3133 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9b726ac366
release 0.50
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3132 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
036a0c828e
fix for auto-configuration of crawler thread memory
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3131 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
16f88c49d9
applied patch from http://www.yacy-forum.de/viewtopic.php?p=27392#27392
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3130 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2aa6f978c2
fix for http://www.yacy-forum.de/viewtopic.php?p=29125#29125
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3129 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
a4e90bc1dc
fix + debug-code for http://www.yacy-forum.de/viewtopic.php?p=29126#29126
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3128 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
6b5f28b746
answer for last commit: no
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3126 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
d98ba7bc33
fix for memory limit computation ?
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3125 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
49ffedfd8b
patch from Fuchs against deadlock,
...
see http://www.yacy-forum.de/viewtopic.php?p=28804#28804
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3122 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
a4f63d187d
better map2string and NullPointer fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3119 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c48374d14a
new memory limit computation for indexing queue
...
shall better prevent outofmemory errors
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3118 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
a9f1d3a065
*) SOAP: adding function to get the local peer hash
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3117 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
9659c004c3
fix for wrong date (Lotus-1 Peer)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3111 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
08ac4c5ed0
bugfix for http://www.yacy-forum.de/viewtopic.php?p=29045#29045
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3110 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
8e3bd17554
adopted DetailedSearch page to new ranking options
...
- fixed bug http://www.yacy-forum.de/viewtopic.php?t=3265
- more attributes on page
- attributes can be set as default for main search page
- option to re-set the attributes to built-in values
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3109 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
93a7e88245
more ranking parameter usage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3108 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2dbea612c9
fixed display bug for image search preview
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3107 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
0a050bc043
enhanced ranking
...
- redesign of data storage in plasmaSearchRankingProfile
- profiles are extended by new ranking parameters
- new RWI ranking parameters are considered during ranking
- appearance attributes (i.e. emphasised text) is now considered
- faster ranking
- some attributes that had been checked during post-ranking can now be
checked during pre-ranking phase
- removed old ranking parameter on index.html page (will be replaced by profiles in the future)
- ranking can now consider appearances of media content
- snippet-loading for media types now work correctly (fetches only from the wanted media)
- ranking-profiles can be handed over the remote peers and apply there also
- re-search of same query with different domain now also re-triggers remote search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3105 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d0c32c6aeb
better protection against fraud peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3104 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
61798f0ae6
added option to distinguish between text crawl and media crawl
...
- for each crawl start, there is now a flag for text and media
- the localCrawl flag is superfluous
- added new crawl profiles
- if an image search is done, only media links are crawled for the snippets
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3100 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
be941c4475
- "javascript:"-URLs are recognized as well (as intended formerly I assume)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3097 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
a619ba3f49
- fix for String index out of range during URL parsing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3096 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
febe6b114a
design update of crawler monitor
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3094 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
40049e0635
fixed media search snippet flow
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3092 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
5763d20795
- added DATA/yacy.running file to indicate whether YaCy is running or not. Helpful for i.e. the updater and linux init-scripts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3089 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
05d0464377
only do 16 checks if "address" starts with "172.";
...
better readably;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3087 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
rramthun
1a525710c1
*) cursor jumps now to searchbox on searchpages again
...
*) added missing private IP-ranges for APIPA/Zeroconf and 172.16.0.0–172.31.255.255
*) Changed some seed-download-errors to warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3086 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
e17591acc3
- parse HTML arguments as UTF-8 strings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3085 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
d30932c7d8
- fix for fix... sry
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3084 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
6118fb73ec
- added decode of UTF-16 escapes in url-arguments (%u0123), bugfix for http://www.yacy-forum.de/viewtopic.php?t=2762
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3083 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
782db9099d
version independent name for commons-pool lib
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3082 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7ff86d6ba6
- image search now shows thumbnails (in bad order, but it works)
...
- repaired DHT selection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3081 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ee3d91cb6b
print-out of links that result from contraint-filtering
...
in search result
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3078 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
f4737eebd6
Properties
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3076 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
1fc75c0f67
better logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3075 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e4570bffaf
-implemented a specialized snippet-fetch for media content
...
-changed search result preparation for media search presentation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3073 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
694a6e4f44
*) better text snipptes: any possible searchword (welt, linux, tag) in welt-linux-tag will be marked correctly now
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3072 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bddc197453
reverted by-mistake removed change from low012/SVN 3068
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3070 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
1377c53aa3
extraction of media links from search results
...
these links are mixed to the snippets for testing purpose
(a final version will handle this differently)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3069 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
586add4c6c
*) Better snippets: words like GNU/Linux will not prevent Linux or GNU from being marked if they are searchword (see http://www.yacy-forum.de/viewtopic.php?t=2891 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3068 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
29922d5dd2
changed writeMap
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3066 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
5b00a04359
changed writeMap
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3065 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
karlchenofhell
52abbd4131
- fix for wrong public IP if no hostname was set and IP was from range 192.168.*.*
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3063 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
8b7c543885
NullPointer fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3061 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
937ccd4e76
fix for snippet-generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3060 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
559f41a001
fix for http://www.yacy-forum.de/viewtopic.php?p=28607#28607
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3059 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
auron_x
c086c71f17
*) fixed ArrayIndexOutOfBoundsException
...
--> http://www.yacy-forum.de/viewtopic.php?t=3210
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3058 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c93cfdc23a
fix for http://www.yacy-forum.de/viewtopic.php?p=28564#28564
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3057 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
93a5ace330
fix for http://www.yacy-forum.de/viewtopic.php?p=28544#28544
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3056 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bf0d820659
- added correct flagging of word properties
...
- added self-healing to database in case that wrong free-pointers exist
- added presentation of media links in snippets (does not yet work correctly)
- code cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3055 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
10d888e70c
- added a media search for images, audio, video and applications
...
- new search options on search page
- new option in ViewInfo to display all links of a file
- enhanced collection data structure
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3054 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
a603c4d5e8
more code simplifications
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3052 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9a85f5abc3
cleanup
...
- removed 'deleteComplete' flag; this was used especially for WORDS indexes
- shifted methods from plasmaSwitchboard to plasmaWordIndex
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3051 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
fbe1ee402b
plasmaCrawlLURL$kiter cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3050 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
773ba1e91a
- generalized object order handling
...
- controlled object order for all database tables
- migrated DHT position computation to correct base64-decoded values
this also closed the 'gaps' in the dht positions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3049 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
15381cbf73
other bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3048 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
ad65cc9d2f
NullPointer fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3047 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
borg-0300
d33745a7ea
NullPointer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3046 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
3a4933b63c
bugfix for
...
http://www.yacy-forum.de/viewtopic.php?p=28493#28493
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3045 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
109ed0a0bb
- cleaned up code; removed methods to write the old data structures
...
- added an assortment importer. the old database structures can
be imported with
java -classpath classes yacy -migrateassortments
- modified wordmigration. The indexes from WORDS are now imported
to the collection database. The call is
java -classpath classes yacy -migratewords
(as it was)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3044 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
052f28312a
removed assortments from indexing data structures
...
removed options to switch on assortments
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3041 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
2372b4fe0c
release 0.49
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3040 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f8efb3c948
fixed a null pointer exception problem reported in the forum.
...
I cant find the forum entry any more because my girlfriend switched
off the power while the forum window was open.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3039 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ad1e4aa88e
added selection of audio, video, image and application resources
...
to search procedure. This function can currently not used through the
search interface, but only through remote search.
added accumulation of search attributes to enable the audio, video,
image and application selection.
fixed a problem with external URL representation generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3036 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
fb7902aa68
fix for http://www.yacy-forum.de/viewtopic.php?p=26142#26142
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3033 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7cc4cec9c9
bugfix for assertion bugs documented in
...
http://www.yacy-forum.de/viewtopic.php?p=28261#28261
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3030 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
97596af843
patch for wrong old RWI entries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3028 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
2c69cc969a
*) more special chars removed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3027 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
ebb42906f8
*) removed special characters
...
*) added Copyright comments
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3026 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
954db729db
*) Bugfix for ArrayIndexOutOfBoundsException during SSL detection
...
See: http://www.yacy-forum.de/viewtopic.php?p=28247#28247
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3025 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
5afd2f283c
fix for http://www.yacy-forum.de/viewtopic.php?p=27633#27633
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3024 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
9dfae90da8
better control of sortBound; should help to avoid an indexOutOfBound Exception
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3023 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
4ce590622f
- more asserts
...
- better memory usage during remove in kelondroRowSet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3022 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7dbcd358b4
fix for http://www.yacy-forum.de/viewtopic.php?p=28231#28231
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3021 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
86394e7a56
fix for cache-delete problem:
...
- better synchronization
- files are only deleted if they have been in the cache for 5 minutes
- hash-path for the HTCACHE is now default
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3018 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ceb9e3aa17
- enhanced parser: collection of audio, video, image and application links
...
- enhanced condenser: better handling of utf-8 and pre-formatted texts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3017 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
984285bdd6
better organisation of dns hit/miss cache flush
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3016 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
73c63578ad
- activated the dns miss cache
...
- added a cache-control for cache miss flush to the dns miss cache
- better naming of cache variables to distinguish hit- and miss- cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3015 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
eb20ec3837
*) soap-service: adding function to check if a specific url is blacklisted
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3014 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
0b9370a9dc
fix for http://www.yacy-forum.de/viewtopic.php?p=28108#28108
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3013 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b5a29e9651
- fix for snippets that are too short
...
- added keyword to snippet fetch to suppres removal of not-found snippet words (for debugging)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3009 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
13868ebb3e
reverted IP-patch in yacySeed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3007 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
allo
4cb688018d
wikiAdmin Recht
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3006 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
29fa17bd40
*) simplified some code in wikiCode.java
...
*) deleted outdated text in Settings_p-html (see http://www.yacy-forum.de/viewtopic.php?p=28027 )
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3005 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f1528672b1
filtering of non-index pages during index-of search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3004 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
0a0c3edeb6
fixed a bug in index transfer
...
- the encoding within the new entry format for binary data was wrong
- the string parser of RWI receive had to be enhanced
added some mor debugging tools
- a target peer for index transfer can now be selected by typing in the peer name
- the RWI result list has an entry counter
enhanced routing
- if communication is between two peers that have the same IP address,
the loopback address 127.0.0.1 is used instead the public IP
to contact the peer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3003 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
8e7215475b
- extended ViewFile to use is as debugging-tool: you can now use the
...
post-parameter url to submit an url directly
- fixed some bugs in text parser (not all parts had been analysed)
- fixed a bug in remote search interface (could not handle constraints)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3001 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
30888e7a2f
implementation of search constraints
...
Such constraints may formulate specific restrictions to web searches
This is implemented by scraping information for constraints from a web
page during parsing, and storing flags to the pages within the web index.
In this first step, only information for index pages ("index of", directory listings)
are scraped and stored in flags
- added new flag class kelondroBitfield
- added scraper method in condenser
- added bitfield structure for all scrape types (see also condenser)
- added bitfield structure for appearance locations (see RWIEntry)
- added handover protocol for remote search and index distribution
- extended kelondroColumn class to hold bitfield types
- added another search attribute on search page (index.html)
- extended search-filter to enable filtering of non-matching constraints
- set all new database types to be default
- refactoring: moved word hash generation to condenser class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2999 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
49a83f99d9
- fix for wrong DHT ordering in DHT selection
...
- fix for http://www.yacy-forum.de/viewtopic.php?t=3112&highlight=
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2995 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e55ef0df28
- automatic migration of old RWI entries to new format during remote search
...
if new collections are activated
- one more assert in RowSet, control of removeMarker
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2993 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f4b547dc13
limited index transfer to peer with version 0.486
...
this protects peers with version below 0.486 from new RWI objects
(which they cannot handle)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2988 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
10a4ab5195
disabled some (more) write caches
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2987 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
09bcc10344
bugfix for some problems of last change with assortments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2986 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e3d75f42bd
final version of collection entry type definition
...
- the test phase of the new collection data structure is finished
- test data that had been generated is void. There will be no migration
- the new collection files are located in DATA/INDEX/PUBLIC/TEXT/RICOLLECTION
- the index dump is void. There will be no migration
- the new index dump is in DATA/INDEX/PUBLIC/TEXT/RICACHE
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2983 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
58d79a9ac1
*) more error logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2974 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
73552db482
*) bugfix adding missing lib names to copy and zip commands
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2971 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
5c0669429e
*) soap: adding function to query the peer list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2968 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
c9364246cc
introduced new RWI-Object.
...
This will be used for the final version of the collections.
The new object is not yet used.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2966 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hydrox
f442af956c
*) first version of build-in logalizer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2965 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
9ea9a9f157
*) bugfix for bookmark service: get didn't work when using date=null
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2963 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b94ee53958
some enhancements to url retrieval
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2959 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
203f2bde9a
*) adding function to query the pause/resume state of the crawling queues
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2958 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d34f10c63d
some tests with reverse dns lookup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2954 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
24ac4e8860
Bugfix to "-UNRESOLVED_PATTERN- bei Hostname-Änderung" ( http://www.yacy-forum.de/viewtopic.php?t=3093 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2952 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
e628d34e16
patches for bad data
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2951 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
497428c8ec
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2949 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
a51417d86b
Bugfix: language of ConfigLanguage_p.html was not changed properly when a different language was choosen here
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2948 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
4147c9f490
*) better error logging on soap engine initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2946 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
76fceb9997
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2945 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
f77d624b94
*) bugfix for persistent connection support on transfer-encoded requests
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2942 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
eeda881553
bugfix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2938 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bb7d4b5d5e
refactoring to prepare new RWI entry object
...
- moved all url and index(RWI) entries to index package
- better naming to distinguish RWI entries and URL entries
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2937 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
6412c926bc
new url entry interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2936 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
bdc9216366
- more asserts
...
- some bugfixes
- some patches for bugs that are already in the database
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2935 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
low012
b18273bbb2
*) YaCy will extract URLs from <embed> tags now.
...
*) YaCy will extract URLs from <param> tags now if the name of the parameter is "movie".
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2934 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
1751a799ac
- deactivated all write buffers
...
- fixed a storage bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2933 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ba967c4875
- bugfixes and debug code
...
- ne generalized index class indexCachedRI
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2930 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
eaad91d84f
fixed wrong RAM calculation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2928 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
ee4715a21c
- more asserts
...
- bugfix for performaceMemory
- refactoring of index ram cache: renamed indexRAMCacheRI to indexRAMRI, to make space for a cached indexRI, which should be named indexRAMCacheRI
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2925 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
114a76a86e
- added flag to urlhash that shows that domain is a local domain
...
- enhanced local domain detection
- bugfixing for memory assignment in kelondroFlexSplit
- automatic memory assignment to caches according to available RAM
- bugfixes for details during search process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2924 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
eafb5ecd22
- better usage of memory resources for kelondroFlexSplit
...
- kelondroFlexTables does always load a RAM cache if it has enough
ram assigned. Othervise it creates a kelondroTree file-index.
If more memory is re-assigned, the file-index is deleted again,
and RAM is used. Beware that assignement of too less RAM forces
creation of file indexes and start-up time may last for hours.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2923 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
b2d51be33c
bugfix for latest changes to entry generalization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2922 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
(no author)
e59ff8b657
Bugfix: language of ConfigBasic.html was not changed properly when a different language was choosen here. Note: there's a similair bug on ConfigLanguage_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2921 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
892b9f2fc4
*) additional soap function to query peer status
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2920 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
bd3710a974
*) new xml template to view peer profile as xml
...
*) bugfix for wrong profile display (some fields where displayed twice)
*) new soap functions to get and set peer profile
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2919 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
d1afe1ce6b
*) adding xml template to get the message list as xml
...
*) Bugfix in client stub jar generation (too many files where added)
*) new soap service to manage peer messages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2918 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d454ca44ee
update of cache logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2917 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
hermens
8385557672
Small fix for the Cache Monitor when using proxyCacheLayout=hash
...
see: http://www.yacy-forum.de/viewtopic.php?p=27394#27394
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2916 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
f37e2041e8
*) adding soap function to import yacy bookmarks from xml or html (transfered via soap attachments)
...
*) soapHandler: code cleanup for service deployment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2915 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
f1ed55a5fc
bugfix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2913 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
8fdefd5c68
generalization of payload definition of index storage
...
this is one step forward to the migration to a new collection data format
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2912 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago