theli
413e6b9855
*) direct access to responseheaders of sbQueue.Entry removed to make it more http independent
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2489 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
2126c51906
*) bugfix for ViewFile.java. Wrong http header were used
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2488 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
7a35b8e237
*) direct access to responseheaders of sbQueue.Entry removed to make it more http independent
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2487 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
3870d615e3
*) setting htCache.Entry fields to private
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2485 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
5847492537
*) next step of restructuring for new crawlers
...
- IndexCreate_p.java: correcting problems with ftp urls
- URL.java does not cutout the userinfo anymore
(needed to transport authentication info in ftp urls, e.g. ftp://username:pwd@ftp.irgendwas.de)
- plasmaCrawlLoader.java:
-- hack to re enable https urls
-- adding function getSupportedProtocols
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2482 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
4e2a950ac9
*) next step of restructuring for new crawlers
...
- avoid using the http crawler class directly. Using the interface class instead
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2476 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
f94131c13d
*) Bugfix for Blacklist_p.java
...
- avoid nullpointerexception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2475 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
eb9b138986
*) next step of restructuring for new crawlers
...
- conversion of the crawler pool into a keyed object pool
- crawlers are now loaded based on the url protocol (of course works only for http now)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2473 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
1395aae742
*) starting restructuring which is needed to add crawlers for additional protocols
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2472 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
857a2d76a2
*) better handling of server shutdown
...
See: e.g. http://www.yacy-forum.de/viewtopic.php?p=25234
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2471 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
7df572756a
fist step+attempt so solve the snippet marking problem.
...
See: http://www.yacy-forum.de/viewtopic.php?p=22855#22855
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2469 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
theli
f3ac4dbbb9
*) better handling of server shutdown
...
See: e.g. http://www.yacy-forum.de/viewtopic.php?t=2584
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2468 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
auron_x
57dda1a92c
*)again fixing for wrong version display, now totally working with double instead of float
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2464 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
18b6876860
new cache flush configuration settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2460 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
27a159b401
* documentation update
...
* removed doc from release
* release information in doc/News.html
* release 0.46
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2442 6c8d7289-2bf4-0310-a012-ef5d649a1542
18 years ago
orbiter
d78b824e85
fixed problem with default path after first start-up
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2440 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
c2264962d1
*) Fixed a bug.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2438 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
b387bd47db
*) Added a Wiki Code help page and links to it at every textbox Wiki Code can be used in. Maybe this will encourage people to play around with the Wiki and the Blog more.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2437 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
38c4248814
Some language updates
...
Removes the ; behind Slovenky in language list
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2430 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
23dd972608
fixed memory calculation in performanceMemory web page
...
fixed also maximum cache size computation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2429 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
1ce3c22761
better memory control:
...
- added memory monitor for preNURL-db in performanceMemory
- changed default memory assignments
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2427 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
39b4c26bdc
more memory control:
...
- catchup of OutOfMemoryError in server threads
- automatic adoption of word cache size after a Short Mem Cycle
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2426 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3e9d509c39
some small fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2425 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
479861a3cf
*) Value entered in searchfield will not get lost anymore when link "more options..." is clicked. If Javascript is disabled or function does not work for any other reason, there should be no change to current behaviour.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2424 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
16993e84dd
replaced non-7bit umlaut ascii
...
@Franz: sorry, ich hab hier ein ant das daran rumzickt. Bitte in deinem
Namen ein 'ss' statt der sz-Ligatur benutzen
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2422 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0187c60010
because of a bug in the JRE 1.4.2 there was no memory protection
...
see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4686462
this commit fixes the bug by using a memory-computation patch.
All uses of Runtime.maxMemory had been replaced by serverMemory.max
The bug is not present any more in Java 1.5
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2419 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
4eca0f8830
*) fixed PPM calculation for multiple indexer-threads
...
*) fixed totalPPM calculation and added total PPM to Network.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2418 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
6ad471ef96
* applied many compiler warning recommendations
...
* cleaned up code
* added unit test code
* migrated ranking RCI computation to kelondroFlex and kelondroCollectionIndex
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2414 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
9add56dc09
*) Cursor gets placed in search field when page is loaded. If Javascript is disabled or function does not work for any other reason, there will be no change to current behaviour.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2411 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
34831d2d9f
*) Check validity of crawl filter reg.exp. before adding it into the crawler queue
...
See: http://www.yacy-forum.de/viewtopic.php?p=24671
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2410 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5e0b6f8f83
*) sorting peer name list on Blacklist_p.html
...
*) restructuring of sharedBlacklist_p.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2405 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6c8366aea1
*) Bugfix for blacklist import function
...
- wrong property name
- list was accidentally imported into a new blacklist file
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2404 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
d0000a15d6
adjusted opensearchdescription to match requirements of A9
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2403 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
248627ab92
added open search compatibility
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2401 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
eee44be602
*) adding an interface for customized blacklist classes
...
- now it's possible to use a customized blacklist engine
instead of the default one
- this can be done by configuring the property BlackLists.class
See: http://www.yacy-forum.de/viewtopic.php?t=2108
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2397 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
79f27c298e
*) next bugfix for blacklist
...
- added items were not displayed properly
- bugfix for disabled usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2395 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4abc04dac0
*) make Blacklist_p.html more failsafe if no blacklist file is available of the user has entered empty strings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2394 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
28c84e5b57
*) Bugfix for NullPointerException in Blacklist_p if blacklist directory is empty
...
See: http://www.yacy-forum.de/viewtopic.php?p=24563
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2392 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
66f1eb07d9
*) Bugfix for IllegalArgumentException in transferURL
...
See: http://www.yacy-forum.de/viewtopic.php?p=24560
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2391 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d2e8e76218
*) now it's possible to configure the yacy blacklist separately for dht, search, proxy, crawler
...
See: http://www.yacy-forum.de/viewtopic.php?t=2541
http://www.yacy-forum.de/viewtopic.php?p=24516
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2389 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f43c90fa98
fixed handling of null referer in crawlOrder
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2384 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
23a99b8283
Small changes to the language
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2383 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
abf22f6e60
removed url normalform computation from htmlFilterContentScraper.
...
This method was implemented in de.anomic.net.URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2377 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
718fbc2dae
enhancements in kelondroCollectionIndex:
...
* synchronized array and index objects
* auto-fix function for slightly corrupted index entries
* generalized internal access methods
also extended kelondroIndex interface to support ordering access
which is used in kelondroCollectionIndex for string comparisments
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2366 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ec5149ff3b
fix for busyCacheFlush detection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2365 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f58283def2
better control of index flush
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2364 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
80b6c90d54
enhancements to prevent blocking during dht transfer receive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2362 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
auron_x
4fb8fddd99
*)made the domainlist of the blacklist sorted
...
if a new domain is added it is still appended to the end of the list and sorted in with next refresh, may need a fix.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2361 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
d56f06401e
- Cache known URLs during indexReceive to avoid getting blocked during loadedURL.exists() whenever possible
...
- Small logging updates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2359 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c09f734d06
*) offer router configuration on ConfigBasic.html
...
- checkbox to allow router configuration is shown if
- a) the UPnP forwarder is installed
- b) a UPnP enabled router was found
- c) no other forwarder was configured
See: http://www.yacy-forum.de/viewtopic.php?p=24264
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2358 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago