orbiter
18d9e1a256
fix for http://www.yacy-forum.de/viewtopic.php?p=10026#10026
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@768 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
1dd7047af5
finals;
...
cleaned;
Properties;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@767 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ef85fce661
change of memory-consumption constants (had been much too low)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@764 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
07f30931ec
various configuration options in memory performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@763 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
2f732e32a2
enhancements to memory menue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@762 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e17df64b54
removed IS_ADMIN - feature. This was covered by plasmaSwitchborad.adminAuthenticated
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@760 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b5337a122c
some more information about available memory in PerformaceMemory menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@759 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
96a5b6e8fb
removed yacy peer types from serverSwitch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@758 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b990dc1ad1
*) Replacing jsch 0.1.19 lib with newer version 0.1.21
...
*) Replacing PDFBox 0.7.1 lib with newer version 0.7.2
*) Refactoring of classes httpd/httpc/httpHeaders to
make many methods for httpHeader/Requestline parsing
reusable for new icap implementation
*) adding chunked input stream support
- needed by new icap implementation
- needed by future httpc HTTP/1.1 support
*) httpd.java
- moving all connection property contants to class httpHeader
- moving readHeader function to class httpHeader
- moving parseQuery function to class httpHeader
- moving handleTransparentProxy function to class httpHeader
*) httpHeader.java
- adding new fuction to parse the http response line
- adding new function to converte http headers to a string that
can be send to the client
- adding a function that generates a proper url using all parsed
connection properties
*) ICAP Support
- yacy now supports handling of icap response modification requests
- this feature can be used by other icap enabled proxies to contact
yacy as icap server, and to handover the downloaded content to yacy.logging
for indexing
- functionality was successfully tested with squid 2.5Stable 10 + icap patch
- further icap services e.g. URL filtering based on yacy's blacklists are possible
*) plasmaSwitchboard.java
- htcache entries that are still needed for indexing are now properly registered
as in use after system restart
- extended logging: log message now shows parsing and indexing time for each sb. entry
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@757 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
6d1de8abfd
finals; cleaned;
...
Properties;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@756 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
e748ba3f6e
super(), finals;
...
other; cleaned;
Properties;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@755 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
af9021e956
fixed bug with news caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@754 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
14bc880fa4
fixed bug with crashed profile database
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@753 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
71a31f0902
integrated and extended new memory performance menu; found and fixed bug in DHT caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@752 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
fb52a82008
added new performance page for memory settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@751 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
9dfbd93c7b
Updated german language file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@748 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
e6b30911c3
small changes to caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@747 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
42cd2cea65
added final constants, so that other class can reach it;
...
cleaned;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@741 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
900ab97422
change of memory-allocation blocking value for GC prevention
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@740 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0ffca99886
added priority-organization to kelondroRecord cache. This should virtually double the cache capacity.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@738 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
2d22626386
automatic switch-off of cache control in kelondroRecords in case that cache is big enough (so that no cache-aging needs to be controled)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@737 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
50a9500035
fixed 100% CPU bug with news queue deletion
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@735 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
cddd9aaa33
fixed SERIOUS bug with kelondroStack; affected all stack processing since 729
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@732 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
416c126815
fix for a profile = null problem and new monitor in crawl queue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@730 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
19547f1821
changed node manipulation methods in kelondro core to reduce object allocation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@729 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
debb207a74
removed file sync
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@725 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
2148c0cf49
replaced kelondro storage core; much less objects in kelondro cache now; less IO from DB
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@724 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
dff96601fe
*) Bugfix for transferURL:
...
URL list index was not incremented properly.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@723 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
beefddf0e8
*) Adding option which allows to do a Index-Transfer without deletion of index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@722 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
4036ee812a
Updated german language file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@721 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
40925f4fb7
*) Improving complete index transfer performance by automatically increasing size of transfered word chunk
...
for fast connections (much similar to normal dht behavior)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@719 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
91ab4d044b
*) Adding automatic retry functionality to complete index transfer function
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@718 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a62677f761
*) Adding additional logging output for complete index transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@717 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
10e7d6f02b
Bugfix for http://www.yacy-forum.de/viewtopic.php?t=1053
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@713 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b991d2e7dd
*) Additional logging message for complete index transfer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@712 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3c00c5f6c7
*) Complete Index Transfer
...
See: http://www.yacy-forum.de/viewtopic.php?p=9622
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@711 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f783061414
*) Changing redirection code from 307 to 302
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@710 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a6a8af0f04
*) httpdFileHandler templateCache can now be disabled
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@708 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
2cb084d426
*) Complete Index Transfer
...
See: http://www.yacy-forum.de/viewtopic.php?p=9622
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@707 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d1de71e9f6
*) Suppress stacktrace on proxy error for "No route to host Exception"
...
See: http://www.yacy-forum.de/viewtopic.php?t=1153
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@704 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
56160cbd01
*) Bugfix for "YaCy verzählt sich ..." Bug.
...
See: http://www.yacy-forum.de/viewtopic.php?p=9559
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@701 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
43b42854a0
fix for null-entries and http://www.yacy-forum.de/viewtopic.php?p=8649
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@699 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
801e902795
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@698 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
019cc716db
*) Undoing last changes on yacySeed. Seems not to work properly.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@697 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
177e8af5b7
*) Bugfix for ConcurrentModification in kelondroAbstractRA.writeMap caused by yacySeed.getMap()
...
See: http://www.yacy-forum.de/viewtopic.php?p=9523
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@695 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d3c923e6b9
*) Bugfix for "ConcurrentModificationException in hello.class"
...
See: http://www.yacy-forum.de/viewtopic.php?t=723
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@694 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
02c242ae22
minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@693 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
44b36d087e
*) Implementing a Cache for the servelet template files (.html)
...
should help to reduce IO
See: http://www.yacy-forum.de/viewtopic.php?t=749
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@690 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
394b4440d2
*) adding isLoggable function to serverLog class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@689 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
73ded2f0b6
*) Trying to fix bug for Seed-Upload-Failed
...
Bug may be caused because of timing issues
See: http://www.yacy-forum.de/viewtopic.php?p=9439
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@688 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3587407039
*) Fixing problems of list operation if index and queue size are both 0.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@687 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
51b48a10e8
*) Suppress stacktrace on proxy error for "ValidatorException: No trusted certificate found"
...
See: http://www.yacy-forum.de/viewtopic.php?t=1110
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@686 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7809b382bf
*) Bugfix for Blacklist support for https (only initial connect)
...
See: http://www.yacy-forum.de/viewtopic.php?p=9419
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@684 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7fe8784231
*) URLs pointing to a server having a private ip addess will not be indexed anymore
...
See: http://www.yacy-forum.de/viewtopic.php?p=9408
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@682 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
7c86c36210
undoing one part of the last commit. do not know, why it didn't work...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@681 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
a79913c6ea
updated german language file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@680 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
718950c5da
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@679 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
0aafb83edc
*) Bugfix for robots.txt isDisallowed Check.
...
Setting path to "/" if it is null or empty.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@677 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
2cd695f376
*) Bugfix path-entries of robots.txt were not decoded correctly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@676 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
8260128ee9
changed getFreeSize();
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@675 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f8ad65eae1
*) First trial implementation of robots.txt support
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@674 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9444852896
*) Correcting problems if the port number was set to -1, e.g. because of the usage of function
...
URL.getPort()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@673 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
0a57fbcde5
Added new HashSet filesInUse;
...
Added new Function getFreeSize();
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@672 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
8cd6a52dd0
Convention
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@671 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
c0e3d18bbf
*) remove import java.lang
...
*) Added Super()
*) replaced startsWith()
*) cleaned
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@670 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
b1cd1fa917
cleaned
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@669 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
da9c6857fb
*) changed a misunderstand, no BUG ;)
...
*) finals and other
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@668 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
fbac053c03
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@665 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
578f36ae18
*) Speedup of indexer. Proxy files will not be enqueued by the cachemanager
...
into the sb-queue anymore if the mimeType or fileExtension is not supported
by the installed parsers.
- Advantage: Avoiding unnecessary enqueueing and dequeueing from queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@664 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
1219ef99f0
*) Bugfix for NullpointerException in yacyDebugMode Init
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@663 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
9300689dde
bugfix *gr*
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@662 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e09f1fe8e4
*) IfsL: Suppressing stacktraces on further proxy errors
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@661 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6c722706b7
*) Moving yacyDebugMode intialization to switchboard
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@660 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
ebc39a7b9a
minor fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@659 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
e19ededd66
*) prevents problems due to HTML code in profiles (my posting from 05.Sep.2005 http://www.yacy-forum.de/viewtopic.php?t=964 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@657 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4e07828807
*) httpdProxyHandler.java
...
- harmonizing proxy exception handling
- adding malformed URL + blacklist check for http head method
- adding malformed URL check to http post method
- chunked encoding is now not used anymore for http post if clients
are http/0.9 or http/1.0 clients (same behaviour as already implemented for get)
- now an exception will be thrown on internal httpc errors to force an error output
to the client or a connection close. This should help to fix the "binary data in browser window" bug
*) plasmaSwitchboard.java
- fixing the following Bug
E 2005/09/03 18:02:42 PLASMA Could not index URL http://mis04.de/FAIL/snot.php : null
java.lang.NullPointerException
at de.anomic.plasma.plasmaSwitchboard.processResourceStack(plasmaSwitchboard.java:1000)
at de.anomic.plasma.plasmaSwitchboard.deQueue(plasmaSwitchboard.java:625)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at de.anomic.server.serverInstantThread.job(serverInstantThread.java:95)
at de.anomic.server.serverAbstractThread.run(serverAbstractThread.java:243)
This bug could occure if the cached responseHeader is null
- getting the mimeType now from the parsed document instead of the responseHeader because the
mimeType could have been changed during content parsing (e.g. because of the mimetypeParser)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@656 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
f90f699ab1
missing package line.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@655 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
89d7c6e45e
*) Bugfix: offset of 1 prevented correct replacement of HTML
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@654 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a7256e8f4e
*) Adding X-Forwarded-For Header
...
See: http://www.yacy-forum.de/viewtopic.php?t=1118&highlight=xforwardedfor
*) httpc.java: Bugfix for incorrect http response statuscode parsing
In some situations the statustext whas chopped
*) Adding a lot of fileheaders containing YaCy copyright and license
*) httpd.java: Adding additional debugging http header that should help du detect
the "binary data in browser window" bug.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@653 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
06a451768f
a simple robotsParser.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@652 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
81cb8feb15
back to 649 :/
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@651 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
5194511e8e
*) attempt to find bug
...
See: http://www.yacy-forum.de/viewtopic.php?t=1121
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@650 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8f3d19b331
*) Suppress stacktrace on proxy error for "Connection reset"
...
See: http://www.yacy-forum.de/viewtopic.php?t=1107
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@646 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6991b9e2b9
*) Suppress stacktrace on crawler error for "Connection reset"
...
See: http://www.yacy-forum.de/viewtopic.php?p=9071
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@645 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a20814291f
*) Bugfix for "Race condition zwischen httpc und switchboard"
...
See: http://www.yacy-forum.de/viewtopic.php?p=9036
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@644 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
286853fd39
*) Bugfix for "YACY hängt sich beim Beenden auf" Bug
...
See: http://www.yacy-forum.de/viewtopic.php?p=8997
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@643 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a47f9238fe
*) Blacklist is now also used by the crawler
...
See: http://www.yacy-forum.de/viewtopic.php?t=1069
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@642 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
dc0a2d4c11
*) Bugfix for Loader Queue:
...
Job count was not displayed correctly
*) IndexingQueue:
- now it's possible to delete single entries from the queue
- now it's possible to clear the whole queue
See: http://www.yacy-forum.de/viewtopic.php?t=995
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@641 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
732a107160
*) Bugfix for "-UNRESOLVED_PATTERN-" Bug on IndexCreateWWWLocalQueue_p.html and "urlEntry.url() == null" Bug
...
- Logging message for "urlEntry.url() == null" is now displayed as info
- IndexCreateWWWLocalQueue_p.html now detects null entries while looping throug the list and removes them automatically
See:
- http://www.yacy-forum.de/viewtopic.php?t=532#8781
- http://www.yacy-forum.de/viewtopic.php?t=639
- http://www.yacy-forum.de/viewtopic.php?t=1071
- http://www.yacy-forum.de/viewtopic.php?t=338
- http://www.yacy-forum.de/viewtopic.php?t=980
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@640 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
33aaffbfc6
*) Displaying content size of each entry in indexing queue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@639 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
022c1ab179
performance fix for yacyDebugMode and useYacyReferer.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@638 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
286442fbc5
do not Use YaCy-Sites as Referer, if useYacyReferer = false
...
http://www.yacy-forum.de/viewtopic.php?p=8896#8896
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@637 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f9eb550fbc
*) Bugfix for NullpointerException in serverAbstractSwitch.setConfig
...
See: http://www.yacy-forum.de/viewtopic.php?t=692#5575
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@636 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
7626823519
BUGFIX for last 'commit'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@635 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
971756e8dd
the delete size is smaller
...
See: http://www.yacy-forum.de/viewtopic.php?t=1084
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@634 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
0471019606
*) IndexCreateIndexingQueue_p.html now also shows indexing jobs that are currently in process
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@633 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3dc6845bef
*) Logging error message to logging output if no errormessage can be send to the user by the proxy
...
Note: This is only done if you set the logging level of PROXY to FINE
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@632 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
6d43a4970c
small changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@631 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3df5c7a6cf
*) Displaying an proxy error page instead of a white page if the server has closed
...
the connection before yacy was able to receive the http response line
See: http://www.yacy-forum.de/viewtopic.php?p=8866#8866
http://www.yacy-forum.de/viewtopic.php?t=704
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@630 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
cc493ef8c1
Added change from Hermes
...
See: http://www.yacy-forum.de/viewtopic.php?t=1050
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@629 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4edb5b6f1e
*) Bugfix for "ProxyAccess logging" Bug
...
Loglevel was not set corretly for Proxy.access logger
See: http://www.yacy-forum.de/viewtopic.php?p=8875#8875
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@628 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
7a7254713d
*) Moving Logging directory per default to DATA/LOG
...
See: http://www.yacy-forum.de/viewtopic.php?t=940#7656
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@627 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bead8a32aa
*) IndexCreate_p.java:
...
Crawler StartURLs will now also added to the errorURL-DB if an error occures on this url
*) kelondroStack.java, plasmaSwitchboardQueue.java
Adding method which returns a list of all entries in the queue. This list is used by IndexCreate_p.java
instead of an iterator to display the indexing-list.
Advantages: avoid concurrent modifications of the list while displaying it.
Speedup because now we have to access only one sync function instead of multiple ones
(one for each entry)
*) IndexCreateIndexingQueue_p.java
Using new list() function of plasmaSwitchboardQueue
*) httpdFileHandler.java
If a servelet returns the special value "LOCATION" the httpFileHandler does a Redirection of
the Browser to the URL specified by the servelet. This can e.g. be used when a http get request is
used insead of a post request, but a refresh should not be allowed.
*) IndexCreateWWWLocalQueue_p.html
Now it's possible to delete single entries of the local crawler queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@626 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
48aaf703cc
*) Adding additional logging output to detect crawling problems
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@625 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
59b8a98c7e
*) Bugfix for suppressing of stacktrace in log on crawler error "MalformedURLException"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8840
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@623 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
2ee4f9f4e5
*) added escape characters [= and =] to yacyWiki
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@622 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
c1d7527929
better cache cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@621 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
2e6df95786
*) adding toString method
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@620 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
ebbd063c92
*) Making mimeTable static final
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@619 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
60e77dcc60
*) Adding additional loglevel finer + finest according to Thread http://www.yacy-forum.de/viewtopic.php?p=8750#8750
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@618 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
fa54b5f38d
cleanup spaces
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@617 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4fd5b95b1f
*) Renaming Logger function names to reflect the proper Java Logging API Loglevels
...
- please use logFine instead of logDebug
- please use logSevere instead of logFailure and logError
See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@615 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6adf8a4bde
*) Renaming Logger function names to reflect the proper Java Logging API Loglevels
...
- please use logFine instead of logDebug
- please use logFailure instead of logError
See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@614 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
f19c09b227
*) Suppress stacktrace on crawler error for "MalformedURLException"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8733#8733
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@613 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
cc1df08069
*) Adding missing synchronized blocks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@608 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
0dfa8b62e2
*) Changing Proxy-Useragent string according to thread http://www.yacy-forum.de/viewtopic.php?p=8183#8183
...
A typical useragent string now e.g. looks like:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10; YaCy 0.401/00602; yacy.net) Gecko/20050716 Firefox/1.0.6
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@607 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
bf14e6def5
*) proxyCache, proxyCacheSize can be changed under 'Proxy Indexing'
...
- path now are absolute
*) move path check from plasmaHTCache to plasmaSwitchboard
- only one path check when starting
*) small other
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@606 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3dfda1c9da
*) More verbose output on ftp-seed-upload failure
...
See: http://www.yacy-forum.de/viewtopic.php?p=8000#8000
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@605 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e3aa3a2d75
*) Bugfix for ProxyAccess Logger
...
URL was accidentally logged without the parameters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@604 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9b818b1ce3
*) Pausing Crawlers if there is not enough space on disk
...
See: http://www.yacy-forum.de/viewtopic.php?p=8648
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@603 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b33094e925
*) Trying to solve "Too many open files bug"
...
*) Temp.Bugfix for "Bug in Index Restore"
See: http://www.yacy-forum.de/viewtopic.php?p=8647#8647
Orbiter: Please take a look
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@602 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b67f008eb8
*) Trying to solve "Too many open files bug"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@601 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
34790acf02
*) Bugfix for suppressing of stacktrace in log on crawler error "unknown host"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8615#8615
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@600 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
a812fb86cc
*) Port Forwarding Feature does not detect broken connection properly.
...
Therefor a test-request was added to the isConnected function to detect broken connections
and to keep open connections alive
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@596 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
af7b8f75bd
*) Making proxyAccessLogging configureable via yacy.logging file
...
- logging can be disabled now
- logging directory / filelimit / rotation count can be configured now
See: http://www.yacy-forum.de/viewtopic.php?t=965&postdays=0&postorder=asc&start=30#8280
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@595 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8132a44305
*) Better error handling if yacy SOAP extension is not installed
...
See: http://www.yacy-forum.de/viewtopic.php?t=1040
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@594 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
66ebce1109
use staticIP more often
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@592 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
2a081c9ee5
*) Adding additional logging message for "NURL.entry() == null" Bug
...
See: http://www.yacy-forum.de/viewtopic.php?p=8446
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@591 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
cb1f11c96b
*) Suppress stacktrace on crawler error for "Unknown Host"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8431
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@590 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e338a13de3
*) Suppress stacktrace on crawler error for "Read timed out"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8433
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@589 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
2e43e744de
*) Suppress stacktrace on crawler error for "connect timed out"
...
See: http://www.yacy-forum.de/viewtopic.php?p=8420
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@588 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
36cbe04e3e
*) Bugfix for Crawler Redirection Bug
...
See: http://www.yacy-forum.de/viewtopic.php?p=8422
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@587 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
b70de495a0
*) Remembering Crawler-isPaused setting
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@586 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
e569a84dc0
*) Using the same configuration settings for all indexing threads on server Startup
...
See: http://www.yacy-forum.de/viewtopic.php?p=8349
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@584 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
17be77a468
*) Bugfix for "Crawler data will not be removed from htcache if content parsing failed"
...
See: http://www.yacy-forum.de/viewtopic.php?t=965&highlight=ramdisk
*) Making ACCEPT_LANGUAGE configureable for crawler
See: http://www.yacy-forum.de/viewtopic.php?p=8327
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@583 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
858cb983d7
*) Printout date and system name on proxy error page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@581 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
4a2f06053f
*) Bugfix for "Gehäuselautsprecher" Bug
...
If de.anomic.server.logging.ConsoleOutErrHandler.ignoreCtrlChr is set to true all control chars except
space,tab,newline, are replaced with spaces
See: http://www.yacy-forum.de/viewtopic.php?p=5528
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@579 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
dc7c5237fb
*) Bugfix for "Gehäuselautsprecher" Bug
...
If de.anomic.server.logging.ConsoleOutErrHandler.ignoreCtrlChr is set to true all control chars except
space,tab,newline, are replaced with spaces
See: http://www.yacy-forum.de/viewtopic.php?p=5528
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@578 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
5f55dff297
*) Bugfix for "Binäre Nullen auf der page: Index Creation: Indexing Queue"
...
See: http://www.yacy-forum.de/viewtopic.php?p=6877#6877
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@577 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
cb97d2972e
*) Bugfix for "peer not accessible via .yacy name if Transparent Proxy Support is enabled" bug
...
See: per Browser nicht erreichbare Peers
*) Proxy Error Page now displays the Peer Name on top
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@575 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
60074b4301
more DebugMode(60 Secs Timeout)
...
needed for Yacy with tor.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@573 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
eb6365c069
local Bootstrapping bug.
...
use yacyDebugMode=true to allow local bootstrapping
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@572 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
330eae7cf3
*) Normalizing CrawlerStartURL now before crawling is started
...
*) CrawlWorker also does a URL normalization now before following the redirection URL
*) CrawlWorker removes redirection URL correctly from noticeURL stack now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@571 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
ab894d26bc
*) Bugfix for "plasmaSwitchboard.deQueue: null" Bug (hopefully)
...
See: http://www.yacy-forum.de/viewtopic.php?p=8135#8135
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@570 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
eaf9f26cc3
*) Bugfix for NULL PROFILE HANDLE 'null' Bug:
...
See: http://www.yacy-forum.de/viewtopic.php?p=7855#7855
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@569 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
0864ea367d
Added preformated changelog.txt
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@567 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
41aa3ae72e
provide a virtuell Headerfield IS_ADMIN.
...
This allows Serverlets to check Admin Status.
http://www.yacy-forum.de/viewtopic.php?t=1003
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@566 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
4cb382decb
Adding changes by borg-0300 from http://www.yacy-forum.de/viewtopic.php?t=997
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@565 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
low012
ecfdc4928a
*) better implementation of definition lists
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@564 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
26289e955c
Updates of the language file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@563 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago