Commit Graph

820 Commits (87ea300d710566912b8e3d82012f8da6836ee39d)

Author SHA1 Message Date
theli 525c8dcbd4 *) Adding Traffic Statistic for Crawler
19 years ago
theli 9a5ab62928 *) Adding yacy specific X-YACY-Index-Control header which can be used by clients
19 years ago
theli 539f9e57aa *) Disabling RemoteProxy if remoteProxyName is null or empty
19 years ago
theli 5a25ad9109 *) Bugfix for useRemoteProxy4YACY and useRemoteProxy4SSL check
19 years ago
theli 02d9af1a70 *) Restructuring and extending of Remote Proxy Support
19 years ago
low012 222607ef0f *) added <pre>-tag for preformatted text to yacyWiki (as requested by theli)
19 years ago
allo 117a424d00 bugfix for sharing png/gif files in WWW/SHARE
19 years ago
theli af3060938b *) Bugfix for manual peer ping functionality
19 years ago
theli 6e3201b74d *) Bugfix in httpc.java
19 years ago
borg-0300 58b670201d now, changed HTCacheSize needs no restart
19 years ago
allo f97c303ebd rights for Admin and Proxy.
19 years ago
allo 97de600a68 another bugfix for share/www.
19 years ago
allo df147d0fb0 rightsmanagement for upload/download User - preparations for proxy and admin User
19 years ago
allo 2dfd6bf36a fix for networkimage
19 years ago
allo ec10220d57 Fix for last Commit: .class Files in htroot, not in the dir of the localized HTML-Files
19 years ago
allo 4db2080188 Bugfix for www and share.
19 years ago
theli 40777556c5 *) Connection Tracking
19 years ago
borg-0300 bcb0d6d5ff changed setLastSeen(long rd) to setLastSeen();
19 years ago
borg-0300 cdbaf637fb added
19 years ago
borg-0300 e642a5d8b7 more constants
19 years ago
borg-0300 7b510b558a small change
19 years ago
borg-0300 ee7d847869 small change
19 years ago
borg-0300 d77b982083 small fix for last commit
19 years ago
borg-0300 b00cd5640b bugfix for 'hello.class java.util.ConcurrentModificationException:null'
19 years ago
rramthun a98bafb939 Changes to german language file
19 years ago
allo 43a127ff3a allow httpsTunnels to other Ports than 443. (if secureHttps=false)
19 years ago
allo 93c1cb9226 ipAuth works. thx theli.
19 years ago
allo 6430fa520e bugfix for broken HTDOCS
19 years ago
allo 4320425a17 ipAuth (this does not work yet)
19 years ago
theli 95abdeb685 *) Bugfix for nextElement function of URL Enumerator
19 years ago
orbiter 6260942590 changed search process: received indexes are now buffered and written to wordIndex after search
19 years ago
theli b7e21ec107 *) Adding DB import function which allows to import an foreign yacy DB (from directory PLASMADB)
19 years ago
borg-0300 7ee03acce0 new function cutUrlText added to shortens the URLs on IndexMonitor.html
19 years ago
allo b88a9584f8 New Errorpage
19 years ago
theli b177a80bb7 *) Bugfix for sendRespondError StackOverFlowException problem
19 years ago
theli 219acc1e8f *) Bugfix for wrong http version in response to http/1.0 requests
19 years ago
orbiter bc56a88cc8 further refactoring of search
19 years ago
allo 0f2f783e46 no no-cache for mediaExts
19 years ago
allo 7ca60f97bf localization Support for Includes
19 years ago
orbiter d29dfb0a12 refactoring of search / preparation for better search methods
19 years ago
allo 9ff2bd2f0e allow surfing, if no timelimit set.
19 years ago
theli 0ae166c522 *) Small changes to Index Transfer.
19 years ago
theli 461374e175 *) Restricting amount of files that yacy is allowed to open during index transfer/distribution
19 years ago
theli c8a35a0130 *) Adding new connection tracking page (currently only for incoming connections)
19 years ago
allo 141417821e limitless Accounts (set timelimit = 0)
19 years ago
orbiter b80b2fbdcc crawling peers now produce waves in network graphic
19 years ago
orbiter b45ffecd39 log to fix http://www.yacy-forum.de/viewtopic.php?p=11111#11111
19 years ago
orbiter 10d3627c90 changed word cache flush scheduling and removed possible locks
19 years ago
orbiter dced5c761e added 'corona' for crawling peers in network graphic
19 years ago
orbiter 4cf0d1eb6d less memory occupation in ImagePainter
19 years ago
allo 6bd9348bf1 small change
19 years ago
orbiter aae5af9686 imagePainter may now paint arbitrary large circles; small update to network graphics
19 years ago
allo be8e95d86a debug printlns removed
19 years ago
allo f1ff33177d reset Timelimits on Daychange
19 years ago
theli 54a97a7355 *) IfesL: Suppressing "Broken pipe" stacktrace in log file
19 years ago
theli a9e25c26e1 *) adding new sendRespondError method to httpd which accepts a template include file
19 years ago
allo 5605cc8018 TimeLimits
19 years ago
orbiter 6a72f06c40 resizable network picture + greater on click
19 years ago
orbiter 839db8869c added high/low priority for index adding
19 years ago
theli 3966b38360 *) Bugfix for userDB time limit calculation
19 years ago
theli 1688be8590 *) plasmaSwitchboard.java
19 years ago
theli e3a586d7bd *) Using serverByteBuffer instead of ByteArrayOutputStream
19 years ago
orbiter e9eb5e4b56 refactoring of index-entity join methods
19 years ago
orbiter 258fd9eb8e adding missing file for websearch refactoring
19 years ago
orbiter 77ae30063d refactoring of websearch process
19 years ago
orbiter 579b22d8ff small update to network drawing
19 years ago
orbiter 2b5829c3da small fix
19 years ago
orbiter e85989510a update to network image; added disconneced peers by disconnection time and changed colors
19 years ago
orbiter 4c7918f5b5 added shotdown to crawl stacker (moved from 882)
19 years ago
orbiter 2851658c2a re-integrated Martins last change to crawl stacker from svn 882 that I had deleted accidently
19 years ago
orbiter c83594528c integrated crawl stacker into thread control
19 years ago
orbiter 16a49c1c9d fix for graphics generation bug, see http://www.yacy-forum.de/viewtopic.php?p=10987#10987
19 years ago
orbiter d666b61b83 fix for news-deletion, see also http://www.yacy-forum.de/viewtopic.php?p=11000#11000
19 years ago
borg-0300 4180c422e8 cleaned, finals, Properties
19 years ago
theli 959eefbc4f *) Robots.txt parser/ppt
19 years ago
orbiter c1c94111b0 added new network picture at Network menu using the new image-servlet method
19 years ago
orbiter d98ed41f31 update to image painter and test on http://localhost:8080/imagetest.png
19 years ago
allo f65c939a60 userDB Auth
19 years ago
orbiter 5153ec0f3e update to image painter
19 years ago
orbiter 1b2db0b52a fix for file-share access; damaged some commits before by me :-(
19 years ago
orbiter 1a5d98cd6d better imagePainter example and fix for typo http://www.yacy-forum.de/viewtopic.php?p=10920#10920
19 years ago
borg-0300 afc5ef2819 cleaned, Properties
19 years ago
orbiter 60ab32c313 removed obsolete htmlPlotter
19 years ago
orbiter 06f4ac494a image-painter test; try with http://localhost:8080/imagetest.png
19 years ago
borg-0300 fb27428674 added restart to Status.html
19 years ago
borg-0300 0dcfc25d98 cleaned, finals, Properties
19 years ago
borg-0300 4191b21e73 cleaned, finals, Properties
19 years ago
orbiter f6cf3967de fix for compile-bug in svn 583 (Martin guck mal ob das richtig ist: fifo oder filo-stack?)
19 years ago
theli a2fa75e688 *) Asynchronous queuing of crawl job URLs (stackCrawl)
19 years ago
orbiter 01db66dc69 implemented image-servlets. the imagetest will stay there only for a limited time. Now images can be generated on-the-fly from servlets
19 years ago
orbiter f6a0e0f162 small bugfix to readFully
19 years ago
orbiter 6d5d0ac801 bugfix for startup problems
19 years ago
orbiter 5dc0d41900 bugfix in kelondroRA (hint by Martin)
19 years ago
orbiter 0c3a20d44f more + changed log for better understanding of outOfMemory bug and others
19 years ago
orbiter e5f8163203 fixed a bug with news; news moving could lead to shurtcut loop / 100% CPU; appeared when clicked on a 'Profile' news in Network menu
19 years ago
orbiter fbb5e36b80 documentation update
19 years ago
allo cd77078aa0 old Version restored before Release
19 years ago
allo a4b747fe97 ProxyAccounts based on userDB
19 years ago
theli 0fd9aa6c6e *) Bugfix: supportedFileExt Function didn't detect the file extension correctly because of missing conversion to lower case
19 years ago
theli 8a33c9b309 *) Bugfix: supportedFileExt Function didn't detect the file extension correctly if there was a dot
19 years ago
theli 28c5687ff9 *) Bugfix for "download of non supported file content" via crawler
19 years ago
theli 2b3f964037 *) Bugfix: supportedFileExt Function didn't chop http parameters before trying to detect the file extension
19 years ago
theli d388292f24 *) adding function for user accounting which is called after each http request
19 years ago
theli c7b7c69484 *) adding some functions to userDB.Entry
19 years ago
theli d9c08a47f9 *) Minor changes to userDB
19 years ago
theli af7829df25 *) adding some functions to userDB.Entry
19 years ago
allo ff1d3d0680 Init of userDB
19 years ago
theli 19648702cc *) adding iterator to userDB
19 years ago
theli dd59c1ace9 *) Adding first version of yacy user DB
19 years ago
theli 595e0c7e56 *) Bugfix for ProxyErrormsg: Wrong base URL
19 years ago
allo 9e1485c13b new Class for UserAccounts
19 years ago
orbiter 9c4306e41e fixed problem with htcache path
19 years ago
orbiter 25a59a51ad fixed problem created with last svn commit
19 years ago
orbiter 0054d3b1a6 added age in network menu
19 years ago
orbiter 3dd7e90cdd kbytes instead of bytes in performance settings; new default values
19 years ago
orbiter 1669eaaa1a fixed svn 805
19 years ago
borg-0300 ca82d690a9 changed in SVN 805 one line too much
19 years ago
borg-0300 4bb1f849a0 Bugfix for http://www.yacy-forum.de/viewtopic.php?t=1233
19 years ago
orbiter 2c7b490e30 memory-logging
19 years ago
orbiter 7fc822a59b changed handling of time-zones
19 years ago
theli 9b7f37fc37 *) Minor changes
19 years ago
orbiter 3c1d968d29 fix-fix for 792 and small changes in ftpc/download/dir experiments
19 years ago
theli b5a8992d29 *) Setting some object fields to final
19 years ago
theli 5f95a1cf62 *) Bugfix for ProxyErrormsg: Wrong http host header
19 years ago
theli 023be89586 *) Bugfix for "Robots.txt wird immer wieder geladen"
19 years ago
theli 35c6c5ead7 *) Bugfix for "Blacklist und Crawlen" Bug.
19 years ago
orbiter dc474aa22f various bug-fixes
19 years ago
orbiter 9e2fc7e5fe load balancing of crawl target domains
19 years ago
orbiter 3fcc95a82c integrated crawl-profiles db in memory-performance monitor
19 years ago
orbiter 72ce36baba cleanup in kelondroRecords
19 years ago
theli fe6a6abc0b *) Adding robots.txt db to Performance Settings for Memory menue
19 years ago
orbiter 3274ae725e increased cache size of robots database; however, this should be integrated into new memory control
19 years ago
orbiter c6d2f50375 changed order of robots and double-check
19 years ago
orbiter 68d5ff2ef1 added stringbuffer in condenser
19 years ago
theli 90f02ea455 *) removing metainfo from serverargs
19 years ago
theli 70a5681a4f *) Bugfix for inactive scp seed uploader
19 years ago
orbiter e380d4e55e cleanup (no functional change)
19 years ago
orbiter de0a58d79c no more sync
19 years ago
orbiter 495bc8bec6 removed cache-control from low and medium priority caches which reduces memory use and computation overhead
19 years ago
theli 1dc94e7753 *) Adding support for gzip content-encoding of http post requests
19 years ago
borg-0300 a1f5027a88 finals;
19 years ago
orbiter 979a3ee3c0 exceptions for better testing of bug http://www.yacy-forum.de/viewtopic.php?p=9852#9852
19 years ago
orbiter 18d9e1a256 fix for http://www.yacy-forum.de/viewtopic.php?p=10026#10026
19 years ago
borg-0300 1dd7047af5 finals;
19 years ago
orbiter ef85fce661 change of memory-consumption constants (had been much too low)
19 years ago
orbiter 07f30931ec various configuration options in memory performance
19 years ago
orbiter 2f732e32a2 enhancements to memory menue
19 years ago
orbiter e17df64b54 removed IS_ADMIN - feature. This was covered by plasmaSwitchborad.adminAuthenticated
19 years ago
orbiter 96a5b6e8fb removed yacy peer types from serverSwitch
19 years ago
theli b990dc1ad1 *) Replacing jsch 0.1.19 lib with newer version 0.1.21
19 years ago
borg-0300 6d1de8abfd finals; cleaned;
19 years ago
orbiter af9021e956 fixed bug with news caching
19 years ago
orbiter 14bc880fa4 fixed bug with crashed profile database
19 years ago
orbiter 71a31f0902 integrated and extended new memory performance menu; found and fixed bug in DHT caching
19 years ago
orbiter fb52a82008 added new performance page for memory settings
19 years ago
rramthun 9dfbd93c7b Updated german language file
19 years ago
orbiter e6b30911c3 small changes to caching
19 years ago
borg-0300 42cd2cea65 added final constants, so that other class can reach it;
19 years ago
orbiter 900ab97422 change of memory-allocation blocking value for GC prevention
19 years ago
orbiter 0ffca99886 added priority-organization to kelondroRecord cache. This should virtually double the cache capacity.
19 years ago
orbiter 2d22626386 automatic switch-off of cache control in kelondroRecords in case that cache is big enough (so that no cache-aging needs to be controled)
19 years ago
orbiter 50a9500035 fixed 100% CPU bug with news queue deletion
19 years ago
orbiter cddd9aaa33 fixed SERIOUS bug with kelondroStack; affected all stack processing since 729
19 years ago
orbiter 416c126815 fix for a profile = null problem and new monitor in crawl queue
19 years ago
orbiter 19547f1821 changed node manipulation methods in kelondro core to reduce object allocation
19 years ago
orbiter debb207a74 removed file sync
19 years ago
orbiter 2148c0cf49 replaced kelondro storage core; much less objects in kelondro cache now; less IO from DB
19 years ago
theli dff96601fe *) Bugfix for transferURL:
19 years ago
theli beefddf0e8 *) Adding option which allows to do a Index-Transfer without deletion of index
19 years ago
rramthun 4036ee812a Updated german language file
19 years ago
theli 40925f4fb7 *) Improving complete index transfer performance by automatically increasing size of transfered word chunk
19 years ago
theli 91ab4d044b *) Adding automatic retry functionality to complete index transfer function
19 years ago
theli a62677f761 *) Adding additional logging output for complete index transfer
19 years ago
rramthun 10e7d6f02b Bugfix for http://www.yacy-forum.de/viewtopic.php?t=1053
19 years ago
theli b991d2e7dd *) Additional logging message for complete index transfer
19 years ago
theli 3c00c5f6c7 *) Complete Index Transfer
19 years ago
theli f783061414 *) Changing redirection code from 307 to 302
19 years ago
theli a6a8af0f04 *) httpdFileHandler templateCache can now be disabled
19 years ago
theli 2cb084d426 *) Complete Index Transfer
19 years ago
theli d1de71e9f6 *) Suppress stacktrace on proxy error for "No route to host Exception"
19 years ago
theli 56160cbd01 *) Bugfix for "YaCy verzählt sich ..." Bug.
19 years ago
orbiter 43b42854a0 fix for null-entries and http://www.yacy-forum.de/viewtopic.php?p=8649
19 years ago
borg-0300 801e902795 small change
19 years ago
theli 019cc716db *) Undoing last changes on yacySeed. Seems not to work properly.
19 years ago
theli 177e8af5b7 *) Bugfix for ConcurrentModification in kelondroAbstractRA.writeMap caused by yacySeed.getMap()
19 years ago
theli d3c923e6b9 *) Bugfix for "ConcurrentModificationException in hello.class"
19 years ago
rramthun 02c242ae22 minor changes
19 years ago
theli 44b36d087e *) Implementing a Cache for the servelet template files (.html)
19 years ago
theli 394b4440d2 *) adding isLoggable function to serverLog class
19 years ago
theli 73ded2f0b6 *) Trying to fix bug for Seed-Upload-Failed
19 years ago
theli 3587407039 *) Fixing problems of list operation if index and queue size are both 0.
19 years ago
theli 51b48a10e8 *) Suppress stacktrace on proxy error for "ValidatorException: No trusted certificate found"
19 years ago
theli 7809b382bf *) Bugfix for Blacklist support for https (only initial connect)
19 years ago
theli 7fe8784231 *) URLs pointing to a server having a private ip addess will not be indexed anymore
19 years ago
rramthun 7c86c36210 undoing one part of the last commit. do not know, why it didn't work...
19 years ago
rramthun a79913c6ea updated german language file
19 years ago
borg-0300 718950c5da small change
19 years ago
theli 0aafb83edc *) Bugfix for robots.txt isDisallowed Check.
19 years ago
theli 2cd695f376 *) Bugfix path-entries of robots.txt were not decoded correctly
19 years ago
borg-0300 8260128ee9 changed getFreeSize();
19 years ago
theli f8ad65eae1 *) First trial implementation of robots.txt support
19 years ago
theli 9444852896 *) Correcting problems if the port number was set to -1, e.g. because of the usage of function
19 years ago
borg-0300 0a57fbcde5 Added new HashSet filesInUse;
19 years ago
borg-0300 8cd6a52dd0 Convention
19 years ago
borg-0300 c0e3d18bbf *) remove import java.lang
19 years ago
borg-0300 b1cd1fa917 cleaned
19 years ago
borg-0300 da9c6857fb *) changed a misunderstand, no BUG ;)
19 years ago
borg-0300 fbac053c03 small change
19 years ago
theli 578f36ae18 *) Speedup of indexer. Proxy files will not be enqueued by the cachemanager
19 years ago
theli 1219ef99f0 *) Bugfix for NullpointerException in yacyDebugMode Init
19 years ago
allo 9300689dde bugfix *gr*
19 years ago
theli e09f1fe8e4 *) IfsL: Suppressing stacktraces on further proxy errors
19 years ago
theli 6c722706b7 *) Moving yacyDebugMode intialization to switchboard
19 years ago
allo ebc39a7b9a minor fixes
19 years ago
low012 e19ededd66 *) prevents problems due to HTML code in profiles (my posting from 05.Sep.2005 http://www.yacy-forum.de/viewtopic.php?t=964)
19 years ago
theli 4e07828807 *) httpdProxyHandler.java
19 years ago
allo f90f699ab1 missing package line.
19 years ago
low012 89d7c6e45e *) Bugfix: offset of 1 prevented correct replacement of HTML
19 years ago
theli a7256e8f4e *) Adding X-Forwarded-For Header
19 years ago
allo 06a451768f a simple robotsParser.
19 years ago
borg-0300 81cb8feb15 back to 649 :/
19 years ago
borg-0300 5194511e8e *) attempt to find bug
19 years ago
theli 8f3d19b331 *) Suppress stacktrace on proxy error for "Connection reset"
19 years ago
theli 6991b9e2b9 *) Suppress stacktrace on crawler error for "Connection reset"
19 years ago
theli a20814291f *) Bugfix for "Race condition zwischen httpc und switchboard"
19 years ago
theli 286853fd39 *) Bugfix for "YACY hängt sich beim Beenden auf" Bug
19 years ago
theli a47f9238fe *) Blacklist is now also used by the crawler
19 years ago
theli dc0a2d4c11 *) Bugfix for Loader Queue:
19 years ago
theli 732a107160 *) Bugfix for "-UNRESOLVED_PATTERN-" Bug on IndexCreateWWWLocalQueue_p.html and "urlEntry.url() == null" Bug
19 years ago
theli 33aaffbfc6 *) Displaying content size of each entry in indexing queue
19 years ago
allo 022c1ab179 performance fix for yacyDebugMode and useYacyReferer.
19 years ago
allo 286442fbc5 do not Use YaCy-Sites as Referer, if useYacyReferer = false
19 years ago
theli f9eb550fbc *) Bugfix for NullpointerException in serverAbstractSwitch.setConfig
19 years ago
borg-0300 7626823519 BUGFIX for last 'commit'
19 years ago
borg-0300 971756e8dd the delete size is smaller
19 years ago
theli 0471019606 *) IndexCreateIndexingQueue_p.html now also shows indexing jobs that are currently in process
19 years ago
theli 3dc6845bef *) Logging error message to logging output if no errormessage can be send to the user by the proxy
19 years ago
theli 3df5c7a6cf *) Displaying an proxy error page instead of a white page if the server has closed
19 years ago
borg-0300 cc493ef8c1 Added change from Hermes
19 years ago
theli 4edb5b6f1e *) Bugfix for "ProxyAccess logging" Bug
19 years ago
theli 7a7254713d *) Moving Logging directory per default to DATA/LOG
19 years ago
theli bead8a32aa *) IndexCreate_p.java:
19 years ago
theli 48aaf703cc *) Adding additional logging output to detect crawling problems
19 years ago
theli 59b8a98c7e *) Bugfix for suppressing of stacktrace in log on crawler error "MalformedURLException"
19 years ago
low012 2ee4f9f4e5 *) added escape characters [= and =] to yacyWiki
19 years ago
borg-0300 c1d7527929 better cache cleanup
19 years ago
theli 2e6df95786 *) adding toString method
19 years ago
theli ebbd063c92 *) Making mimeTable static final
19 years ago
theli 60e77dcc60 *) Adding additional loglevel finer + finest according to Thread http://www.yacy-forum.de/viewtopic.php?p=8750#8750
19 years ago
borg-0300 fa54b5f38d cleanup spaces
19 years ago