Commit Graph

1490 Commits (314021453f0f37603fb18527860b701130849be2)

Author SHA1 Message Date
orbiter d0dd8b14d2 fixed picture tag and presentation
19 years ago
borg-0300 da6a8bafa2 rename currCacheSize -> curCacheSize;
19 years ago
borg-0300 92110aea32 nullpointer fix for profile(); other minor change;
19 years ago
orbiter f0833b0328 introduced simple search interface
19 years ago
orbiter bd283b8443 fixed bugs:
19 years ago
orbiter 47b541b2d1 added better option handling in yacysearch
19 years ago
orbiter c9e16bfd48 first try to insert image search (does not work yet)
19 years ago
orbiter f77775220b fixed parser error
19 years ago
auron_x 0bf2eb4813 *) fixed Blog updating timestamp on edit (now always keeps creation-date)
19 years ago
orbiter 22de954a57 added some log output to parser
19 years ago
orbiter 83e0e765ec redesigned some parts of the html scanner & parser
19 years ago
orbiter ac114d69c0 tried to fix some problems with time-outs during search
19 years ago
orbiter e2e8d0c188 some kind of refactoring of yacysearch:
19 years ago
orbiter 6b63e26cbb - removed search function from index.html/java, only imput left
19 years ago
orbiter bc3e80fe42 quickfix
19 years ago
orbiter d8d0ac29c3 added image-viewer servlet that can do:
19 years ago
orbiter ddc6394d9b fixed bug about auto-depth 0
19 years ago
orbiter 60351fa3f7 small fix to previous commit
19 years ago
orbiter a469874e3f added and fixed time-out behaviour during search
19 years ago
orbiter 1d0b0d6e2a synchronized local searched to prevent that several searches are performed at the same time
19 years ago
hermens 22b9d03bbf Correcting remaining time issue in getContainers
19 years ago
orbiter 5defe97ccb fixed a problem / possible bug with ordering implementation (did not rotate always)
19 years ago
orbiter d58788b753 added some synchronisation
19 years ago
orbiter e566d1d8d6 some bugfixes regarding new crawling options
19 years ago
orbiter c7f1300300 -fixes for last commit
19 years ago
orbiter f2421f6a47 some small attribut changes regarding cache flush
19 years ago
orbiter 7a650d0023 several bugfixes
19 years ago
borg-0300 245cc34d51 small fix for last commit
19 years ago
borg-0300 e2853f357d added more lowercase to url normal form generation
19 years ago
orbiter 59d52fb4a9 fixed some problems with crawl profiles
19 years ago
orbiter 4a5b5515b5 added a lowercase to url normal form generation
19 years ago
allo 44d72f06c4 more Caching
19 years ago
allo 918445a2f4 Bugfix for last commit.
19 years ago
orbiter 708cc6c8d9 fixed some bugs for auto-filter and added monitor in profile list
19 years ago
allo c58789177f bookmarkCache
19 years ago
rramthun 250864406f ...
19 years ago
rramthun 42b0b10a95 -Adding Windows Media to types which are not sended compressed
19 years ago
allo 2ed4fa96b7 tagCache
19 years ago
allo 330eb9c74f bookmarkDB cleanup
19 years ago
orbiter e82899ba57 fixed missing urls map initializer
19 years ago
orbiter 63f39ac7b5 added 3 new crawling steering options:
19 years ago
orbiter 1fc3b34be6 some pre-work (without function yet) to implement:
19 years ago
theli c9e6b5e391 *) check size of indexing-queue and crawler pool before processing remote triggered crawl jobs
19 years ago
orbiter 1509314ea6 set tighter control during DHT index and peer selection
19 years ago
hydrox fcc0683200 *) undoing last commit
19 years ago
hydrox 9411961eec *) another little fix for DHT-Transfer
19 years ago
borg-0300 77f3237de3 adapted for isListed()
19 years ago
hydrox 8b14a0c833 *) little fix for DHT-Transfer
19 years ago
orbiter 1f4412a146 adopted isListed to discussed new behavior as discussed (url, getFile)
19 years ago
orbiter 59fc55ea1e added checks to protect peers from wrong seeds
19 years ago
orbiter 063ef4660a bug?
19 years ago
orbiter 82358677a9 added another shiftK2W to flushCacheSome
19 years ago
orbiter 128e4ab199 - in serverSystem: maxPathLength is now a variable, not a method
19 years ago
orbiter a37b09e303 implemented automatic adoption of chunk-read-ahead in kelondroTree to needed chunk size
19 years ago
orbiter 30e3e3a0fd adopted MAXPATHLENGTH to host system capabilities
19 years ago
borg-0300 d808765087 something Javadoc
19 years ago
borg-0300 85bb8e32a1 Bugfix for last commit
19 years ago
borg-0300 3fe402069f try to fix
19 years ago
orbiter f16f1f15cd bugfix for 100% CPU bug; thanks to Matthias for analysis
19 years ago
borg-0300 254a13efd9 MAXPATHLENGTH used
19 years ago
borg-0300 8865948e4e Cleanup;
19 years ago
orbiter 6c70f4a0cf renamed wordHashes for a word hash set generation to wordHashSet
19 years ago
orbiter d5f8f40c31 removed correcting iterator
19 years ago
orbiter 526407f32e adoptions, fixes for last commit
19 years ago
orbiter 488a0ed580 replaced old keyIterator and rowIterator by buffered iterators
19 years ago
hermens 4e9a8f41fd rwiDBCleaner + dbImporter: Iterate over small excerpts of
19 years ago
hermens 474379ae63 remove TABs from plasmaDbImporter.java
19 years ago
orbiter dba02f399f starting of re-design of kelondroTree iterator
19 years ago
orbiter f02b426073 made kelondroTree.nodeIterator private
19 years ago
borg-0300 5f6fdf1786 Bugfix for getCachePath(URL url)
19 years ago
orbiter 303b6463a8 added debug line to URL storage for testing
19 years ago
orbiter 431a4f3609 eliminated correcting iterator in kelondroTree
19 years ago
orbiter 91dca2cd8d fixed a bug in last commit: LURL entries cannot be written,
19 years ago
orbiter 3286b1f498 re-organisation of lurl-creation and -stacking
19 years ago
orbiter 9cca36a107 no more strict comparator checking in map exclude method if not needed
19 years ago
orbiter 0b903c5317 removed usage of kelondroNaturalOrder from plasmaCondenser to experimental
19 years ago
orbiter 4239db0d1c fixed new ordering for backup iterator TreeSet
19 years ago
borg-0300 399538b7de Bugfix: wrongly compared
19 years ago
orbiter 33eba5ecb8 temporary disabling last change, does not work (cannot debug right now)
19 years ago
orbiter f0464042fc fix for latest iterator-replacement-fix:
19 years ago
borg-0300 ec21c585cb try to fix path too long
19 years ago
orbiter a6a3f4b694 fix for svn 1888
19 years ago
hydrox 8da13088e9 *)removed multiple DHT_Distribution_Threads
19 years ago
orbiter 283a7181c6 try to fix new 100% cpu bug, possibly caused by iterator method
19 years ago
theli 34c075c1c7 testcommit with subversive
19 years ago
orbiter f588c0724f removed cache flush in case of DHT receive
19 years ago
orbiter e94b374d56 update to cache flush method
19 years ago
orbiter bcd99fe83e introduced a second RAM cache for DHT transfer
19 years ago
hydrox 360a460da8 *)URL-Cleaner: moved logging-statement to correct position
19 years ago
orbiter 02f9765013 quickfix for time problem during cache restore
19 years ago
orbiter 69d26045eb quick-fix for http://www.yacy-forum.de/viewtopic.php?p=18805#18805
19 years ago
allo b3df5ce2b8 another local iprange
19 years ago
hermens ad119f06af *) Don't overwrite new entries with older ones
19 years ago
orbiter be88687d8c fixed some problems with new cache flush karenz
19 years ago
theli d3da7c9a08 *) Adding support for robots Allow directive
19 years ago
theli 734d18f283 *) more correct robots.txt validation
19 years ago
hydrox f046e1814a *fix or last commit
19 years ago
hydrox c55c51e2a8 *)added keywords to IndexCleaner_p.java
19 years ago
theli f0ad0d2b2b *) better robots.txt support
19 years ago
orbiter ddbeda738e added minimum age of word in cache to performance menu
19 years ago
orbiter f188611fc6 apply blacklist on rwis during dht receive
19 years ago
theli 915812f597 *) Undoing robots parser policy changes from svn rev. 1421
19 years ago
orbiter 0ec28b8f8e added DBCleaner from Hydrox
19 years ago
theli 5ee0125046 *) adding possibility to configure the server port for seed uploading via scp.
19 years ago
allo 2065138a41 reverted last Commit.
19 years ago
allo 7afa5c1b8e staticIP fix
19 years ago
theli fb4100d47b *) undoing last commit.
19 years ago
theli a84cc71218 *) removing getTotalRuntime
19 years ago
low012 1965a8cf70 *) bugfix for previous commit
19 years ago
low012 1c3b415c35 *) nicer anchors (TOC)
19 years ago
low012 6000a38a05 *) changed anchors (TOC) according to thelis request in http://www.yacy-forum.de/viewtopic.php?t=1987
19 years ago
auron_x dce08771d1 *) Fix for wrong estimated and elapsed times when import was paused
19 years ago
hermens b34713324a DBImport: remove words from source index even if nothing has been added to home index
19 years ago
orbiter 3237fe1cc7 added IOException for httpc client error
19 years ago
orbiter 520b60f15b fix for http://www.yacy-forum.de/viewtopic.php?p=18610#18610
19 years ago
theli c7ececbfb2 *) httpd.mime: adding jar mimetype
19 years ago
orbiter ae536c4608 fixed soap service: index.soap -> yacysearch.soap
19 years ago
orbiter bae3783d38 added a snippet marking
19 years ago
orbiter f0a38873eb * added yacysearch page with better view on search results
19 years ago
orbiter f0041d504d remove of several results from a single domain is stopped if the result set is smaller than the wanted number of results
19 years ago
auron_x b92a00172e *) modified Blog
19 years ago
allo 68ec572df0 bugfix for staticIP
19 years ago
theli 89286478e7 *) removing thread pool eviction for now. Not needed at the moment
19 years ago
theli 759800f543 *) Bugfix for storeHTCache problem
19 years ago
orbiter a4682e2810 fixed problems in basic config and added language setting
19 years ago
orbiter a8548c0484 * several bugfixes regarding basic configuration
19 years ago
orbiter 1b9b8922d9 * fixed problems with new basic 1-2-3 configuration (now authentication required)
19 years ago
auron_x 8c6f38fe70 *) added Blog to YaCy (atm not reachable through interface) -> Blog.html
19 years ago
orbiter c51603a405 added two new kelondro classes that will handle the new index data structures (not used yet)
19 years ago
orbiter ce5274c194 yacybot user agent
19 years ago
allo 3b7e66ab48 staticIP should now work
19 years ago
theli 8558000bd9 *) better handling of port forwarding
19 years ago
allo 32d4c3e4c6 static IP is proper
19 years ago
orbiter f96238e9be cosmetic
19 years ago
low012 d6cc0ec3d8 *) Fixed a bug: Images in tables were not displayed if alt or align attributes were added.
19 years ago
hermens 351bd0a678 *) dbImport: convert cacheSize to kb when creating plasma* objects
19 years ago
orbiter eaffcfefe2 * added more ranking attributes (without function; this will be added later)
19 years ago
orbiter 87e90b9d8c refinements in ram cache flush procedure and default timing
19 years ago
orbiter d31a4e0b4f some small enhancements with cache flushing parameters and data structures
19 years ago
orbiter 3703f76866 - fixed re-search bug: after a search with several words, a second search could not
19 years ago
theli fbbbf5f411 *) remote trigger for proxy-crawl
19 years ago
low012 f31b633b2e *) Fixed yet another small bug in the wiki. (no blank between image URLs, align and alt)
19 years ago
low012 8bdbf0dfff *) Fixed a bug in code for tables and hopefully eliminated XSS vulnarability which Allo pointed out to me.
19 years ago
theli dc9174c809 *) Implementing snippet fetching via ajax
19 years ago
theli 468ca5b0e6 *) Bugfix for url.toString problem in yacyClient crawlOrder
19 years ago
low012 62d0f7a334 *) added TOC for pages in wiki as requested in http://www.yacy-forum.de/viewtopic.php?t=1987
19 years ago
orbiter 1d8ca6e082 serialized dhtChunk deletion with indexing
19 years ago
allo ff476c19a5 tuning User.html
19 years ago
low012 2e8221050a *) removed redundancy, no changes in functionality
19 years ago
theli 2336f0f013 *) allow pausing/resuming of crawlJob Threads separately
19 years ago
allo af1da090bb blogRight in UserDB.
19 years ago
borg-0300 2ee5796e76 added keywords, properties;
19 years ago
orbiter 60dac4325e serialized indexing with dht selection
19 years ago
orbiter a840755964 moved parts of index transfer logic back to switchboard
19 years ago
orbiter 134253a603 fixed bug with cache flush
19 years ago
borg-0300 149409ba5c move description -> javadoc
19 years ago
borg-0300 00975359c5 added keywords, properties
19 years ago
orbiter c2d863855d different flush limit
19 years ago
borg-0300 64441b1f78 ADDED: yacy.badwords list to filter the topwords
19 years ago
allo d7da273d7e using ArrayList instead of Vector
19 years ago
allo d6f7bd44ae fix for public tags
19 years ago
orbiter f9063e2040 added some synchronization to avoid that several tasks can trigger a cache flush simultanously
19 years ago
orbiter 2c4e4ae6a2 further refactoring of dht selection, transfer and flushing
19 years ago
orbiter 73dad68cf1 outsourced thelis DHT flush class into own file
19 years ago
allo aa4b04e3dd reverted last change
19 years ago
allo 467216d4c7 bugfix for umlaute and html-specialchars in wikiCode
19 years ago
allo 4b0dae8fcf added a possiblity to get the ranking values for an url.
19 years ago
orbiter 85ac7d8386 * moved DHT transfer thread to own class file, needed for further modularization
19 years ago
orbiter 34341a868e code cleanup
19 years ago
orbiter 7df2e6e571 bugfix for last commit
19 years ago
orbiter cd41e9a0eb moved DHT index selection to new object that holds indexes to be send away to other peer.
19 years ago
allo e3dd67bba0 bookmarks import.
19 years ago
allo dda143df79 We NEED this here for the XML-Files
19 years ago
theli 42a5f56723 *) Bugfix for broken dht thread configuration
19 years ago
allo bfd37e34aa using other XML Parser
19 years ago
theli aa87df35e1 *) To avoid confusion location will now also be displayed for own peer
19 years ago
rramthun 15ed57f9b7 Updated German language, by VT100, NN, rramthun
19 years ago
allo 0b5a736280 bugfix for nullpointer on unparsable date.
19 years ago
allo f503df04f8 sort the Bookmarks with newest First.
19 years ago
allo 3b4a99ff6a fix for java 1.4.x
19 years ago
theli 651bce8e2f *) adding missing function to transmit url chunks for crawl-order jobs
19 years ago
theli f95d98142f *) displaying amount of items in the existsIndex caches
19 years ago
theli 9b941fb773 *) bugfix for usage of yacy with extended port binding (e.g. #eth0:8080, 192.168.0.1:8080, etc.)
19 years ago
allo 2d4e1325cf UTF-8 fix
19 years ago
hydrox e2af2a3f45 *) it's now possible to run more then one indexDistribution-Thread
19 years ago
theli 40dd6ec4fd *) experimental restructuring of db import function
19 years ago
hermens c8f5adea4d - don't send Message Body on HEAD requests, even in the case of an error
19 years ago
theli a7248fbb0a *) bugfix for http/0.9 responses
19 years ago
theli 2da18ab359 *) correcting logging output
19 years ago
theli a354bc2ec1 *) Bugfix for content length check
19 years ago
theli 8ffc6e35ad *) correcting logging output
19 years ago
hermens e974d0cb99 Improve compliance to rfc
19 years ago
theli 556d242be0 *) Limited support of content-range requests
19 years ago
theli 8fcb25f9f9 *) Setting via header according to rfc
19 years ago
theli 980e986b64 *) Re enabling short cycle for already removed nurl entries
19 years ago
orbiter 218cd6561c fixed problem with wrong hash length in file share
19 years ago
orbiter 56516fd8e6 added refresh parser to htmlFilterContentScraper
19 years ago
orbiter ba5fe0b287 fixed (again) nullPointerException in kelondroTree
19 years ago
allo e6c2f700b1 public Tagview
19 years ago
allo 672b384fc7 refactoring setTagsTable and setBookmarksTable.
19 years ago
theli 040624e361 *) better support for http head requests of servlets
19 years ago
theli 62ffb5ece0 *) httpdFileHandler.java: adding real streaming support for lage files
19 years ago
hermens 3b6328ad02 *) Consistent use of minCount for index transfer
19 years ago
orbiter 0017812b2e bugfix for null compare
19 years ago
allo b137c90015 BugFix for too long Usernames.
19 years ago
allo 9d6bae77ee the best PPM ever! ;-p
19 years ago
hermens 0b60b9bf51 *) Remove entries from AssortmentCluster before reinserting the rest into the ramCache
19 years ago
hydrox 8ab1d6ff4b *) fixed NullPointerException in plasmaWordIndexEntity
19 years ago
allo a26574c894 Migration from tagName as key to wordhash(tagName) as key for bookmarkTags.db
19 years ago
theli eeba8b055e *) guessing, testing and suggesting alternative hostnames on "unknown host" error
19 years ago
orbiter 7eb10675b3 re-organization of index management
19 years ago
allo a8a5ad959a using TAG_NAME for getFriendlyName() and the lowercaseversion for getTagName()
19 years ago
allo 73f18ed5b2 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1627 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo bc6d400b03 bookmarktags with wordhashs.
19 years ago
allo addc802cf3 workaround for kelondro Umlaut-Problem.
19 years ago
allo fb5c387368 XML import works.
19 years ago
allo a4513523d6 hide add/edit/import bookmarks per default.
19 years ago
orbiter 1e4578aab6 VERY EXPERIMENTAL removal of index ram cache flushing thread.
19 years ago
theli 44996afd79 *) Bugfix for handling of http/0.9 clients.
19 years ago
orbiter 01c668fe1b experimental disabling of db write buffer
19 years ago
hermens 954f02d22e *) Bugfix: Prevent wordIndex.getContainer() from returning and even manipulating
19 years ago
orbiter 4438a96b87 tried to fix "Object type or Object type combination not supported" bug
19 years ago
orbiter c70befbe9c added xml read/write example file: kelondroXMLTables
19 years ago
allo 27b6b3d714 public Tags.
19 years ago
hydrox a627162f13 *)fixed logginglevel for Debugmsg
19 years ago
orbiter fe39493145 changed default ranking parameters
19 years ago
allo 5ef57211a6 trim was just too easy...
19 years ago
allo 38e79aec94 Fix for wrong(?) httpheader:
19 years ago
allo a8eff9a0ae xml/bookmarks/posts/all.xml to list all public Bookmarks
19 years ago
allo d04e7d724e fix for deleting tags.
19 years ago
orbiter 365a3fff8e fixings for ranking attributes
19 years ago
orbiter 785763d070 added debug output for "Object type or Object type combination not supported" bug.
19 years ago
theli 2a88232cee *) Bugfix for httpd security but
19 years ago
orbiter 8e55098b74 fixed detailed search
19 years ago
theli ebc5b1eafb *) adding a servlet that can be used to generate a Firefox search-plugin for yacy.logging
19 years ago
orbiter 0cb940a8e5 added detailed search.
19 years ago
orbiter c695928f7c adopted search page to new detailed search (to be commited later)
19 years ago
orbiter 45323e7b76 fixed null pointer exception during search
19 years ago
allo 1fce0ea263 xml api for deleting bookmarks.
19 years ago
allo 6b16c13b20 showing the public/privat flag.
19 years ago
rramthun 49647583b2 language update
19 years ago
allo 670356fa4e code cleanup
19 years ago
orbiter fb7411d7bb re-structuring of ranking application:
19 years ago
hermens 5f5eee1ae9 *) replace System.out.println with log
19 years ago
orbiter d98418390b - introduced rankingProfile Class
19 years ago
orbiter eab1805bca refactoring: plasmaSearchProfile -> plasmaSearchTimingProfile
19 years ago
orbiter 6eef848954 re-design of post-ranking process
19 years ago
allo af2127f3ad allow multiple #File: Sektions for one File in translations.
19 years ago
allo 1bb768178f simple check for unused Translations.
19 years ago
orbiter be77fe1a88 code clean-up
19 years ago
orbiter 0bc2aaeb42 added normalization to search attributes
19 years ago
allo 7bd61ab0e5 Locales will now be in DATA/HTDOCS. So it works with readonly htroot.
19 years ago
theli 008bcb7fb8 *) simplifying code by moving closeTransferIndexes into final block
19 years ago
theli 50d85657b8 *) new import function for IndexImport_p.html
19 years ago
theli 214302284e *) undoing last commit because of problems with getUpdateTime
19 years ago
theli 408de3beee *) avoiding to search in the treemap two times for the same key
19 years ago
borg-0300 139ba4e0c8 Bugfix for getCachePath(URL url)
19 years ago
theli 442807cb29 *) Bugfix for last commit
19 years ago
theli 22fd1ca9aa *) minor changes
19 years ago
theli 6a99304b2b *) Redesign of db import functionality
19 years ago
hermens 56936139ae Fix logging in kelondroTree:
19 years ago
orbiter 3834675084 fixed bug that caused wrong behavior of search result preparation
19 years ago
hermens 31c8476b5d plasmaWordIndexCache.getContainer:
19 years ago
hermens bb1664b63e *) Remove workaround from SVN 1472: It is not needed anymore
19 years ago
orbiter ec5d88664a tried too fix serverSwitch synchronization problems
19 years ago
orbiter 3419b3bcdd fix for bug that caused the peer-counter problem.
19 years ago
allo 9bc62bfdac typo
19 years ago
allo cdc595ab67 another ppm fix. no more unknown ppm
19 years ago
hermens 4f43816ec0 *) Fix wrong class cast in indexSize()
19 years ago
orbiter a7f0adf6fa bugfix in entity iterator
19 years ago
orbiter fa90c3ca7a - removed some usage of indexEntity
19 years ago
orbiter aea3e00864 cleanup: removed unused temporary index management in indexEntity.
19 years ago
orbiter 03c65742ba changes towards the new index storage scheme:
19 years ago
rramthun 84a00e5673 Use YaCy logging instead of something I don't understand.
19 years ago
theli ab7a911bb3 *) Trying to solve pool not open problem
19 years ago
allo a6245a302f even better ppm ;-)
19 years ago
hydrox d665f3c39c *) fixed Threadnames for stackCrawl-Threads
19 years ago
theli 3d5347bc8e *) changing loglevel for some messages
19 years ago
theli 0fcd113c42 *) last bugfix part. Seems to work now for the stackCrawler
19 years ago
theli b9c9eaeb44 *) next try todo a bugfix :-((
19 years ago
theli 4b4b93c413 *) next try todo a bugfix :-(
19 years ago
theli d9fbad71b9 *) next try todo a bugfix
19 years ago
theli 6da97bd2e4 *) next bugfix for threadpool problem
19 years ago
theli bea2b9edee *) further redesign of threadpools to solve too many thread problem
19 years ago
hermens 2d1283da34 This is an extremely ugly workaround for an incompatibility between yacySeed hashes and kelondroDyn keys
19 years ago
theli 784fd50437 *) more verbose thread names
19 years ago
theli 56e4dbeb71 *) displaying current active + current idle threads in PerformanceQueues_p.html now
19 years ago
theli 859c6a88f5 *) testing various thread pool eviction settings to avoid outOfMemory - Thread creation problem
19 years ago
allo 7197f171d3 better ppm calculation
19 years ago
orbiter f2b18cede9 AND-bugfix
19 years ago
orbiter b946e28e61 some ranking enhancements
19 years ago
rramthun 6c02f889f7 Cosmetic changes.
19 years ago
theli b191f06d16 *) Adding additional logging message to locate problems with stackcrawl threads
19 years ago
theli 5c56b9ed59 *) catch exceptions that could occur during url decoding
19 years ago
theli d9bcd73d93 *) Bugfix for exception
19 years ago
theli f5abfe8d57 *) more failsafe threadpools
19 years ago
orbiter 47344e8df0 removed referrer fake (too many complaints, too less use)
19 years ago
hermens ad0de69607 Yet another bug fix for svn 1441. It should work now.
19 years ago
hermens 58fd40e1c1 Aaargh
19 years ago
hermens b08af0c2cb *) Force download of seed file when checking upload success
19 years ago