Commit Graph

426 Commits (958ec20cd079d0f6b25522e8d42ef8a4ed6fb9a9)

Author SHA1 Message Date
orbiter ecd7f8ba4e - added NEAR operator (must be written in UPPERCASE in search query)
17 years ago
orbiter 03e7782269 more generics
17 years ago
fuchsi d517e96714 last cleanup bits to serverDate before the release. only safe refactoring (method renaming) changes outside of serverDate.
17 years ago
fuchsi 33ee6745f6 more cleanup in serverDate
17 years ago
fuchsi 21b8d1b918 small cosmetic change for static fields in serverCore (special protocol ASCII entities) to improve readability
17 years ago
orbiter 270d016d89 fix for missing anonymization in search profiling
17 years ago
orbiter f243e338cf implemented online caution also for local and remote search
17 years ago
orbiter b46bcaa5d8 changed method of profiling
17 years ago
orbiter f645408ae9 added url retrieve option to uls.xml interface
17 years ago
orbiter cc20870267 fix for constraint handover problem:
17 years ago
orbiter 9b0ae4b989 added referrer to remote crawl url list
17 years ago
orbiter d59c1a7936 removed test data
17 years ago
orbiter 89b9b2b02a redesigned remote crawl process:
17 years ago
orbiter 2fcd18a972 - fixed bad behaviour of search event worker processes
17 years ago
orbiter edba2b7bcc fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=543
17 years ago
orbiter c48b73cda2 redesign of ranking data structure
17 years ago
orbiter 6f1308da2f - some enhancements to IndexControlURLs (shows more links, connects referrer to another query)
17 years ago
orbiter c527969185 - enhanced monitoring of ranking parameters
17 years ago
orbiter bc2368e907 fix for problem with remote crawl referrers
17 years ago
orbiter 6eaa5a0e64 enhanced local search speed. The ranking process is now 6 times faster that before.
17 years ago
fuchsi 425e4ead66 Allow absolute paths in configuration settings.
17 years ago
orbiter a31b9097a4 preparations for mass remote crawls:
17 years ago
fuchsi 0e1738899f * Complete number localization and provide a more reasonable interface to serverObjects:
17 years ago
fuchsi f717beecb1 - Changed yFormatter handling to be more flexible and produce more readable code for server pages. There are serverObject.putNum() methods to allow adding of number type values in a formatted form, and put() methods for number types that add them without formatting. This reduces the need to transform them into Strings in server pages and removes the HTML encoding step which is unecessary for numbers.
17 years ago
fuchsi ce0bb1dc8a Increase defaults for the DHT Recieve Limits to prevent "busy" states.
17 years ago
orbiter 01e0669264 re-designed some parts of DHT position calculation (effect is the same as before)
17 years ago
orbiter 341f7cb327 steps to enhance remote search performance:
17 years ago
orbiter 76e4c2d69e fix for peer-ping in case that remote peer does not respond with valid values
17 years ago
orbiter f4a5c287fe re-implemented post-ranking of search results
17 years ago
orbiter 8ff5e2c283 - fixed/re-implemented media search
17 years ago
orbiter daf0f74361 joined anomic.net.URL, plasmaURL and url hash computation:
17 years ago
orbiter e90afa9483 fixed search access tracker
17 years ago
orbiter 4779f314fe first version of next-generation search interface:
17 years ago
orbiter f9e6cf6a3d more refactoring of search:
17 years ago
orbiter a34d9b8609 * added a search history cache that maintains search results for 10 minutes
17 years ago
orbiter ae86d010bb more refactoring of search processes; also some small speed enhancements
17 years ago
orbiter bb426565f0 added new yacy protocol for mass url-pull for better remote crawling distribution
17 years ago
orbiter 16c203f759 fixed remote search access tracker
18 years ago
orbiter 947fc46904 refactoring of search process:
18 years ago
orbiter 1af0e3bd84 refactoring
18 years ago
orbiter 5605887571 refactoring of search processes
18 years ago
orbiter e76fe1c078 - replaced unicode characters in copyright holder name ('Brausse')
18 years ago
orbiter 9ca46a8c69 indexing of local (intranet) urls enabled
18 years ago
orbiter f5a4efb76e fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=192&hilit=&p=1034#p1034
18 years ago
orbiter 40b0547611 - documentaton changes (removed old forum links)
18 years ago
orbiter b6d9cca67e - fixed problem with yacyVersion and own version generation
18 years ago
orbiter f40566f9bb separate YaCy networks:
18 years ago
orbiter 9bbd39b67c - removed unfinished auto-updater from roland and martin
18 years ago
orbiter 069562a14d fixed problem with re-crawl; replaced error file-db with ram-db
18 years ago
orbiter 07b4e5066b bugfix in messages
18 years ago
orbiter f04add6cb4 limitation of remote search result number
18 years ago
orbiter 4f5496062c protection against too large seeds
18 years ago
karlchenofhell 601fc7d1c5 - added source to J7Zip-modifed.jar and it's license (changelog is still to come)
18 years ago
orbiter d6480dc670 fix for long transfer pauses
18 years ago
orbiter 06b6e35484 fix for a null pointer exception if clusters are not defined
18 years ago
orbiter d4428947af fix for http://www.yacy-forum.de/viewtopic.php?p=34962#34962
18 years ago
orbiter 81844e85b2 - fixed more cluster routing problems
18 years ago
orbiter e48189c710 enhanced cluster routing
18 years ago
karlchenofhell 97d4ab2053 - handle null from iterator in IndexCreateWWWLocalQueue_p.java
18 years ago
orbiter b33cef421e better routing for public clusters
18 years ago
orbiter f8de19fb2f robinson cluster: added client-side protocol implementation
18 years ago
orbiter 657585fe0d network functions for robinson peers: server-side protection
18 years ago
orbiter 2e052eb816 fixed a bug in remote search with remote search tracker
18 years ago
orbiter b79b4082e2 completed search exclusion:
18 years ago
orbiter 40c14a4f0e - better implementation of search query properties
18 years ago
orbiter 2cb16824e3 removed support for old database structures.
18 years ago
orbiter 861f41e67e redesigned NURL-handling:
18 years ago
orbiter dc0c06e43d PLEASE MAKE A BACK-UP OF YOUR COMPLETE DATA DIRECTORY BEFORE USING THIS
18 years ago
karlchenofhell c016fcb10f - added streaming-support to CrawlURLFetchStack_p servlet
18 years ago
karlchenofhell d114a0136e - crawl profile: don't add null-values
18 years ago
karlchenofhell e6ddf135bb - enabled fetching new crawls via /yacy/list.html?list=queueUrls for testing purposes
18 years ago
orbiter 30d79d69a6 fix for wrong display of search statistics
18 years ago
orbiter c464157a6e replaced some toString()
18 years ago
orbiter b2f4087400 redesign of last-seen fieln inside seed:
18 years ago
orbiter e00e850a98 removed constants (no connection with yacySeed.dna identifier)
18 years ago
orbiter c2d6edf21d integrated number of remote targets as 'partitions' into remote search protocol
18 years ago
orbiter 4f6eed5623 QPM increment
18 years ago
orbiter f3f99b19c6 extended search statistics
18 years ago
orbiter c0851ee943 refactoring: moved and renamed de.anomic.data.searchResults to plasma package
18 years ago
orbiter 76fab83395 fixed bugs in seach statistics
18 years ago
karlchenofhell fdb45378fb - don't spam log because of some old URLs
18 years ago
allo 0c81bd39d4 XSS-safe put as default.
18 years ago
orbiter 52c6461e6b some bugfix for statistics
18 years ago
(no author) fe72b772cf added a monitor page for search requests
18 years ago
auron_x 9699b094e8 *) fixed hello reporting yourip=UNRESOLVED_PATTERN
18 years ago
orbiter 0a050bc043 enhanced ranking
18 years ago
orbiter 1377c53aa3 extraction of media links from search results
18 years ago
orbiter bf0d820659 - added correct flagging of word properties
18 years ago
orbiter 109ed0a0bb - cleaned up code; removed methods to write the old data structures
18 years ago
orbiter ad1e4aa88e added selection of audio, video, image and application resources
18 years ago
orbiter ceb9e3aa17 - enhanced parser: collection of audio, video, image and application links
18 years ago
orbiter 0a0c3edeb6 fixed a bug in index transfer
18 years ago
orbiter 8e7215475b - extended ViewFile to use is as debugging-tool: you can now use the
18 years ago
orbiter 30888e7a2f implementation of search constraints
18 years ago
orbiter d66dbd0d65 bugfix for received number in transferRWI
18 years ago
orbiter c9364246cc introduced new RWI-Object.
18 years ago
orbiter 497428c8ec refactoring
18 years ago
orbiter 76fceb9997 refactoring
18 years ago
orbiter bb7d4b5d5e refactoring to prepare new RWI entry object
18 years ago
orbiter 114a76a86e - added flag to urlhash that shows that domain is a local domain
18 years ago
orbiter 8fdefd5c68 generalization of payload definition of index storage
18 years ago
orbiter d3431433b0 more anonymization in logging
18 years ago
orbiter 78b7f6f7fd bugfix for index remove bug,
18 years ago
orbiter 06854988da - full integration of new LURL database in INDEX
18 years ago
theli 52466067d8 *) Bugfix for ArrayIndexOutOfBoundsExceptions which occure because SimpleDateFormat is not thread-safe
18 years ago
orbiter b79e06615d - added new LURL.Entry class for next database migration
18 years ago
orbiter 77a59a115d refactoring of indexing methods
18 years ago
orbiter a5dd0d41af - refactoring of plasmaCrawlLURL.Entry to prepare new Entry format
18 years ago
orbiter afbb547f3d extended options for abstracts generation in remote search interface
18 years ago
orbiter 2e4aa6a170 refactoring of Advanced Config:
18 years ago
orbiter dbc2e039bb added time-out option parameter to call hierarchy
18 years ago
orbiter 00746ca232 identified and fixed search performance problem caused by
18 years ago
orbiter df1629b05a - code cleanup
18 years ago
orbiter 3aac5b26da - added automatic tag generation when a web page from the search results is added
18 years ago
orbiter e03740c306 small fix for last commit
18 years ago
orbiter c89d8142bb replaced old 'kCache' by a full-controlled cache
18 years ago
orbiter 6e2907135a bugfixes for remote search server part
18 years ago
orbiter cf9884e22b first attempt to implement a secondary search
18 years ago
orbiter 75b198bc02 - updated references to indexContainer
18 years ago
orbiter 4f9e42d5ed more changes towards better join-search
18 years ago
orbiter 82a6054275 - fixed bug with new indexAbstract generation
18 years ago
orbiter 74d1dea30b changes towards better join-search
18 years ago
orbiter c543028dd4 fixed double/missing null check for LURLs
18 years ago
orbiter 96c6e4e322 - enhancements to detailed search page
18 years ago
orbiter 9340dbb501 fixed all possible problems with nullpointer exception for LURLs
18 years ago
hermens ff4362b02d some more fixes for new plasmaCrawlLURL.load behavior
18 years ago
orbiter 4866868c0e added write cache for LURLs
18 years ago
orbiter 8a0e35618b enhancements to search result preparation
18 years ago
theli f3ac4dbbb9 *) better handling of server shutdown
18 years ago
orbiter 18b6876860 new cache flush configuration settings
18 years ago
orbiter 6ad471ef96 * applied many compiler warning recommendations
19 years ago
theli 5e0b6f8f83 *) sorting peer name list on Blacklist_p.html
19 years ago
theli 6c8366aea1 *) Bugfix for blacklist import function
19 years ago
theli eee44be602 *) adding an interface for customized blacklist classes
19 years ago
theli 66f1eb07d9 *) Bugfix for IllegalArgumentException in transferURL
19 years ago
theli d2e8e76218 *) now it's possible to configure the yacy blacklist separately for dht, search, proxy, crawler
19 years ago
orbiter f43c90fa98 fixed handling of null referer in crawlOrder
19 years ago
orbiter abf22f6e60 removed url normalform computation from htmlFilterContentScraper.
19 years ago
orbiter ec5149ff3b fix for busyCacheFlush detection
19 years ago
orbiter f58283def2 better control of index flush
19 years ago
orbiter 80b6c90d54 enhancements to prevent blocking during dht transfer receive
19 years ago
hermens d56f06401e - Cache known URLs during indexReceive to avoid getting blocked during loadedURL.exists() whenever possible
19 years ago
theli c7b6389ca1 *) renaming indexDistribution.dhtReceiptLimitEnabled property to indexDistribution.transferRWIReceiptLimitEnabled
19 years ago
orbiter 9183d21f25 renamed new index class to old name
19 years ago
orbiter c4e922885a replaced indexURLEntry by new class that uses a kelondroRow.Entry object
19 years ago
orbiter 5f72be2a95 some redesign of EURL storage
19 years ago
orbiter 58df8b7bbf a large collection of different changes
19 years ago
hydrox 8ba8e2b7d9 *) added cache for blacklists urlhashs recieved by DHT. DHT does not request URLs listed in this cache.
19 years ago
hermens 53cbcc6d6e Implement emergency break in index receive when the limit of the ramCache is exceeded by more than cacheLimit
19 years ago
theli b20496e42b *) make DHT DoS check configurable (requested by KoH)
19 years ago
hermens 38a1410361 Don't test a remote peer's seed during hello.respond as its IP might not be proper, especially while still virgin
19 years ago
orbiter 5041d330ce refactoring
19 years ago
orbiter 90d569d70f refactoring of index management:
19 years ago
orbiter a930be4ba3 refactoring of index management:
19 years ago
orbiter 7dd57a3828 added a busy-time estimation at DHT/RWI-Receive
19 years ago
theli fcec40fcc6 *) don't accept messages without subject or payload
19 years ago
orbiter 82b2bc6932 patch for index-transfer DoS problem
19 years ago
orbiter a474669338 start with refactoring of index management
19 years ago
allo 799c04091d Bugfix for Spam-Bug (Header manipulation)
19 years ago
orbiter dbe96e6541 added hand-over of search filter and prefer ranking to yacy protocol
19 years ago
orbiter 00a5d435e2 - fixed some bugs with domain filter
19 years ago
orbiter bd283b8443 fixed bugs:
19 years ago
orbiter 0a4c2e89ed remote crawl orders are now only accepted if sum over all
19 years ago
orbiter 1f4412a146 adopted isListed to discussed new behavior as discussed (url, getFile)
19 years ago
orbiter 3286b1f498 re-organisation of lurl-creation and -stacking
19 years ago
hermens 289da326e5 *) Bugfix: remove blacklisted URL from loadedURL, when received via DHT transfer
19 years ago
rramthun 9f979d4fa5 Domain-lists gzip-compressable and sendable via cr-send/receive
19 years ago
orbiter f188611fc6 apply blacklist on rwis during dht receive
19 years ago
theli 5ee0125046 *) adding possibility to configure the server port for seed uploading via scp.
19 years ago
allo 7afa5c1b8e staticIP fix
19 years ago
theli f108048a2c *) Bugfix for NullpointerException in hello.java
19 years ago
orbiter bae3783d38 added a snippet marking
19 years ago
allo f73d51f94b reverted last change
19 years ago
allo 8997b83806 store the staticIP(dyndns) in seed, not the real IP
19 years ago
allo 7c5f8f997a some more staticIP fixes
19 years ago
orbiter d31a4e0b4f some small enhancements with cache flushing parameters and data structures
19 years ago
hermens 3208fe14ed *) log exceptions in crawlOrder.java to the logfile instead of stdout
19 years ago
orbiter 7eb10675b3 re-organization of index management
19 years ago
theli d0f76fc9bc *) setting logging level for thread pools to info
19 years ago
orbiter fb7411d7bb re-structuring of ranking application:
19 years ago
orbiter d98418390b - introduced rankingProfile Class
19 years ago
allo 1f3eaf9f8e use DATA/HTDOCS for notifier.gif. Works even if htroot is readonly
19 years ago
orbiter fa90c3ca7a - removed some usage of indexEntity
19 years ago
orbiter 03c65742ba changes towards the new index storage scheme:
19 years ago
rramthun 5942f6334c Some language fixes.
19 years ago
orbiter f4ffa9aee5 - implemented more attributes to index entries
19 years ago
borg-0300 c5b6154136 added CRDistOn = true/false
19 years ago
borg-0300 8d8a40c2d9 added properties
19 years ago
orbiter cfd1e5e376 more security for index transfer protocol:
19 years ago
orbiter 423ce9bf59 quickfix for http://www.yacy-forum.de/viewtopic.php?p=15336#15336
19 years ago
allo 5eba6c66c6 thelis fix
19 years ago
rramthun c59027e520 Translated status_p.inc a bit further, but it didn't work.
19 years ago
orbiter 9544c47684 added some UTF-8 handling.
19 years ago
orbiter 9086261476 refactoring of base64 encoding:
19 years ago
orbiter b3dca06bb1 added location column to network pages.
19 years ago
orbiter bb79fb5d91 - changed handling of error cases retrieving urls from database
19 years ago
orbiter 37f88b4017 code cleanup
19 years ago
orbiter 8f1f2daa5e implemented interactive link deletion of search results.
19 years ago
orbiter 7920e1547d code cleanup
19 years ago
orbiter 1d6a6d1f85 code cleanup
19 years ago