Commit Graph

5793 Commits (f00168ecc506b10933167bac2c8187f9cc7abccb)

Author SHA1 Message Date
orbiter 7f851d62a7 replaced HashARC with SizeLimited Objects which are less costly
13 years ago
orbiter d4291ac1f3 more tolerance when creating solar document
13 years ago
orbiter 78fc3cf8f8 refactoring and new usage of SentenceReader: this class appeared as one
13 years ago
orbiter bb8dcb4911 automatically adopt size of word cache to available memory
13 years ago
Michael Peter Christen ad09b786bf clean up parser data
13 years ago
Michael Peter Christen 276a66a793 Adding a limit of 1000 links that a parser shall store during indexing.
13 years ago
Michael Peter Christen 613b45f604 - better data structures in secondary search
13 years ago
Michael Peter Christen de903a53a0 parser refactoring & hacks
13 years ago
Michael Peter Christen 8a82609360 - smaller caches to save memory
13 years ago
Michael Peter Christen 7249d9c9de bugfix for concurrent seed loader
13 years ago
Michael Peter Christen c72d3b12cd concurrently initialize the seed list during p2p network bootstrap
13 years ago
Michael Peter Christen 1825f165b8 better integration of blacklist according to use case
13 years ago
Michael Peter Christen c18fa9fa75 Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
13 years ago
Michael Peter Christen ce8d4b87d9 fixes for new eclipse 'Juno' warning 'Resource leak'.
13 years ago
Michael Peter Christen 0c345d1559 giving threads name so its easier to see whats happening during
13 years ago
reger 067728bccc add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages)
13 years ago
Michael Peter Christen 03280fb161 removed segments-concept and the Segments class:
13 years ago
Michael Peter Christen 508a81b86c added solr field 'refresh_s' which stores the refresh url contained in
13 years ago
Michael Peter Christen f3167def64 do not fill the keywords with title content if keywords do not exist.
13 years ago
Michael Peter Christen 9116013c64 - allow lazy initialization of solr value (if using 'lazy', then no
13 years ago
sixcooler 97f60010d8 fix crawl start from file
13 years ago
Michael Peter Christen 0294a53459 - add canonical field only if requested by solr schema
13 years ago
Michael Peter Christen 3fd4a01286 added option to record urls that are forwarded to the solr index
13 years ago
Michael Peter Christen d763e4d94b fixed bad referer computation in SSIs which causes a NPE during host
13 years ago
Michael Peter Christen 358b04885e more IPv6 hacks
13 years ago
Michael Peter Christen 96aeb127e3 generalized localhost naming.
13 years ago
Michael Peter Christen 77f795756c fixing redirects and status codes: storing of status code in
13 years ago
Michael Peter Christen 8dd469b9dd added option to configure the autocommit delay time of solr on-the-fly
13 years ago
Michael Peter Christen b9dfca4b0a - fixed IndexFederated Servlet / a embedded Solr can now be selected
13 years ago
Michael Peter Christen fad3b14813 added jetty libraries, needed for future use as web server and as
13 years ago
Michael Peter Christen a38b0a2c46 extended embedded solr tests to ensure that it will be usable within a
13 years ago
Michael Peter Christen b9d42fd9c8 using com.google.common.io.Files instead of homebrew methods
13 years ago
Michael Peter Christen a5eb91fa60 refactoring
13 years ago
Michael Peter Christen 1be0025a9c - added test for EmbeddedSolrConnector
13 years ago
Michael Peter Christen dbdd697f4d moved RDFaParser.xsl configuration file to defaults
13 years ago
Michael Peter Christen 90b82ce994 using guava for host resolution (non-blocking for ips) and time-out
13 years ago
Michael Peter Christen e12bb254b4 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 3f55dc7c1e - added solr core and libraries that solr needs (lucene is missing, will
13 years ago
Michael Peter Christen c337190a00 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
cominch c63c3a4495 Show additional interaction elements in footer section on each page, if
13 years ago
Michael Peter Christen 786be7d175 better integration of RDFaParser
13 years ago
Michael Peter Christen de3ef8ad73 removed unimportant warnings
13 years ago
Michael Peter Christen 82a682b31d fixed problem with seed when switching network
13 years ago
Michael Peter Christen 8c544edee4 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 7dc59979bc fix for npe, possibly for http://bugs.yacy.net/view.php?id=195
13 years ago
Michael Peter Christen 1d4e206b2b bugfix in vocabulary generation
13 years ago
cominch 2c89975378 Merge remote-tracking branch 'original yacy/master'
13 years ago
Michael Peter Christen 52f5d40043 better abstraction of document model generation
13 years ago
Michael Peter Christen 8b7c4d3144 produce a rdf output containing the triplestore with yacydoc; ie:
13 years ago
cominch f7160dae5c Merge remote-tracking branch 'original yacy/master'
13 years ago
cominch e4555cbee3 Augmented browsing: Pass on additional action parameter
13 years ago
Michael Peter Christen 24bbe359ca integrate also geonames library files for less cities. these are more
13 years ago
Michael Peter Christen 223a5440ab preventing that an empty pnd is inserted into the vocabularies
13 years ago
Michael Peter Christen 8e97ada7c9 IPv6 bugfix
13 years ago
Michael Peter Christen 963f92ed9a - merged files
13 years ago
Michael Peter Christen dd88d0ace2 more logging
13 years ago
Michael Peter Christen 94d54e2d91 added recognition of multi-word terms in vocabulary matching
13 years ago
Michael Peter Christen 64c0268b2b show triplestore metadata in yacydoc and viewfile
13 years ago
Michael Peter Christen 0fbd749207 ipv6 update
13 years ago
Michael Peter Christen c2f0d16d2c fixed vocabulary initialization
13 years ago
Michael Peter Christen fbded1f466 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen df3531f8d5 added the generation of virtual vocabularies using the pnd
13 years ago
Michael Peter Christen e806106b10 jquery bugfix
13 years ago
Michael Peter Christen a0f1decd82 - added loading of the dbpedia pnd triplestore in the dictionary loader
13 years ago
cominch 2ac7a5c1f2 Augmented browsing: Add overlay bar which shows the vocabulary tags
13 years ago
cominch 3c255c025b Show tags in search results (if activated in ConfigPortal_p.html)
13 years ago
Michael Peter Christen 16d8f33795 added objectlink generation to vocabulary generation and editor
13 years ago
cominch f49d92d8da Cleanup of interaction class and helper routines
13 years ago
cominch 56b0115054 Triplestore: modify routines to access per user store
13 years ago
cominch a95127c9af Triplestore: initalize per-user triplestores
13 years ago
Michael Peter Christen d45718251e refactoring (Localization -> Location)
13 years ago
Michael Peter Christen b8b3c87ba7 - renamed localization to location (that was confusing)
13 years ago
Michael Peter Christen e89747bb67 - added automated generation of vocabularies from url stubs
13 years ago
Michael Peter Christen 79464189a4 The 'Locale' vocabulary, which is generated by geo data, has now the
13 years ago
Michael Peter Christen eca38c53e7 added a vocabulary editor
13 years ago
Michael Peter Christen 61bb52d55c - using http://purl.org/dc/terms/references to refer from an
13 years ago
Michael Peter Christen 2bbb6c52cf added option to clean the triplestore when deleting the index
13 years ago
Michael Peter Christen 50c576599b allow multiple parser options instead of printing an error
13 years ago
Michael Peter Christen c02d742e53 proper namespaces in triplestore dump
13 years ago
Michael Peter Christen 8b53771db2 changed behavior of navigation processing:
13 years ago
Michael Peter Christen 5fc6524ca8 - moved triple store to net.yacy.cora.lod (should be generalized there
13 years ago
cominch 8d2e6355f8 augmented browsing: remove non-existing external snippet file
13 years ago
cominch c90f174799 preparation and generalization of augmented browsing methods
13 years ago
Michael Peter Christen bef823c247 close the reader if finished
13 years ago
Michael Peter Christen 4ee6fb1de9 added missing blacklist dht cache storage (maybe due to mistakes in
13 years ago
Roland 'Quix0r' Haeder e4d36fa5eb Fix to make all values lower-case (this should make all existing blacklists compatible with the new enum)
13 years ago
Roland 'Quix0r' Haeder edaa09b9b1 Rewrote all String blacklist types to enum 'BlacklistType', closes bug
13 years ago
Roland 'Quix0r' Haeder af5a597e47 Scroogle is not comming back, remove dead code
13 years ago
cominch 7a4dab6d1d - removed unused variables
13 years ago
Michael Peter Christen ca93835713 removed usage of deprecated methods
13 years ago
Michael Peter Christen 23e38bd918 do not load the "_triplestore.rdf" files which are of special use in
13 years ago
Michael Peter Christen 90c6fc4b63 load all - but not the persistent local.rdf - triples from
13 years ago
cominch bbfc53b663 bugfix
13 years ago
cominch 65c5826d93 bugfix
13 years ago
cominch aa0295917c augmentation
13 years ago
cominch ed2ea0f08e augmented browsing modification
13 years ago
cominch 6b32f7c1f6 re-enable augmented proxy
13 years ago
cominch 3b08edec2e bugfix
13 years ago
cominch 5f8ba7f4f2 small changes
13 years ago
cominch 300b235ce8 Updated Demo Servlet
13 years ago
cominch 90512640bf Added config switches for custom parser
13 years ago
cominch b5a8fb5fd8 Catch malformed URL when submitted in encoded style
13 years ago
cominch df47f31235 interaction: add special table interaction
13 years ago
cominch e14f2881ae interaction: add special table interaction
13 years ago
cominch d7326079a8 interaction: add global variable store
13 years ago
cominch 4e4e7a99f8 interaction: add global variable store
13 years ago
cominch 8e80894812 create virtual web folder /currentyacypeer/ which always points to local
13 years ago
cominch bde07ed7a8 Add tagging overlay element
13 years ago
cominch b0bc0b4572 Add new demonstration module for client-side key-value store (backend:
13 years ago
cominch c9dc6cda02 Demonstration: include value from interaction in search results
13 years ago
cominch ae8adb0e58 Small changes
13 years ago
cominch bcbd8eee33 Add several parsers, for RDFa and rdf files.
13 years ago
cominch 9ef5a80f4e add interaction for triples and selector for augmented browsing
13 years ago
cominch 282c1620d6 Allow TripleStore to be persistent after reboot
13 years ago
cominch 5d20cd324a Add Triplestore and RDF query interface
13 years ago
cominch bc9a618e0a augmented browsing: ignore js and css, integrate more user interaction
13 years ago
cominch b21048892b augmentedParser add features and integrate external html parser to
13 years ago
cominch 9cbfc1a1c0 augmentedProxy, which forwards every proxy request to a
13 years ago
Michael Peter Christen 3b992e6b00 using utf8 String compression in Webstructure database
13 years ago
Michael Peter Christen 26301a538d bugfix in Domains - dns-lookup
13 years ago
Michael Peter Christen cde20911bb saved a bit more ram using UTF8 String compression for OpenGeoDB and
13 years ago
Michael Peter Christen 225ee42879 made the GeoLocation into an interface with the current
13 years ago
Michael Peter Christen 2280a7b276 - changed initialization order to prefer allocation of memory for table
13 years ago
Michael Peter Christen 0746308bc2 only the metadata tables shall be able to use the tail cache
13 years ago
Michael Peter Christen 7ec9bef0c3 fix for OOM
13 years ago
Michael Peter Christen 41c02cb10e - less restrictions for usage of Table RAM copy
13 years ago
Michael Peter Christen b8f56a9803 npe bugfix
13 years ago
Michael Peter Christen dd14b19c26 lazy initialization of block rank table ... only normal web search uses
13 years ago
Michael Peter Christen ba10caf89a lazy initialization of database tables
13 years ago
Michael Peter Christen 701b9a28a0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen ab7107b34b fixed RWIProcess queue limits: now discovering hidden results for mass
13 years ago
Michael Peter Christen 10c9c17d51 fixed handlemap spread factor and null iterator handling
13 years ago
Michael Peter Christen b0095c8d3c flush the compressor cache when a cleanup is done
13 years ago
Michael Peter Christen a61f44f9e4 lazy initialization of block rank table.
13 years ago
Michael Peter Christen 96e9d77270 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 00f2df1120 a variety of possible memory leak fixes
13 years ago
Michael Peter Christen 3dd8376825 added automatic cleaning of cache if metadata and file database size is
13 years ago
Michael Peter Christen d0ec8018f5 fixes for bad long computation
13 years ago
Michael Peter Christen 6bb07afcc3 accept also files with other file prefix; used to read 'foreign' cache
13 years ago
Michael Peter Christen 96c8119b50 added GeoLocation / GeoPoint classes which uses less memory than
13 years ago
Michael Peter Christen 461a0ce052 removed warnings
13 years ago
Michael Peter Christen 62ae9bbfda allow more POIs, get more at once
13 years ago
Michael Peter Christen 407fdf6968 more bug fixes and performance hacks for search process
13 years ago
Michael Peter Christen a1fe65b115 performance hacks
13 years ago
Michael Peter Christen 2fe207f813 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 5aee19daa4 added show from cache in search results (not yet finished)
13 years ago
Michael Peter Christen 5e562dcdb7 adopted vocabulary usage within anotation/naviagtion feature of search
13 years ago
Michael Peter Christen 514700291a moved Vocabulary to cora package (added in git
13 years ago
Michael Peter Christen 0284a4d88f more fixes for double precision of coordinates
13 years ago
Michael Peter Christen 964406ad17 added concurrency enhancement to xml parser
13 years ago
Michael Peter Christen 240045cf7c fix for bad distance computation
13 years ago
Michael Peter Christen e0d8643226 - performance hacks
13 years ago
Michael Peter Christen 7a329465b3 using pre-compile pattern in blacklist; should enhance search speed
13 years ago
Michael Peter Christen 6e83b02b83 - bugfix for surrogate file reader
13 years ago
Michael Peter Christen 9b4c699526 ehanced location search:
13 years ago
Michael Peter Christen 834dc6b263 store more data from interface access
13 years ago
Michael Peter Christen 1f48d1528b performance hacks
13 years ago
Michael Peter Christen c70aaccdc9 better location to generate a guid for rss messages
13 years ago
Michael Peter Christen 10da7335ea performance hack: use a hash cache for all hashes that are computed by a
13 years ago
Michael Peter Christen f8a0cf6d7c RSSMessages do not need a concurrent hash map -> removed overhead
13 years ago
Michael Peter Christen 07ca7e4dd1 enhanced RSS parsing by ensuring that it is parsed with a buffered input
13 years ago
Michael Peter Christen 7c1feefb28 introduced a default 10 second time-out in rwi normalization time
13 years ago
Michael Peter Christen 8d997d55b6 better logging
13 years ago
Michael Peter Christen 65d37e6a20 only ASCII needed in seed bitflags
13 years ago
Michael Peter Christen 0f82fb3628 using double instead float for a better release ordering
13 years ago
Michael Peter Christen 43c2c6e588 better logging
13 years ago
sixcooler 56087c1f23 bump to httpclient- httpcore-, httpmime- 4.2
13 years ago
Michael Peter Christen 20e0cc0822 fix for bad location evaluation
13 years ago
Michael Peter Christen 71c3163f3d - fixes to node identification
13 years ago
Michael Peter Christen 4d3cc02168 replaced old bzip2 library against better documented commons-compress
13 years ago
Michael Peter Christen ad222be7f8 added node state icon in network list
13 years ago
Michael Peter Christen eff7667554 fix for http://bugs.yacy.net/view.php?id=188
13 years ago
Michael Peter Christen 3c2bec681f added a root node flag: identifies peers with short ping time
13 years ago
Michael Peter Christen c846e9ca14 redesign of the crawler monitor page: show crawled pages instead of
13 years ago
Michael Peter Christen 8b974905ee changed log-in text for all servlets with authentication:
13 years ago
Michael Peter Christen 16b21f7a5b Added more steering in Crawler_p.html interface
13 years ago
Michael Peter Christen acc19e190d hack against 100% cpu during crawl delete
13 years ago
Michael Peter Christen c15fcde1c8 add-on to latest commit
13 years ago
Michael Peter Christen cf47d94888 performance hack to parse numbers inside of substrings without actually
13 years ago
Michael Peter Christen 7e0ddbd275 added a "fromCache" flag in Response object to omit one cache.has()
13 years ago
Michael Peter Christen 81737dcb18 removed stack trace from swf parser since we cant do anything there
13 years ago
Michael Peter Christen 7bf421b9dd - fixed image search page navigation
13 years ago
Michael Peter Christen 125d47b3c1 added more interruptions in DidYouMean because that was the cause for
13 years ago
Michael Peter Christen c6a09eab0b synchronization needed
13 years ago
Michael Peter Christen fb94b47b1a changed queue sizes to have less memory occupied during indexing
13 years ago
Michael Peter Christen 76157dc2c3 bugfix for http://bugs.yacy.net/view.php?id=173
13 years ago
reger 6696cb1313 bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer
13 years ago
Michael Peter Christen c6558cba08 more classification bugs
13 years ago
Michael Peter Christen 082831b9d6 search contentdom was checked in wrong way - fixed
13 years ago
reger ee553d971e correct typo in scripts_txt comment
13 years ago
Michael Peter Christen f294f2e295 bugfix to http://bugs.yacy.net/view.php?id=181
13 years ago
Michael Peter Christen acf8d521a2 fix for http://bugs.yacy.net/view.php?id=126
13 years ago
Michael Peter Christen bb88878b4d the last commit was incomplete..
13 years ago
Michael Peter Christen d320a31ae1 bugfix for http://bugs.yacy.net/view.php?id=186
13 years ago
Michael Peter Christen fa735f4f04 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 3e1bc9477f Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
Michael Peter Christen 6f8a2fef1f small speed enhancement using a column factory
13 years ago
Roland 'Quix0r' Haeder d10627d591 More sync in close() methods
13 years ago
Roland 'Quix0r' Haeder b3ae2aa41f With or without 'final'? At least please try it in other methods
13 years ago
Roland 'Quix0r' Haeder fbb946f913 Made a method static (Eclipse suggested it), removed unused import, pk=null check does now output a warning in logfile
13 years ago