96aeb127e3
generalized localhost naming.
...
this is also a preparation for a better IPv6 implementation.
13 years ago
77f795756c
fixing redirects and status codes: storing of status code in
...
ResponseHeader to make it available for late evaluations, like storage
in solr.
13 years ago
8dd469b9dd
added option to configure the autocommit delay time of solr on-the-fly
13 years ago
b9dfca4b0a
- fixed IndexFederated Servlet / a embedded Solr can now be selected
...
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
13 years ago
fad3b14813
added jetty libraries, needed for future use as web server and as
...
application server for the solr search interface
13 years ago
a38b0a2c46
extended embedded solr tests to ensure that it will be usable within a
...
jetty instance
13 years ago
b9d42fd9c8
using com.google.common.io.Files instead of homebrew methods
13 years ago
a5eb91fa60
refactoring
13 years ago
1be0025a9c
- added test for EmbeddedSolrConnector
...
- added needed libraries for this test
this includes most (all) files needed for an embedded solr
13 years ago
e12bb254b4
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
3f55dc7c1e
- added solr core and libraries that solr needs (lucene is missing, will
...
follow later)
- added embedded solr connector which can connect to solr
programmatically (without using a server in between)
13 years ago
786be7d175
better integration of RDFaParser
13 years ago
0752983fbd
- automatic periodic saving of triplestore
...
- transaction-safe storage of triplestore
13 years ago
9264d8b4af
removed old navigation practice using subject tags in favor of
...
triplestore-tags
13 years ago
64c0268b2b
show triplestore metadata in yacydoc and viewfile
13 years ago
a95127c9af
Triplestore: initalize per-user triplestores
13 years ago
e89747bb67
- added automated generation of vocabularies from url stubs
...
- added clear of all terms for vocabularies
- added deletion of vocabularies
13 years ago
8b53771db2
changed behavior of navigation processing:
...
- vocabulary annotation is not done any more into the metadata of urldb
- vocabularies are written into the jena triplestore using a rdf
vocabulary
- vocabularies for rdf tripel must be updated; refactoring done
- with the new navigation tags in the triplestore a faster
pre-urldb-lookup is possible: navigation is processed now within the RWI
during pre-ranking retrieval
- added also a Owl vocabulary stub to add the plain-text url to the
triplestore using the owl:sameas predicate
13 years ago
5fc6524ca8
- moved triple store to net.yacy.cora.lod (should be generalized there
...
later
- added abstract add, delete, get methods in the triplestore
- added generation of triples after auto-annotation
- migrated all MultiProtocolURI objects to DigestURI in the parser since
the url hash is needed as subject value in the triples in the triple
store
13 years ago
4ee6fb1de9
added missing blacklist dht cache storage (maybe due to mistakes in
...
cherry picking)
13 years ago
edaa09b9b1
Rewrote all String blacklist types to enum 'BlacklistType', closes bug
...
#143
Conflicts:
htroot/Supporter.java
htroot/yacy/crawlReceipt.java
htroot/yacy/transferRWI.java
htroot/yacy/transferURL.java
source/de/anomic/crawler/CrawlStacker.java
source/de/anomic/data/ListManager.java
source/net/yacy/peers/Protocol.java
source/net/yacy/repository/Blacklist.java
source/net/yacy/repository/LoaderDispatcher.java
source/net/yacy/search/Switchboard.java
source/net/yacy/search/index/MetadataRepository.java
source/net/yacy/search/index/Segment.java
source/net/yacy/search/query/RWIProcess.java
source/net/yacy/search/snippet/MediaSnippet.java
13 years ago
af5a597e47
Scroogle is not comming back, remove dead code
...
Conflicts:
source/net/yacy/search/Switchboard.java
13 years ago
65c5826d93
bugfix
...
Conflicts:
source/net/yacy/document/parser/augment/AugmentParser.java
13 years ago
cde20911bb
saved a bit more ram using UTF8 String compression for OpenGeoDB and
...
Geonames data files.
13 years ago
2280a7b276
- changed initialization order to prefer allocation of memory for table
...
files first
- bugfixes in memory amount calculation
13 years ago
0746308bc2
only the metadata tables shall be able to use the tail cache
13 years ago
41c02cb10e
- less restrictions for usage of Table RAM copy
...
- new limit to use the table copy (instead of flag): 400MB available. If
less is available, then a copy is never used. If more is available, then
it can be used if there is a remaining space of at least 200MB
- flush caches more often: flush the Digest cache
13 years ago
dd14b19c26
lazy initialization of block rank table ... only normal web search uses
...
this. When interactive search or location search is used, the block rank
is switched off
13 years ago
701b9a28a0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
htroot/PerformanceMemory_p.java
13 years ago
ab7107b34b
fixed RWIProcess queue limits: now discovering hidden results for mass
...
result retrieval
13 years ago
b0095c8d3c
flush the compressor cache when a cleanup is done
13 years ago
a61f44f9e4
lazy initialization of block rank table.
...
this causes that the table is not initialized when there is no search is
done. the effect is most strong if YaCy is started headless which causes
no browser pop-up which otherwise would load the search page and
therefore trigger the initialization of the table.
13 years ago
96e9d77270
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
13 years ago
00f2df1120
a variety of possible memory leak fixes
13 years ago
d0ec8018f5
fixes for bad long computation
13 years ago
461a0ce052
removed warnings
13 years ago
407fdf6968
more bug fixes and performance hacks for search process
13 years ago
a1fe65b115
performance hacks
13 years ago
2fe207f813
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
5e562dcdb7
adopted vocabulary usage within anotation/naviagtion feature of search
...
to new SimpleVocabulary class
13 years ago
240045cf7c
fix for bad distance computation
13 years ago
e0d8643226
- performance hacks
...
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
13 years ago
9b4c699526
ehanced location search:
...
- search request are now made using a map boundary
- search results are only computed for the map boundary
- the number of results is adopted to the results in the visible range
- added a double-buffering for the search result markers
- added a search query option for the search results:
/radius/<lat>/<lon>/<radius>
13 years ago
834dc6b263
store more data from interface access
13 years ago
10da7335ea
performance hack: use a hash cache for all hashes that are computed by a
...
byte array. If this hash is used in a HashMap (which is very often the
case) then this hack eliminates a lot of re-computations of the same
hash.
13 years ago
7c1feefb28
introduced a default 10 second time-out in rwi normalization time
...
uring search process to prevent endless deadlocks after a very long
running search
13 years ago
c846e9ca14
redesign of the crawler monitor page: show crawled pages instead of
...
queue of urls that shall be crawled
13 years ago
c15fcde1c8
add-on to latest commit
13 years ago
cf47d94888
performance hack to parse numbers inside of substrings without actually
...
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
13 years ago
7e0ddbd275
added a "fromCache" flag in Response object to omit one cache.has()
...
check during snippet generation. This should cause less blockings
13 years ago
Michael Peter Christen