ca313e404f
- if a "/date" modifier is used, the solr remote query applies an
...
ordering by date (ascending)
- added also some 'anti-timetravel' protection (check if date is in the
future within any metadata date field)
13 years ago
a4214694df
We assert that no other metadata storage than solr is used now.
...
Therefore a property like solrConnected() must be true all the time.
Removal of this method causes removal of all write operations to the old
metadata index.
13 years ago
0cec7e761a
enhanced snippet extractor to find snippets also inside of tokens of an
...
url
13 years ago
6c50d016ed
pdf- and zipParser should not use forced Memory-Limits
13 years ago
562183932b
- removed ip_s from default profile since that needs a DNS lookup to
...
create an document entry. This makes remote search much slower.
- removed synchronization of add method if ip_s is activated to prevent
that a user configuration causes bad behavior. The disadvantage of that
is, that a index dump can cause data loss if an indexing is running
during index dump
- catched more exceptions and more NPE
- better abstraction in MirrorSolrConnector
- slight performance enhancement when only the index count is requested
(rows=0 is sufficient to get a total count)
13 years ago
24f4ca4d85
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
116f429e35
fix for java.lang.RuntimeException: TableColumnIndex not available...
13 years ago
5ac61591f3
better abstraction for solr query params
13 years ago
c913b2ba77
- fix for NPEs during remote solr configuration
...
- fixed remote solr setting switch
- added more logging
13 years ago
1533bfd63b
refactoring
13 years ago
e49359cc95
removed tenant query attribute since it is not used any more and is
...
replaced by the site-operator in the GSA interface. This operator can
also be simulated in the Solr interface using the collections_sxt field.
13 years ago
872f83ebe0
refactoring
13 years ago
fb9460f0a8
using the search filter to drill down search to file types.
...
A search like "mp3 filetype:mp3" will now maybe surprise you.
13 years ago
15ea053c3a
- added xml output in IndexControlURLs to get the storage page of index
...
dump commands
- adjusted the apicall.sh script to get the downloaded text as output to
stdout which is necessary to parse the content out of it
- added indexdump.sh script which creates a solr dump and prints out the
storage path for the index dump
- added synchronization to the Fulltext class to prevent that data is
stored to a non-existing solr index while this index is disabled during
the storage of the dump
13 years ago
1b474139dd
used the new zip writer/reader to add a solr dump process: the whole
...
solr index can be written to a zip dump and also restored during runtime
13 years ago
4a3e684f8c
added a directory-to-zip writer and zip-to-directory reader
13 years ago
d9ebf4a40f
a bit more logging
13 years ago
5683162bd3
simplifications in DHT Distribution class and more documentation
13 years ago
e57bf2ca39
simplified DHT classes
13 years ago
a053b356ee
added new classes to renovate the YaCy protocol based on simple data
...
structures in cora:
- added the Peer object, which is a fresh version of Seed
- added the Peers object, which is a fresh version of Network
- added the Network api access class to retrieve a list of peers based
on the Network.xml servlet in all YaCy peers.
13 years ago
8219a445f3
refactoring
13 years ago
f879a344e7
fix for no depth limit default value
13 years ago
00c1c777fa
refactoring
13 years ago
563d584420
removed more dependencies in cora from kelondro
13 years ago
aa65282259
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
63762d8f89
removed kelondro dependencies from cora
13 years ago
6e0f4557f8
added ftp to getName
13 years ago
23204d2245
change parameter to support the smw extension for list import
13 years ago
c235d5c0f1
fixed size parsing in RSS message parser (for YaCy size parameter)
13 years ago
5bc8f34150
fix for success query counter
13 years ago
60b1e23f05
added new crawl options:
...
- indexUrlMustMatch and indexUrlMustNotMatch which can be used to select
loaded pages for indexing. Default patterns are in such a way that all
loaded pages are also indexed (as before) but when doing an expert crawl
start, then the user may select only specific urls to be indexed.
- crawlerNoDepthLimitMatch is a new pattern that can be used to remove
the crawl depth limitation. This filter a never-match by default (which
causes that the depth is used) but the user can select paths which will
be loaded completely even if a crawl depth is reached.
13 years ago
4987921d3d
fixed the size() method which counted also failed pages (which are also
...
inside the solr index)
13 years ago
6ec02deec6
added new crawl attributes in crawl profile (not active yet)
13 years ago
975bc95ddf
added default facet fields for json response format (stub)
13 years ago
0504b01bdc
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
9413f77b65
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
a55e77a115
added twitter search heuristic
13 years ago
e54ac38095
- some corrections in usage of getFile() and getFileName()
...
- added more attributes in json response writer according to yacy
servlet
13 years ago
62add1d564
added the protocol and the file name extension to the solr fields since
...
these fields are probably facets in file search
13 years ago
e072632a54
no complaints about memory if the database is empty
13 years ago
b846f585fa
fixed a bug with size_i field usage
13 years ago
9db032664e
activate two solr fields which will be used by administration interface
...
(later)
13 years ago
fcd5c7eec3
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
6171143b4a
added facet stub in JsonResponseWriter
13 years ago
e84ffdb4f3
enhanced solr writers
13 years ago
5df553c152
- added a json writer for solr (yes there was one using xslt but this
...
one writes the same way as yacysearch.json)
- using the new json solr result to change the ajax search in
IndexControlURLs to the new solr search
13 years ago
4634f0e626
fix for images_withalt
13 years ago
e65cecc419
- updated lucene libraries to 3.6.1
...
- added lucene-grouping which enables faceted search; try this:
http://localhost:8090/solr/select?q=*:*&start=0&rows=3&facet=true&facet.field=host_s
13 years ago
4d29f59a27
removed warnings
13 years ago
8c099d2106
Merge remote-tracking branch 'origin/master'
...
Conflicts:
htroot/api/ymarks/import_ymark.java
source/de/anomic/data/ymark/YMarkEntry.java
source/de/anomic/data/ymark/YMarkTables.java
13 years ago
Michael Peter Christen