e8acd542b5
- added faceted drill-down for host and geolocation to solr queries
...
- added a new geolocation field to index schema, the old values are
migrated if possible
13 years ago
af764c106c
re-activated audio and video search because they obviously work (!)
13 years ago
716ea0cfe2
sorted the solr schema into mandatory and optional fields; reduced
...
number of used field to reduce solr index size
13 years ago
db6863db77
reduced solr cache sizes to check if that solves memory problems a bit
13 years ago
23226676c6
FOR THE BRAVE.. this is a forced migration to solr which is now ready
...
for production as a replacement of the metadata-db.
This intermediate release 1.041 will switch on the previously optional
solr index and the old metadata-db will still work as it did before.
Solr+metadata are accessed in mixed mode, no migration is done yet.
If this causes not a catastrophe until the end of the weekend, we will
do a YaCy 1.1 main release containing this as default.
13 years ago
a1b2c9a67d
doctype2mime fix, influences metadata conversion between old metadata
...
and solr
13 years ago
703f427303
fixed some peer-ping connection details
...
- larger time-out
- removed too old seedlist
- fixed a bug in connection test
13 years ago
ea49a8aa8c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
aab0b680c3
- added xslt support for solr result formats.
...
try i.e.
http://localhost:8090/solr/select?q=*:*&start=0&rows=10&wt=xslt&tr=json.xsl
- added servlet-side mime-type configuration for streamed servlets. this
is used for the result formatters in solr result formats
13 years ago
e2119f4e76
augmented browsing: replace htmlparser by jsoup, which is more stable
...
and reliable
13 years ago
b51df6c7e8
- added coordinate storage in solr schema
...
- fixed shutdown process
- fixed some solr-to-metadata reading
- added a large number of metadata attributes in ViewFile.html
13 years ago
f9c0e6e950
- Implemented and integrated the URIMetadataNode object which is a
...
metadata representation from the solr index. This shall replace metadata
from the built-in database in the future.
- added the Solr-driven metadata into the search index of YaCy which
makes it now possible to run YaCy without the old metadata index. This
is a major stept forward to a full migration to Solr.
13 years ago
bca4a16603
replaced the multivalue generic string field name suffix _ss by _txt
...
because _ss is not part of the standard solr example schema.
13 years ago
67edfd991c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
d9173ba7ed
added more solr fields to integrate values from URIMetadataRow. All
...
writings to the Metadata-DB are now also done to solr. This includes
metadata transfer during search and rwi transfer.
The new/added solr fields are:
## time when resource was loaded
load_date_dt
## date until resource shall be considered as fresh
fresh_date_dt
## id of the host, a 6-byte hash that is part of the document id
host_id_s
## ids of referrer to this document
referrer_id_ss
## the md5 of the raw source
md5_s
## the name of the publisher of the document
publisher_t
## the language used in the document; starts with primary language
language_ss
## an external ranking value
ranking_i
## the size of the raw source
size_i
## number of links to audio resources
audiolinkscount_i
## number of links to video resources
videolinkscount_i
## number of links to application resources
applinkscount_i
13 years ago
3ce04cecf3
bad hack to prevent a bug appearing in solr
13 years ago
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
13 years ago
1517a3b7b9
added webm mime-type
13 years ago
0301aba1e9
removed unused method parameters
13 years ago
4de50fe808
adding more principal peers for bootstraping
13 years ago
067728bccc
add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages)
13 years ago
508a81b86c
added solr field 'refresh_s' which stores the refresh url contained in
...
the meta-refresh html header field.
13 years ago
9116013c64
- allow lazy initialization of solr value (if using 'lazy', then no
...
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
13 years ago
c03d306afa
shorter autocommit time (now: 1 second) to prevent that user cannot see
...
results in solr the first time they try it out. The value can now be
easily set to a higher number using the IndexFederated_p interface.
13 years ago
3fd4a01286
added option to record urls that are forwarded to the solr index
13 years ago
8dd469b9dd
added option to configure the autocommit delay time of solr on-the-fly
13 years ago
b9dfca4b0a
- fixed IndexFederated Servlet / a embedded Solr can now be selected
...
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
13 years ago
1be0025a9c
- added test for EmbeddedSolrConnector
...
- added needed libraries for this test
this includes most (all) files needed for an embedded solr
13 years ago
dbdd697f4d
moved RDFaParser.xsl configuration file to defaults
13 years ago
8738336408
set Xms lower than Xmx
13 years ago
96f6a5869f
more robust OAI-PMH client (large time-out, three re-tries). OAI-PMH
...
server appeart to be very slow sometimes
13 years ago
6d17686258
made triplestore persistent by default
...
added a size display in triplestore servlet
13 years ago
3c255c025b
Show tags in search results (if activated in ConfigPortal_p.html)
13 years ago
a5cdfb91de
- fixed Cache link (below snippet)
...
- added 'Augmented Proxy' link below snippet
- added configuration options for augmented proxy
13 years ago
af5a597e47
Scroogle is not comming back, remove dead code
...
Conflicts:
source/net/yacy/search/Switchboard.java
13 years ago
90512640bf
Added config switches for custom parser
...
Conflicts:
source/net/yacy/document/TextParser.java
13 years ago
5d20cd324a
Add Triplestore and RDF query interface
...
Conflicts:
build.xml
defaults/yacy.init
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
a32943b382
add json mimetype
13 years ago
41c02cb10e
- less restrictions for usage of Table RAM copy
...
- new limit to use the table copy (instead of flag): 400MB available. If
less is available, then a copy is never used. If more is available, then
it can be used if there is a remaining space of at least 200MB
- flush caches more often: flush the Digest cache
13 years ago
8002fd2578
use less cache space since a large cache would cause more memory usage
...
in index files.
13 years ago
5aee19daa4
added show from cache in search results (not yet finished)
13 years ago
0d32a766ed
relax verify attribute for search widget to make it faster:
...
set to "cacheonly"
13 years ago
7eece0256f
moved yacy.logging to defaults according to request in
...
http://bugs.yacy.net/view.php?id=55
13 years ago
db9d81cb7a
ups
13 years ago
e7e381d110
added configuration to switch off redirection following in crawler
13 years ago
2be327b5ab
update location update
13 years ago
99c74699de
removed scroogle (scroogle is dead)
13 years ago
8bee1472c9
there is no noindex, only nofollow in links
13 years ago
4c5edab1ec
added option to have exception search result windows
13 years ago
696ee5fc16
removed pdf from default parser deny list
13 years ago
Michael Peter Christen