c67c5c0709
added new solr schema fields which record the occurences of vocabulary
...
matchings. These matches can be used for result boosting, i.e. if a
document contains words from a specific vocabulary, boost it.
10 years ago
a67a465415
fix field counter for multi-fields in html writer for the solr servlet
10 years ago
ec9d021568
added option in vocabulary editor to import CSV files with different
...
encodings (preselected windows-type character encoding which is typical
for CSV files). Fixed also other problems with character encoding in
dictionary files. Automatically generated vocabularies are now also
noted in the API steering.
10 years ago
3c818fc912
add a check of java version string >=1.7 to startup class
...
stopping start with error msg on version < 1.7
10 years ago
0550b54d56
added fix to postprocessing: avoid caching of postprocessing collection
...
to always get fresh lists of documents. This is necessary since the
postprocessing changes the same documents which the
postprocessing-collection query selects.
10 years ago
68e8039fd1
added high-precision scheduler for API processes. This allows also to
...
make the execution in dependency of available RAM or CPU load. The
default value for CPU load is 4.0 and the check runs once a minute.
10 years ago
8aee7f940e
added missing class for latest changes
10 years ago
97039049e4
fix in key enumeration methods for cases where the enumeration is done
...
in reverse order.
10 years ago
7e1b0b6712
fix for wildcard patch in search queries
10 years ago
0a879c98e7
added new 'firstSeen' database table and necessary data structures which
...
hold a date for each URL to record when a url was first seen. This is
then used to overwrite the modification date for urls upon recrawl in
case that the first-seen date is before the latest document date. This
behaviour is necessary due to the common behaviour of content management
systems which attach always the current date to all documents. Using the
firstSeen database it is possible to approximate a real first document
creation date in case that the crawler starts frequently for the same
domain. As a result the search results ordered by date have a much
better quality and the usage of YaCy as search agent for latest news has
a better quality.
10 years ago
421ee64f33
another fix to ordering of table indexes; fixes also network stats
...
graphics
10 years ago
1db476c67e
fix for bad table iteration
10 years ago
e4316e2d74
skip creation of local var in proxyhandler.storetocache
10 years ago
9c6e3a6b1c
fix assertation-failure in version-string for Solr-4.10.2 by changing
...
the assert - hope that is ok
+ add forgotten NB-Projekt-changes
10 years ago
725b206fb4
update to solr-/lucene-4.10.2
10 years ago
5c97ecb30f
fix of bad query generation for search facets
10 years ago
95d87f00b3
fix for bad query generation in doublecheck in postprocessing
10 years ago
72c2bc5189
fix for search in case where local peer has no local seed address in
...
portal mode
10 years ago
5be352da99
Merge branch 'master' of git@gitorious.org:yacy/rc1.git
10 years ago
0fcd8097a3
removed unused options from BusyThreads
10 years ago
fe8b1d137d
emergency bugfix for 100% CPU in image drawing
10 years ago
92007e5d2d
more enhancements to posprocessing speed
10 years ago
9a7fe9e0d1
fix for bad timing computation in postprocessing
10 years ago
bd16119a00
another fix for postprocessing (the query for "" on numeric field did
...
not work in external solr)
10 years ago
327e83bfe7
more fixes in postprocessing: partitioning of the complete queue to
...
enable smaller queries
10 years ago
2bc6199408
more concurrency for postprocessing
10 years ago
a83cf26c38
more fixes and enhancements to postprocessing
10 years ago
71758f0d62
enhanced postprocessing by usage of a field-list generation to prevent
...
lazy initialization of the documents. This is useful because the
documents must be read completely anyway.
10 years ago
7856fbdbe8
fix for npe (in rare cases)
10 years ago
8a2b569d7c
fix for literal computation
10 years ago
856da2712b
Merge branch 'master' of git@gitorious.org:yacy/rc1.git
10 years ago
ca9cd7b58a
more IPv6 fixes
10 years ago
b4585e9546
added new index size history image in /Status.html page
10 years ago
167c5a51f0
IPv6 fix
10 years ago
fe537679de
fix for exact_signature_unique_b, exact_signature_copycount_i,
...
fuzzy_signature_unique_b and fuzzy_signature_copycount_i: apply same
criteria for 'valid document' as for title and description uniqueness
test.
10 years ago
eb9d2705d2
fix for ConnectionInfo.cleanup of server-connections
10 years ago
2e5214eb21
added field postprocessing.partialUpdate to settings which can be used
...
to switch on or off partial updates. Both options should cause the same
result. Default is on.
10 years ago
11074d8d24
fix for a ssl bug that appear only in java 7.
...
The bug was reported in
http://forum.yacy-websuche.de/viewtopic.php?f=23&t=5407&p=30956#p30956
a solution was described in
http://teknosrc.com/javax-net-ssl-sslprotocolexception-handshake-alert-unrecognized_name-solved/
which worked for this example given in the yacy forum
10 years ago
e96490e3a1
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
77662e08e1
concurrently initialize the error cache; extended also the cache by
...
factor 10 up to 1000 entries. This error cache is only used to catch up
paused crawls between shutdown+startup
10 years ago
d8fcc4a2f5
added a timeout on Jetty connectors
10 years ago
0f0b60404b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
72561926aa
do not overwrite yacy.conf in case of an exception
...
may be a fix for http://mantis.tokeek.de/view.php?id=180
10 years ago
07c5b57953
removed warnings
10 years ago
fa2ad101ec
enhanced graphics computation (avoiding long string parsing for colours)
10 years ago
ef813cec91
added proper copyright notice to OSM tiles presented at the search
...
result page
10 years ago
fca11701f0
better profiling of solr queries
10 years ago
2e09da9832
npe fix
10 years ago
d80418f1b1
added partial updates to solr during postprocessing: during
...
postprocessing the solr documents are now not completely retrieved.
instead, only fiels, needed for the postprocessing are extracted. When
Solr document are written, this is done using partial updates.
This increases postprocessing speed by about 50% for embedded Solr
configurations. For external Solr configurations the enhancement should
be much higher because the postprocessing with remote Solr is very slow.
When doing partial updates to a remote Solr, this method should perform
much better than before, it is expected that this is even much higher
than the increase with local Solr.
10 years ago
b1cfbc4a04
added new solr field url_paths_count_i which can be used to enhance the
...
index browser and maybe also for ranking; possibly also for
SEO-with-YaCy applications.
10 years ago
e69883d5ab
fix-fix for
...
30d4402cd1
10 years ago
30d4402cd1
fixed location search
10 years ago
6983dff334
explain crawl denial when not switched to intranet mode
10 years ago
f818f84adb
more ipv6 fixes
10 years ago
afd5bd5f5f
slightly enhanced Network table computation by using a lazy initialized
...
bitfield for peer flags
10 years ago
2c2b50e65d
refactoring (class name should start with uppercase letter)
10 years ago
bc275dca07
added network history graph image /NetworkHistory.png which can show
...
many different statistics about the history of the peer.
10 years ago
ce9368246b
Merge branch 'master' of gitorious.org:yacy/rc1
10 years ago
5603809deb
Minor changes:
...
*) reduced visibility of a method
*) updated comments
10 years ago
d8beafba3a
fix for values in CrawlProfileEditor table and xml; now the full profile
...
is available in the xml.
10 years ago
ec95dfa2e6
fixed crawl profile xml result which did not show the correct crawl
...
status.
10 years ago
8c1a89cb34
added another decoration flag to switch off network graphics in crawler
...
monitor and index browser: decoration.grafics.linkstructure
Please set this to false to remove the graphics from the interface.
10 years ago
ee27be3399
misc bugfixes (concurrency, memory protection)
10 years ago
9b1958e8ca
more ipv6 bugfixes
10 years ago
7817fc50c9
added a high cpu cycle monitor to PerformanceQueues
10 years ago
5082feb103
less volume for effect sounds
10 years ago
e8392e2ff2
fix for local search
10 years ago
0bfc69b29b
more ipv6 bugfixes
10 years ago
a27563e5c3
removed the atmo sound clips because they had been too large
10 years ago
883622306e
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/peers/Protocol.java
10 years ago
97995a1dd9
fix for remote search process
10 years ago
0843b12ef3
ipv6 fix: avoid that shrinked own ip set is overwritten with (non-valid)
...
set of local IPs
10 years ago
92c5d97486
fix for bad node flag setting with IPv6
10 years ago
c27bad9326
more ipv6 fixes
10 years ago
cddf884bc4
Merge branch 'master' of git@gitorious.org:yacy/rc1.git
10 years ago
460858fb22
more ipv6 fixes
10 years ago
5cef88a315
argh.. adding missing java class for latest audio feature
10 years ago
74957f3760
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
2a052f446a
Added an experimental audio feedback system.
...
This is the first element of a new 'decoration' component which may hold
switches for different external appearance parameters.
The first switch in that context is decoration.audio (as usual in
yacy.init). This value is set to false by default, that means the audio
feedback element is switched off by default. To switch it on, set
decoration.audio = true (using /ConfigProperties_p.html). You will then
hear sounds for the following events:
- remote searches
- incoming dht transmissions
- new documents from the crawler
Sound clips are stored in htroot/env/soundclips/ which is done so
because a future implementation will read these files using the http
client and with configurable urls which will make it very easy for the
user to replace the given sounds with own sounds.
10 years ago
1e6e69bc40
Finished implementation of UPNP:
...
*) will try other ports if YaCy standard ports are not available
*) distinguish between internal and external port (not sure if this
works 100%)
Still to add: propery in config to enter own external port (in case of
manually configured NAT)
10 years ago
d0358e568b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
e1bc768f9d
more IPv6 bugfixes
10 years ago
59c6532a65
add link extraction to pdfParser
...
this extracts clickable links in pdf and adds it to the list of links
include a test case for this function
this is the corrected comment for commit:
aa2e15d846
10 years ago
aa2e15d846
allow url parameter in worktable apicall
...
allow url=wwwl?param=a¶m=b (with ?, & encoded)
fix: http://mantis.tokeek.de/view.php?id=100
fix double adding of '&' in MultiProtocolURL.escape()
10 years ago
f3a12801f0
Merge branch 'master' of git@gitorious.org:yacy/rc1.git
10 years ago
d93325a578
lazy handling of process_sxt field (part of postprocessing)
10 years ago
b31db00010
toString fixes
10 years ago
961f06c0b6
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
209e0f2fe8
allow url parameter in worktable apicall
...
allow url=wwwl?param=a¶m=b (with ?, & encoded)
fix: http://mantis.tokeek.de/view.php?id=100
fix double adding of '&' in MultiProtocolURL.escape()
10 years ago
b5ca20de15
preserve content_type (mime) if supplied in preference of construct in from file type.
...
(this eventually can benefit image search by using mime only)
reduce redundant field assignment for Solrdocuments created from URIMetadataNode (URIMetadataNode = SolrDocument with partially assigned fields)
10 years ago
fe9f1c594e
fix char encoding parameter in UrlProxy
10 years ago
b0c87d8240
fix image search expand box, cut-off of 2nd capture line height
...
tested with IE11 and Firefox 32 (change worked for both to show 2nd line without cutting off height)
+fix charset parameter in metadataImageParser
+update start errMsgTxt to "java 1.7"
10 years ago
2c2ed8bf4e
typo in javadoc
10 years ago
528f583d72
ipv6 fixes
10 years ago
6ee5b4352d
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
247e626083
IPv6 host parsing bugfixes
10 years ago
fb1fcc2b03
handle noarchive tag, skip writing page to cache
...
http://mantis.tokeek.de/view.php?id=44
10 years ago
fe917deb2d
when pinging other peers, be able to select the right IP option
10 years ago
65e6ae52fb
IPv6-enhanced Network monitoring page
10 years ago
3073c69aee
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
Michael Peter Christen