3dc6613096
updating slf4j 1.7.25 -> 1.7.32
3 years ago
63ad8ce6b2
removed ymarks
...
had not been used since a long time
3 years ago
fbf8ddd32d
upgrade of jsoup 1.12.1 -> 1.14.2
3 years ago
4c889b7ff9
fixed build paths
3 years ago
e6a87e0426
enhanced crawler
...
a main problem when crawling is long waiting time cuased by crawl-delay
values from robots.txt entries. that attribute is not supported by
google and interpreted by yandex and bing in different ways. In large
crawls there is always one host which blocks the whole crawl with
extreme large values. YaCy now still obeys crawl-delay but limits them
to 10 seconds.
Additionally the blocking logic when loading new robots.txt was analyzed
and a deadlock was removed. Furthermore the construction of new queue
lists was redesigned and it was ensured that always a large list of
different hosts for host-balancing is provided for the loader.
3 years ago
8a2adb2b15
upgraded commons-compress lib
...
cause: alert in
https://github.com/yacy/yacy_search_server/security/dependabot/pom.xml/org.apache.commons:commons-compress/open
3 years ago
1cdb21592b
added hazelcast and some modifications to align legacy YaCy with
...
YaCyGrid
4 years ago
88c6bc8cd7
adds missing solr lib: opentracing 0.33.0
4 years ago
8b4394a6c5
fixes for solr 8.8.1 migration
...
- replace new guava 30 with older 25 because that is the correct
dependency for solr 8.8.1. The newer one did actually not work!
- index will be crated in a DATA/INDEX/freeworld/SEGMENTS/solr_8_8_1
subfolder. The older solr_6_6 index is not touched but also not
migrated. The index starts with fresh (empty) content.
- Older indexes must be migrated by hand (export/import) so far until a
better solution is found.
- Large schema adoptions for lucene 8.8.1
4 years ago
f4f3808d43
added missing new dependencies for migration to Solr 8
...
after pulling https://github.com/yacy/yacy_search_server/pull/403
4 years ago
43a9f4f574
updated solr 6.6.6 -> 7.7.3
...
dropped GSA support (GSA API is still in YaCy Grid)
The 6.6.6 solr index works without migration also with 7.7.3
4 years ago
3213d9db37
updated jetty from 9.4.17 to 9.4.35
...
and fixed a bug in ServerSideIncludes that appeared only in that recent
version of jetty
4 years ago
dd9d4b1188
replace org.junit.Assert.assertThat by
...
org.hamcrest.MatcherAssert.assertThat from hamcrest 2.2 to avoid
deprecation-warning
4 years ago
e039a797d2
bump to commons-codec-1.14, commons-compress-1.20,
...
commons-fileupload-1.4, commons-io-2.7, httpclient-4.5.12,
httpcore-4.4.13, httpmime-4.5.12 + remove unused commons-jxpath-1.3,
htmllexer
4 years ago
59181e8009
removed old jsoup lib from eclipse classpath
5 years ago
a704ebadcd
build path fix
5 years ago
4ccd1ea3c0
new servlet path "p2p"
...
with a test class.
Call the class with
http://localhost:8090/p2p/seeds.json
5 years ago
8d3e029247
Upgraded Lucene/Solr dependencies from 6.6.5 to 6.6.6
6 years ago
385c6a079d
Upgraded Jetty dependencies from 9.4.15.v20190215 to 9.4.17.v20190418
6 years ago
f8b94f9891
Upgraded PDFBox dependency from 2.0.14 to 2.0.15
6 years ago
fc83d35f3f
Upgraded PDFBox dependency from 2.0.11 to 2.0.14
6 years ago
4c1428bd63
Upgraded Jetty dependencies from 9.4.14.v20181114 to 9.4.15.v20190215
6 years ago
d3a114c7a9
Upgraded icu4j dependency from 62.1 to 63.1
6 years ago
eeae816cc4
bump to HTTPclient-4.5.6
6 years ago
1b71c6bf87
Upgraded Jetty dependencies from 9.4.12.v20180830 to 9.4.14.v20181114
6 years ago
6c3e140083
Upgraded Solr and Lucene dependencies from 6.6.3 to 6.6.5
6 years ago
982179a7eb
Upgraded BouncyCastle dependencies from jdk15:1.46 to jdk15on:1.60
6 years ago
51f4be1807
Upgraded Jetty dependencies from 9.4.11.v20180605 to 9.4.12.v20180830
6 years ago
baa7154486
Upgraded Apache PDFBox dependency from 2.0.9 to 2.0.11
...
Release notes at
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310760&version=12343466
and https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310760&version=12342889
6 years ago
685122363d
Added a parser for XZ compressed archives.
...
As suggested by LA_FORGE on mantis 781
(http://mantis.tokeek.de/view.php?id=781 )
6 years ago
1a91e87b05
Upgraded commons-compress dependency from version 1.16.1 to 1.17
6 years ago
2bbf070f57
Upgraded icu4j dependency from 61.1 to 62.1
6 years ago
8811700e2e
Upgraded Jetty dependency from 9.4.9 to 9.4.11
6 years ago
d5af160e60
upd to slf4j-1.7.25
7 years ago
7525594315
upd to jwat-warc-1.1.1
7 years ago
b81debca2e
upd to jsoup-1.11.3
7 years ago
508050f79c
upd to icu4j-61.1
7 years ago
e7971fb888
upd to pdfbox-2.0.9
7 years ago
e2b2c89feb
upd to jetty-9.4.9.v20180320
7 years ago
c867a52d96
Upgraded Solr dependencies from 6.6.2 to 6.6.3
7 years ago
a57a04a003
upd to commons-codec-1.11
7 years ago
5753ce0ac5
Upgraded Jaudiotagger dependency from 2.0.3 to 2.2.5
7 years ago
aaa0ec6613
upd to commons-compress-1.16.1
7 years ago
73c6ce7ae5
upd to httpclient-4.5.5
7 years ago
5aa4fb1144
upd to metadata-extractor-2.11.0.jar
7 years ago
cedb53be4e
upd to commons-io-2.6
7 years ago
270b77074e
upd to httpclient-4.5.4 and httpmime-4.5.4
7 years ago
6db7f5525b
upd to icu4j-60.2
7 years ago
c94bc82f6a
upd to commons-compress-1.15
7 years ago
e5b4799838
upd to Jetty-9.4.8.v20171121
7 years ago
Michael Peter Christen