This is a major step because solr removed support for embedded solr
instances in 9.0 and we want to keep it because we want to ship
YaCy with an embedded solr. It was necessary to add parts of solr
code into YaCy to make this migration possible. Further on with
Solr 9.1 they removed even more parts which are required for embedded
operation, therefore we cannot migrate yet further without big
changes.
If you are running a YaCy instance with Solr 8.x, the migration should
be done automatically. If not you require to first migrate to a YaCy
version 1.93 with Solr 8.x to migrate to Solr 8 data.
This allows to remove another jar file from the git repo and the
whole libbuild folder.
The ant task is just a wrapper to call the makensis command. So
there should not be any drawback to remove it.
The releaseNr is now set to a static value in build.properties. We
can increment it there manually and eventually switch to another
version number scheme if we like.
The https://reproducible-builds.org project invests a lot of work
to make builds reproducible. This is a security property. It allows
to compare the build of binaries from different builder machines.
If they are identical, it means that either the builds have not
been manipulated or an attacker managed to attack all builder
machines in exactly the same way.
One problem that the reproducible-builds project often sees is
that projects include the build time in their binaries. This
makes builds unreproducible for apparently no reason. The build
date should not be of interest since binaries built on different
dates but from the same source code should not be different.
Thus I decided to remove the build date instead of re-implementing
the functionality without the GitRev task. Anyways the reported
date was not the build date but the date of the last git commit
which is even less informative. The git commit ID would have
information value but should only be relevant for "nightly builds".
PKGMANAGER is always false, thus the java code wrapped in
if statements for this property is dead code and can also
be removed.
The Debian packaging removed in c4659f0fb0
did set the PKGMANAGER property to true. When we do distro
packages again, we can revisit this commit and redo it with
property files instead.
RESTARTCMD is only used inside those dead code.
DESTDIR is never used even in the build.xml
First commit of a series to get rid of the git based versioning
implemented in libbuild/ folder as Ant task. It counts commits
since the last tagged version and uses this number added with
9000 as the last part of the version number.
This is a legacy from Subversion times.
Failed build:
https://github.com/yacy/yacy_search_server/runs/6859314856
Relevant build log:
resolve:
[ivy:retrieve] ivy.instance reference an ivy:settings defined in an other classloader. An new default one will be used in this project.
[ivy:retrieve] :: Apache Ivy non official version - :: http://ant.apache.org/ivy/ ::
[ivy:retrieve] :: loading settings :: url = jar:file:/usr/share/java/ivy.jar!/org/apache/ivy/core/settings/ivysettings.xml
BUILD FAILED
/home/yacy/actions-runner/_work/yacy_search_server/yacy_search_server/build.xml:111:
java.lang.ClassCastException: org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor
cannot be cast to org.apache.ivy.core.module.descriptor.ModuleDescriptor
Finding:
The second call to the resolve target in ant failed apparently due to
two instances of Ivy in the java runtime.
Without full investigation, the problem could be fixed by ensuring that
the resolve target is only called once within one ant build.
- added multiproject in libbuild to compile and install all needed jars (reproduceing old maven build)
- adjusted GitComInf (use a project version, build script with less hardcoded strings)
- adjusted J7Zip-modified
- include common version number for output jar
- add task installJarToRoot to copy output jar to yacycore /lib
- adjust main build with updated jar names
as files are the same - updated also old build.xml
And the Eclipse specific .classpath (with shall be deleted until complete move to gradle)
- new support library GitComInf to generate a property file for use in Gradle
- adjusted build.gradle to make use of it to create the yacyBuildProperties.java
- including additon of properties ext.filterTokens as store for Gradle
- added gradle-compile-htroot (w/o depends) to old build.xml as target until complete migration
as it is imho important
imho: much to complicated to get a running number - should be supplied by repo or simplified
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task
! it is possible that first build fails an compile of GitRevTask.jar !
! solution/workaround -> use "ant all" once to compile GitRevTask.jar !
- adjusted build.xml a little
- split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
- set build-path to reuse Gradles build directory
- (fix javadoc failure)
- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
as current (copy,delete,exclude) is complicated and not migration worthy,
used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
We will use the default value for now on.
This is much better for resource economy and fits better into a
container/docker/kubernetes strategy.
Furthermore, a small memory footprint is essential for the usage on
small devices like RaspberryPi.
- replace new guava 30 with older 25 because that is the correct
dependency for solr 8.8.1. The newer one did actually not work!
- index will be crated in a DATA/INDEX/freeworld/SEGMENTS/solr_8_8_1
subfolder. The older solr_6_6 index is not touched but also not
migrated. The index starts with fresh (empty) content.
- Older indexes must be migrated by hand (export/import) so far until a
better solution is found.
- Large schema adoptions for lucene 8.8.1
Michael Peter Christen