4ba492052e doc: Add minimum supported Android NDK version (Hennadii Stepanov)
6393bdcd53 doc: Move Android dependencies guide into `build-android.md` (Hennadii Stepanov)
ac323a7222 build: Switch to llvm buinutils for Android builds (Hennadii Stepanov)
Pull request description:
The new Long Term Support release of the Android NDK is [available](https://groups.google.com/g/android-ndk-announce/c/MS6Qoub0DKE/m/Zfp5Ys8eAAAJ) since 2021-08-11:
> As r23 is the new LTS, the support windows for r21 and r22 have now ended.
On master (8ae4ba481c), dependency build fails because it expects GNU Binutils are present in the Android NDK. In [fact](https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md#binutils):
> GNU Binutils remains available up to and including r22. All binutils tools with the exception of the assembler (GAS) were removed in r23. GAS was removed in r24.
This PR switches our depends build system to llvm binutils. The usage of `llvm-ar` and `llvm-ranlib` tools effectively makes r21 the minimum supported version of NDK.
With this PR:
- building depends against NDK r23 LTS now is possible with `NO_QT=1`
- building the `qt` package in depends against NDK r23 LTS still fails:
```
Creating qmake...
...
ERROR: Cannot detect Android NDK toolchain.
Please use -android-toolchain-version to specify it.
```
The issue with the `qt` package is going to be addressed in another PR.
ACKs for top commit:
fanquake:
ACK 4ba492052e
Tree-SHA512: cdc8f95ff9a3ad7f12eb55b9ea18b6b6b800d4cceff7e0321985be6e39d15a2b2ea5b1592972307d76d111292a0ed58fd287e5ca285e2f6868b42a286536d310
We need to append-to rather than set CXXFLAGS, otherwise we loose -O2 &
-pipe. Currently this results in zeromq being built without optimizations
at all (or whatever the compiler would default too, essentially always -O0).
Bdb is the same, for the CXX portion of its code. C code has been built
with -O2.
Boost has actually been uneffected because it receives -O3 from it's own
build flags.
acaac6e86a ci: Bump Android NDK to r22 which supports std::filesystem (Hennadii Stepanov)
cac7890386 build: Add support for Android NDK r22+ (Hennadii Stepanov)
Pull request description:
This is required to support [`std::filesystem`](https://github.com/android/ndk/wiki/Changelog-r22#changes) on Android (see #20744).
Fixes#22074.
ACKs for top commit:
icota:
re-tACK acaac6e86a
Tree-SHA512: ecbec374ee590c4cb30012210f1422d469e7e8b68989f9eb53d36b5feee150d31e6bd10e1fc4a2056fbf4f8f8513e435b446e5feaf21a3a4d09dfc561fb22e73
54c7754f31 build: use aarch64 Clang if cross-compiling for darwin on aarch64 (fanquake)
Pull request description:
If we're cross-compiling for darwin on aarch64 hardware, we need to
use a Clang that will run on that hardware.
Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).
ACKs for top commit:
hebasto:
ACK 54c7754f31, I agree it can be merged (fix in #22448 is orthogonal to this one).
Tree-SHA512: 66c530097a5dc072a0a00dc22eb3d4a7d923dfa8ab8160f7c3e395cbe58da324f367548d673c0510606f5225d5d37bb5607a76b1703b8b03ac7d2cceeccbd542
1155978d8f build, qt: Do not install *.prl files (Hennadii Stepanov)
763793b60e build, qt: Fix wrong cross-compiling detection on macOS (Hennadii Stepanov)
30982721ab build, qt: Force bootstrap while building linguist tools (Hennadii Stepanov)
689320e307 build, qt: Drop translations.pro hack (Hennadii Stepanov)
6a1f98f253 build, qt: Drop lrelease dependency patch (Hennadii Stepanov)
39e561e087 build, qt: Add linguist_tools list (Hennadii Stepanov)
27d3def1c6 build: Use Qt top-level build facilities (Hennadii Stepanov)
Pull request description:
This PR:
- uses Qt top-level build facilities without the need to download all-in-one archive
- is based on **BlockMechanic**'s [idea](https://github.com/bitcoin/bitcoin/pull/20600), and is an alternative to #20600
- makes it easy to integrate [new modules](https://github.com/bitcoin/bitcoin/pull/16883) into static builds
- has the minimal diff
- makes the qt package build process streamlined by dropping some patches and hacks (an alternative to #21420 and #20642)
Fixes#18536 (a non-intrusive alternative to #21589 and #19785).
Fixes#14648.
Fixes#21588 (a non-intrusive alternative to #21591).
Required for adding [Wayland support](https://github.com/bitcoin/bitcoin/issues/19950) on Linux.
---
**Note for reviewers**: With 9046de8a4cbc3899fed9eae084115f423e7ac5bd from #21995 it is easy to verify that there are no changes in the resulted `qt` package archive on the per commit basis. For example, for `HOST=i686-pc-linux-gnu` no commit in this PR introduces any changes.
ACKs for top commit:
fanquake:
ACK 1155978d8f
Tree-SHA512: 667b06b72cb7ff26d68b9b88e8dddb51084783ca9e3d80b3392710794c1dc7fd77bbcc3ccf4ccece9919d33b9bf8fce13c5059502bd228021dc7c93fdb87ca7a
fb7be92b09 Mark print-% target as phony. (Dmitry Goncharov)
Pull request description:
.PHONY does not take patterns (such as print-%) as prerequisites.
Have print-% depend on force and mark force as phony.
This change ensures print-% rule works even when there is a file that matches the target.
```
$ # on master
$ make print-host
host=x86_64-pc-linux-gnu
$ touch print-host
$ make print-host
make: 'print-host' is up to date.
$
$ git co mark_print_as_phony
Switched to branch 'mark_print_as_phony'
$ make print-host
host=x86_64-pc-linux-gnu
$ touch force
$ make print-host
host=x86_64-pc-linux-gnu
```
ACKs for top commit:
hebasto:
ACK fb7be92b09, tested on Linux Mint 20.2 (x86_64).
Tree-SHA512: b89ae66aa8c7aa6a7ab5f0956f9eb3b3ef9d56994b60dc2a97d498d4c1bba537845c190723e8a10310280b1b35df2cd935cc30aeb76735cac2dc621ad7823772
If we're cross-compiling for darwin on aarch64 hardware, we need need to
use a Clang that will run on that hardware.
Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).
.PHONY does not take patterns (such as print-%) as prerequisites.
Have print-% depend on FORCE and mark FORCE as phony.
$ # on master
$ make print-host
host=x86_64-pc-linux-gnu
$ touch print-host
$ make print-host
make: 'print-host' is up to date.
$
$ git co mark_print_as_phony
Switched to branch 'mark_print_as_phony'
$ make print-host
host=x86_64-pc-linux-gnu
$ touch FORCE
$ make print-host
host=x86_64-pc-linux-gnu
a58868d201 build: Makes rcc output always deterministic (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2.
The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732).
This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless.
---
Also it improves interaction with ccache:
On master (f6c44e999b):
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:45:43 2021
stats zeroed Sun Apr 11 15:45:05 2021
cache hit (direct) 638
cache hit (preprocessed) 0
cache miss 1
cache hit rate 99.84 %
called for link 10
cleanups performed 0
files in cache 20023
cache size 13.2 GB
max cache size 15.0 GB
```
The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`.
With this PR:
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:28:46 2021
stats zeroed Sun Apr 11 15:28:21 2021
cache hit (direct) 639
cache hit (preprocessed) 0
cache miss 0
cache hit rate 100.00 %
called for link 10
cleanups performed 0
files in cache 20012
cache size 13.2 GB
max cache size 15.0 GB
```
ACKs for top commit:
fanquake:
ACK a58868d201
Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
3caedb4c03 build: don't use cf-protection when targeting arm-apple-darwin (fanquake)
Pull request description:
After two reports on IRC of issues building depends on an Apple M1 machine, this option (obviously) can't be used when targeting `arm-apple-darwin`. For now, just use it for `x86_64-apple-darwin`.
```bash
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.4.0
error: option 'cf-protection=return' cannot be specified on this target
error: option 'cf-protection=branch' cannot be specified on this target
2 errors generated.
```
ACKs for top commit:
promag:
Tested ACK 3caedb4c03.
Tree-SHA512: 8763a5b94000016b0c2f0438e66002fdfcd2cbafd9d2d2acc1972f0c6f707e820186711dbd9d3f72673c179718da75588acb4732f8d84b85f0c1dfc862b6e944
After two reports on IRC of issues building depends on an Apple M1
machine, it turns out that this option can't be used when targeting
arm-apple-darwin. For now, just use it for x86_64-apple-darwin.
```bash
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.4.0
error: option 'cf-protection=return' cannot be specified on this target
error: option 'cf-protection=branch' cannot be specified on this target
2 errors generated.
```
42b589d18f scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin (fanquake)
Pull request description:
Addresses the macOS portion of #21888.
Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.
Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
From the Intel control flow enforcement documentation:
> The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.
Follow up from #21135.
Guix builds:
```bash
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
```
Gitian builds:
```bash
# macOS:
bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96 bitcoin-42b589d18fed-osx-unsigned.dmg
f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9 bitcoin-42b589d18fed-osx-unsigned.tar.gz
29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7 bitcoin-42b589d18fed-osx64.tar.gz
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 src/bitcoin-42b589d18fed.tar.gz
366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98 bitcoin-core-osx-22-res.yml
```
ACKs for top commit:
laanwj:
Code review ACK 42b589d18f
Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
b95f7f8ac0 build, qt, refactor: Drop sed commands for win32-g++/qmake.conf (Hennadii Stepanov)
Pull request description:
Such possibility is [available](https://codereview.qt-project.org/c/qt/qtbase/+/165348) since Qt 5.8.0.
ACKs for top commit:
fanquake:
ACK b95f7f8ac0
Tree-SHA512: e56a3d208a6bd5d42c722f8b344010fe7d1b6f7a28486613dfcb03f0403a47cee8476e2366eeaac401a19836cd09f782e8741a1e781ab4d78f72c500a30e4929
Hennadii Stepanov
