Patch in suport for using -no_fixup_chains, with ld64. This option just
seems to be missing from our version, as it exists in later releases.
This is needed so we can disable fixup_chains in our security checks.
Unfortunately clang 10 does not understand "-mmacosx-version-min=11.0",
as it expects to see only 10.x.
Bump minimally to 11.1 to fix that problem. This will likely be our last
binary toolchain bump, as it will soon be replaced with usage of upstream
vanilla llvm.
Use of -stdlib++-isystem gets rid of any system c++ header include paths and
negates the need for this option. In newer versions of clangs the combo
produces a warning.
This will allow us to use the existing machinery for filtering by
arch, os, debug/release, etc.
For example, the following becomes possible for libevent:
$(package)_config_opts_release=-DEVENT__DISABLE_DEBUG_MODE
Now the define is only set when not building depends with DEBUG=1
It was deprecated in LLVM 15, turned into a compile-time error in LLVM 16:
```bash
In file included from /usr/lib/llvm-16/bin/../include/c++/v1/cassert:19:
/usr/lib/llvm-16/bin/../include/c++/v1/__assert:22:5: error: "Defining _LIBCPP_DEBUG is not supported anymore.
Please use _LIBCPP_ENABLE_DEBUG_MODE instead."
^
1 error generated.
```
and has been removed entirely in LLVM 17 (main),
ff573a42cd.
Building libc++ in debug mode, will also automatically set
`_LIBCPP_ENABLE_DEBUG_MODE` (the new define), so adding it to depends
doesn't seem useful, and would just result in redefinition errors.
I'm wondering if as a followup, we could enable a DEBUG build of libc++
in our MSAN CI job?
Somewhat related to https://github.com/google/oss-fuzz/pull/9828, where
it looks like we'll have to sort out getting a DEBUG build of LLVM.
Compiling bdb with clang-16 on aarch64 (hardware) currently fails:
```bash
make -C depends/ bdb CC=clang CXX=clang++
...
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
```
Looking at config.log we've got:
```bash
configure:18704: checking for mutexes
configure:18815: clang -o conftest -pipe -std=c11 -O2 -Wno-error=implicit-function-declaration -Wno-error=format-security -I/bitcoin/depends/aarch64-unknown-linux-gnu/include -D_GNU_SOURCE -D_REENTRANT -L/bitcoin/depends/aarch64-unknown-linux-gnu/lib conftest.c -lpthread >&5
conftest.c:45:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
main() {
^
int
conftest.c:50:2: warning: call to undeclared library function 'exit' with type 'void (int) __attribute__((noreturn))'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
exit (
^
conftest.c:50:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 warning and 1 error generated.
```
Clang-16 changed `-Wimplicit-function-declaration` and `-Wimplicit-int`
warnings into errors, see:
https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#potentially-breaking-changes.
> The -Wimplicit-function-declaration and -Wimplicit-int warnings now
> default to an error in C99, C11, and C17. As of C2x, support for implicit
> function declarations and implicit int has been removed, and the
> warning options will have no effect. Specifying -Wimplicit-int in
> C89 mode will now issue warnings instead of being a noop.
Add an option that when passed, will disable hardening options, and
pass `--disable-hardening` through to configure. Due to the way
we link libssp for Windows builds, they now fail (after #27118),
if building with depends, and configuring with --disable-hardening.
See:
https://github.com/bitcoin/bitcoin/pull/27118#issuecomment-1492606272.
This change would add a depends opiton such that, if someone wants to
build with, for windows, without hardening, they can do so. This may
also be useful when building for debugging.
Use the same workaround we've applied to qrencode, and other packages.
Fontconfig not building is currently a blocker for fuzz/sanitizer infra
upgrades.
For now, this is also more straightforward than bumping the package,
which introduces more complexity/usage of gperf.
87afcb0029 depends: fix osx build with clang 16 (Cory Fields)
Pull request description:
Current build (using forced system clang as a test) results in:
> error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include'
For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.
See [here for an example of upstream using this syntax](https://github.com/llvm/llvm-project/blob/main/clang/test/Driver/crash-report-with-asserts.c#L9).
There is no change in behavior for previous versions.
I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.
ACKs for top commit:
TheCharlatan:
ACK 87afcb0029
hebasto:
ACK 87afcb0029
Tree-SHA512: 127037c888c37c6ccd9679e96da34037cc43ccdc07915865a0a5494edb62633e83fc1bd6b1c4bb7a0322f5b59622e10090a31987f38496fb6b306488e9941594
0e02f72548 depends: define `__BSD_VISIBLE` for FreeBSD bdb build (fanquake)
Pull request description:
Required for additional definitions (`IPC_R` & friends), to be available, when compiling under C11, which would otherwise cause compile fails.
See: https://github.com/MarcoFalke/btc_nightly/pull/4.
ACKs for top commit:
hebasto:
ACK 0e02f72548, tested on FreeBSD 13.1:
Tree-SHA512: 885d4aa341d9668da360cf6dfafb97ce816803c54e76c0a06e448db39a723666d42cd14b3e713d17ecbe33163f5af69924567cf449d679a2db95b36357005d43
1914e470e3 build: copy config.{guess,sub} post autogen in zmq package (fanquake)
Pull request description:
Otherwise our config.guess and config.sub will be copied over. This problem has been masked by the fact that modern systems ship with versions that recognise all the triplets we use (namely arm64-apple-darwin). However building on ubuntu 20.04 surfaces the issue.
Fixes#26420.
ACKs for top commit:
hebasto:
ACK 1914e470e3, tested on Ubuntu 18.04.
Tree-SHA512: dff64c3c62d9f8fc205e5a4dffe8befd58838418d073a15dfe304a0f64b182dfffd9dcf98b53df44bfab905c12a62d03cd5c0f91fa7c4b246ac21ae5f20540fd
a3a2bd9e8a ci: Drop no longer needed package-specific flags (Hennadii Stepanov)
071eef1e97 build: Propagate user-defined flags to host packages (Hennadii Stepanov)
Pull request description:
On master (4f8b1f8759) `{CPP,C,CXX,LD}FLAGS` that are specified in the command line are not propagated to packages:
```
$ make --no-print-directory -C depends print-libevent_cxxflags CXXFLAGS=-some-fancy-flag
libevent_cxxflags=-pipe -O2
```
This PR:
- propagates `{CPP,C,CXX,LD}FLAGS` to host packages:
```
$ make --no-print-directory -C depends print-libevent_cxxflags CXXFLAGS=-some-fancy-flag
libevent_cxxflags= -some-fancy-flag
```
- does not propagate `{CPP,C,CXX,LD}FLAGS` to native packages:
```
$ make --no-print-directory -C depends print-native_b2_cxxflags CXXFLAGS=-some-fancy-flag
native_b2_cxxflags=
```
- actually addresses the https://github.com/bitcoin/bitcoin/pull/23551#issuecomment-973896518
ACKs for top commit:
TheCharlatan:
Code review ACK a3a2bd9e8a
Tree-SHA512: 243d6b1b0e9c5de46debc36de62a77b6b4d6f638940fd530040c219956ec624e321b0c25290fed164e3a8c88befa7b97b20f765d7b9a428c269b3720f21da099
978852aad8 build: Fix depends build system when working with subtargets (Hennadii Stepanov)
Pull request description:
On master (f3e0ace8ec), the depends build system does _not_ guarantee that dependencies packages are available for `$(package)_built` target because these dependencies being prepared in `$(host_prefix)` at `$(package)_configured` target can be wiped out during building other package.
Please consider:
```
$ cd depends
$ make clean
$ make fontconfig_configured
$ make
...
CC fcdir.lo
In file included from fcftint.h:26,
from fcdir.c:26:
../fontconfig/fcfreetype.h:27:10: fatal error: ft2build.h: No such file or directory
27 | #include <ft2build.h>
| ^~~~~~~~~~~~
compilation terminated.
make[4]: *** [Makefile:642: fcdir.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
make[4]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
make[3]: *** [Makefile:503: all] Error 2
make[3]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
make[2]: *** [Makefile:581: all-recursive] Error 1
make[2]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
make[1]: *** [Makefile:465: all] Error 2
make[1]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
make: *** [funcs.mk:288: /home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/./.stamp_built] Error 2
```
The following commands:
```
$ cd depends
$ make clean
$ make qt_configured
$ make
```
also fail.
The similar issue was reported earlier: #21381.
This PR guarantees that dependencies packages are available for `$(package)_built` target.
Guix builds:
```
accef9ffccfe280fec1114c0440092ed5d792e9c53d95abc7fe65435aa136656 guix-build-978852aad8e2/output/aarch64-linux-gnu/SHA256SUMS.part
a75f1250975525a21d2e213e23f1f0dab516d2b28d0c7d747de292fe5c906013 guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu-debug.tar.gz
d20787af2e7a14a3b7b1d21e0d8784aa6ebad1e916f02aebfa25afe9229ba43c guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu.tar.gz
11c94a39c084763858c6de31b221868e52554f5500c0dc5589def429bb6b54c8 guix-build-978852aad8e2/output/arm-linux-gnueabihf/SHA256SUMS.part
3935b0e14d78800977dc813a3824215797096b6fb29c84b5996f48338908a65f guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf-debug.tar.gz
c828c3f87a453db443a385a266331661f623f8f4684d88eb006290c83bfa8150 guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf.tar.gz
b6ff14e1cc36e568fc726b50300f77498560322b3582738eb70e7144784f7e63 guix-build-978852aad8e2/output/arm64-apple-darwin/SHA256SUMS.part
2a3e6ba5843eaf9562e9dcfdb4595024a71738079cea00e391558feca4d5bde1 guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.dmg
ffd2ad39e8b9f95dd714513ba1e1c77666b0f3cc4b67be4ab763ebd431fe9276 guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.tar.gz
5c9f8acd1777effc1e860b64143ba9d06ba5e3d0330e7341529eeae5cc6b3c5e guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin.tar.gz
e34c693ecef6159c57fdedabff9dc3d69ec20387966083b828532c58e1e6e30b guix-build-978852aad8e2/output/dist-archive/bitcoin-978852aad8e2.tar.gz
8de4681114d96425bf9b0ccc47d49f696293ead514faa3fa83ddcccfdca2eeb2 guix-build-978852aad8e2/output/powerpc64-linux-gnu/SHA256SUMS.part
d780330a105931eb4c66a536c332eb09e4b6d8c288832bb5a74b931c4600c0b3 guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu-debug.tar.gz
af036a6d714ef362a2facfe4e5c63deaa9dfa391d20542ead9ffb4a43b2b7ca2 guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu.tar.gz
dc6b4365632e7de386ead512b1cdcdd3c985623f63cff8b62974bd9ed8c05d5d guix-build-978852aad8e2/output/powerpc64le-linux-gnu/SHA256SUMS.part
5543a6dec58515186b71139a4a5c603d85638672f205e7c9fabb281c98018461 guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu-debug.tar.gz
dd791be2e61ce6cbd3e14c165ce2f8c2d22881992e0df72bd338423d092cc467 guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu.tar.gz
2b740db8e5b9c435be3e7b186c3b4a40885302243326ec990e24fe4ba4f777da guix-build-978852aad8e2/output/riscv64-linux-gnu/SHA256SUMS.part
de899c89874d4f34afeca179a6c7c8f49b0a975983ab2b31afd9f2d365674578 guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu-debug.tar.gz
9052b1db22c56692d99a61c3783b36c6f76537d9aec14f17d87a155beac82532 guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu.tar.gz
5e79ddf57a94c5978ad819896786107f735d5742bbd042c2c64ae2d0681ce53a guix-build-978852aad8e2/output/x86_64-apple-darwin/SHA256SUMS.part
96443ad839f87c723db1c0a96d8ead0afc69e9d96ad45b5814344866da2dae73 guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.dmg
14b0a3081772e81a463398a2702aca039d2f276e301dee9f5a0ccffbb09e2749 guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.tar.gz
6bfb8252524202028308267f5e96bc30f284052f5feaa58ed3697dde27a3130f guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin.tar.gz
5f8ea6297e246b08ffd806913897cc863feeec6522fcfb4456a59c5f154e0c2d guix-build-978852aad8e2/output/x86_64-linux-gnu/SHA256SUMS.part
40d1bcf491660d54fe20b2db24828ebf61be848eefdc38fba09ed43f6bdba4b1 guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu-debug.tar.gz
3200e67a4dea115e8e341b4d71d84dc5e8bd2ae35e550cde6aef88d120c65eae guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu.tar.gz
0b0bf7effc493ecc68398f23fc81647f64fdee115e8ccd7ae91e7881804ec328 guix-build-978852aad8e2/output/x86_64-w64-mingw32/SHA256SUMS.part
e2064c9ddeb4af18468f37ba8cf70004062c31e1387b4cc0fe4b445fae518e8d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-debug.zip
be347a901b896e0a1dc2f0f5a7f84614075805cccf1f2af8ec8df678d086fdbc guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-setup-unsigned.exe
bab8700e9e266970e8c7cad494902058ad12d1f2a6462e0039daa637b1a0ce0d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-unsigned.tar.gz
c8e55e64b248fd7c9056fe811a1eba992bbb92e44857204e3024416d9ba6307d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64.zip
```
ACKs for top commit:
TheCharlatan:
tACK 978852aad8
Tree-SHA512: c195484274433039e327d44b1949afa296e09e7470a2b138b7a8476c8bf9c1302bc21284cd5436f09aa97824aae9f362b7932ff2937b78f79df0b43e50f3dfaa
44f3c7de21 contrib: remove install_db4.sh (fanquake)
14ce84388f doc: add new NO_* options from #26833 (fanquake)
Pull request description:
Now that we can build a bdb-only depends prefix (#26833), there is no need to
maintain a bdb-building bash script, that does the same thing as
depends, except worse, as it's missing patches and workarounds. i.e #26623.
Someone that wants to compile bdb themselves, but doesn't want to use other depends built libs, can do:
```bash
make -C depends NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_UPNP=1 NO_ZMQ=1 NO_USDT=1
...
to: /path/to/bitcoin/depends/x86_64-pc-linux-gnu
```
which gives them a BDB only prefix, and then compile using:
```bash
export BDB_PREFIX="/path/to/bitcoin/depends/x86_64-pc-linux-gnu"
./autogen.sh
./configure \
BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \
BDB_CFLAGS="-I${BDB_PREFIX}/include"
```
Wondering if we should extract the build bdb/legacy wallet docs somewhere, to avoid the repetition?
ACKs for top commit:
TheCharlatan:
ACK 44f3c7de21
achow101:
ACK 44f3c7de21
hebasto:
ACK 44f3c7de21
jarolrod:
ACK 44f3c7de21
Tree-SHA512: 50b33ae9df2ab94a1bd114e846cec16f647a61023b72f0d3e547a18db09c01d60bb7b42a04758212f4930314df03016feb6ebc96962dd8a8e26eb8cd4e0d167d
See https://www.sqlite.org/compile.html.
DSQLITE_DQS
> This setting disables the double-quoted string literal misfeature.
DSQLITE_DEFAULT_MEMSTATUS
> This setting causes the sqlite3_status() interfaces that track
> memory usage to be disabled.
> This helps the sqlite3_malloc() routines run much faster, and since
> SQLite uses sqlite3_malloc() internally, this helps to make the
> entire library faster.
DSQLITE_OMIT_DEPRECATED
> Omitting deprecated interfaces and features will not help SQLite
> to run any faster.
> It will reduce the library footprint, however. And it is the
> right thing to do.
DSQLITE_OMIT_SHARED_CACHE
> Omitting the possibility of using shared cache allows many
> conditionals in performance-critical sections of the code to be
> eliminated. This can give a noticeable improvement in performance.
Also: https://www.sqlite.org/sharedcache.html
> Shared-cache mode is an obsolete feature.
> The use of shared-cache mode is discouraged.
> Most use cases for shared-cache are better served by WAL mode.
> Applications that build their own copy of SQLite from source code
> are encouraged to use the -DSQLITE_OMIT_SHARED_CACHE compile-time
> option, as the resulting binary will be both smaller and faster.
DSQLITE_OMIT_JSON
Starting with sqlite 3.38.0 the JSON extension became opt-out rather
than opt-in, so we disable it here.
--disable-rtree
> An R-Tree is a special index that is designed for doing range queries.
> R-Trees are most commonly used in geospatial systems...
https://www.sqlite.org/rtree.html
--disable-fts4 --disable-fts5
> FTS5 is an SQLite virtual table module that provides full-text
> search functionality to database applications.
DSQLITE_LIKE_DOESNT_MATCH_BLOBS
> simplifies the implementation of the LIKE optimization and allows
> queries that use the LIKE optimization to run faster.
DSQLITE_OMIT_DECLTYPE
> By omitting the (seldom-needed) ability to return the declared type of
> columns from the result set of query, prepared statements can be made
> to consume less memory.
DSQLITE_OMIT_PROGRESS_CALLBACK
> By omitting this interface, a single conditional is removed from the
> inner loop of the bytecode engine, helping SQL statements to run slightly
> faster.
DSQLITE_OMIT_AUTOINIT
> with the SQLITE_OMIT_AUTOINIT option, the automatic initialization is omitted.
> This helps many API calls to run a little faster
> it also means that the application must call sqlite3_initialize()
manually.
Replacing `install` with `install-lib` and `install-bin` is not strictly
necessary just to update the library, but it takes advantage of recent
changes in the new version, and makes the build more minimal.
Otherwise our config.guess and config.sub will be copied over. This
problem has been masked by the fact that modern systems ship with
versions that recognise all the triplets we use (namely
arm64-apple-darwin). However building on ubuntu 20.04 surfaces the
issue.
Fixes#26420.
fanquake