bitcoin

History

fanquake f61019f5a2 Merge #19407 : doc: explain why passing -mlinker-version is required when cross-compiling `a8d39b8840` doc: explain why passing -mlinker-version is required (fanquake) Pull request description: I have been down a 🐇 hole. Closes #19359. When Clang is compiled, [a check is run](https://github.com/llvm/llvm-project/blob/release/8.x/clang/CMakeLists.txt#L353) to define `HOST_LINK_VERSION` as the output of `$CMAKE_LINKER -v`. Note the this is the version of the linker being used to compile Clang itself.. and this check is only run when compiling Clang for macOS. In the Clang driver, if `HOST_LINK_VERSION` has been defined, there is some additional runtime functionality. An `-mlinker-version` argument, with the value of `HOST_LINK_VERSION` [will be added to the linker arguments](`89de0d8dfb/clang/lib/Driver/Driver.cpp (L382)`), if `-mlinker-version` has not been passed in by the user. This is a bit weird, as by default, you are setting `-mlinker-version` to the version of the linker that was used to build the Clang binary, not the linker which will be used when compiling. The commit which introduced the functionality, `628fcf4e3b`, described it as a "hack", that should be replaced. However, that was 10 years ago, and the behaviour is still here. In the Darwin driver, [a check is done](`89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L208)`) for the `-mlinker-version` argument. If there is no argument, the version will default to `0`. Given the above, this should never happen when using Clang for macOS. A series of comparisons are then performed, to check whether the linker version is modern enough to enable certain features, like [`-demangle`](`89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L215)`). ### What this means #### macOS A Clang compiled for macOS, i.e `clang+llvm-8.0.0-x86_64-apple-darwin`, will have `HOST_LINKER_VERSION` set to the version of the linker used to compile Clang itself. At runtime, `-mlinker-version=HOST_LINKER_VERSION` will be added to the linker args, if `-mlinker-version` wasn't passed in. In the Darwin driver, additional arguments, like `-demangle`, will be added to the linker arguments, because `HOST_LINKER_VERSION` was likely some very modern version of `lld` or `ld64`. #### Linux (cross compilation in depends) A Clang compiled for Linux, i.e `clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-14.04`, which we now use for macOS builds in depends, will behave differently. As it's built for Linux, `HOST_LINKER_VERSION` was not defined at compile time, and there will be no default behaviour of appending `-mlinker-version=HOST_LINKER_VERSION` to the linker args. Thus, unless you pass in `-mlinker-version` yourself, when the version checks are done in the Darwin driver, no modern linker features will be enabled, as the version will have defaulted to `0`. Therefore, it's important that we continue to pass `-mlinker-version="our LD64 version"` as part of our compilation flags, if we want to have "modern" linker features enabled for our macOS builds. #### Summary [Clang 8](https://releases.llvm.org/download.html#8.0.0). Building a macOS binary. Link line with path arguments trimmed. \| \| default behaviour \| `-mlinker-version=100` (`-demangle threshold`) \| `-mlinker-version=530` \| \| - \| --------------- \| --------------------- \| ---------------------- \| \| macOS Clang \| `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-b8b9b3.o -lc++ -lSystem ../libclang_rt.osx.a` \| `-demangle -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-a66966.o -lc++ -lSystem ../libclang_rt.osx.a` \| same as default \| \| Linux Clang \| `-dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-bfce57.o -lc++ -lSystem` \| `-demangle -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-a846a3.o -lc++ -lSystem` \| `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-de0280.o -lc++ -lSystem` \| Note: Most links here are pointing to the 8.x branch of LLVM/Clang, as we are using that version in depends. Note: To add a little more confusion, you wont see `-mlinker-version X` in your compile flags, you'll see [`-target-linker-version X`](`431daedee4/clang/lib/Driver/ToolChains/Clang.cpp (L4777)`). ACKs for top commit: laanwj: ACK `a8d39b8840` Tree-SHA512: 92f93079a5e59a0d561e74336b5cb03e3bf5a34437f5850283b9128c7624494b8285ec16290b1fa8103fe87f8789a53ce44b17902b8c1db5fde24d74b76fb168		4 years ago
..
builders	depends: Add --sysroot option to mac os native compile flags	5 years ago
hosts	doc: explain why passing -mlinker-version is required	4 years ago
packages	build: pass _WIN32_WINNT=0x0601 when building libevent for Windows	4 years ago
patches	depends: Patch libevent build to fix IPv6 -rpcbind on Windows	4 years ago
.gitignore	depends: Support for S390X targets	5 years ago
Makefile	depends: Propagate only specific CLI variables to sub-makes	5 years ago
README.md	depends: add MULTIPROCESS depends option	5 years ago
config.guess	depends: latest config.guess & config.sub	5 years ago
config.site.in	Merge #18297 : build: Use pkg-config in BITCOIN_QT_CONFIGURE for all hosts including Windows	4 years ago
config.sub	depends: latest config.guess & config.sub	5 years ago
description.md	doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4	5 years ago
funcs.mk	depends: add MULTIPROCESS depends option	5 years ago
packages.md	doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4	5 years ago

README.md

Usage

To build dependencies for the current arch+OS:

make

To build for another arch/OS:

make HOST=host-platform-triplet

For example:

make HOST=x86_64-w64-mingw32 -j4

Bitcoin Core's configure script by default will ignore the depends output. In order for it to pick up libraries, tools, and settings from the depends build, you must point it at the appropriate --prefix directory generated by the build. In the above example, a prefix dir named x86_64-w64-mingw32 will be created. To use it for Bitcoin:

./configure --prefix=$PWD/depends/x86_64-w64-mingw32

Common host-platform-triplets for cross compilation are:

i686-pc-linux-gnu for Linux 32 bit
x86_64-pc-linux-gnu for x86 Linux
x86_64-w64-mingw32 for Win64
x86_64-apple-darwin16 for macOS
arm-linux-gnueabihf for Linux ARM 32 bit
aarch64-linux-gnu for Linux ARM 64 bit
powerpc64-linux-gnu for Linux POWER 64-bit (big endian)
powerpc64le-linux-gnu for Linux POWER 64-bit (little endian)
riscv32-linux-gnu for Linux RISC-V 32 bit
riscv64-linux-gnu for Linux RISC-V 64 bit
s390x-linux-gnu for Linux S390X
armv7a-linux-android for Android ARM 32 bit
aarch64-linux-android for Android ARM 64 bit
i686-linux-android for Android x86 32 bit
x86_64-linux-android for Android x86 64 bit

The paths are automatically configured and no other options are needed unless targeting Android.

Install the required dependencies: Ubuntu & Debian

For macOS cross compilation

sudo apt-get install curl librsvg2-bin libtiff-tools bsdmainutils cmake imagemagick libcap-dev libz-dev libbz2-dev python3-setuptools

For Win64 cross compilation

see build-windows.md

For linux (including i386, ARM) cross compilation

Common linux dependencies:

sudo apt-get install make automake cmake curl g++-multilib libtool binutils-gold bsdmainutils pkg-config python3 patch

For linux ARM cross compilation:

sudo apt-get install g++-arm-linux-gnueabihf binutils-arm-linux-gnueabihf

For linux AARCH64 cross compilation:

sudo apt-get install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu

For linux POWER 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-powerpc64-linux-gnu binutils-powerpc64-linux-gnu g++-powerpc64le-linux-gnu binutils-powerpc64le-linux-gnu

For linux RISC-V 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-riscv64-linux-gnu binutils-riscv64-linux-gnu

RISC-V known issue: gcc-7.3.0 and gcc-7.3.1 result in a broken test_bitcoin executable (see https://github.com/bitcoin/bitcoin/pull/13543), this is apparently fixed in gcc-8.1.0.

For linux S390X cross compilation:

sudo apt-get install g++-s390x-linux-gnu binutils-s390x-linux-gnu

Dependency Options

The following can be set when running make: make FOO=bar

SOURCES_PATH: downloaded sources will be placed here
BASE_CACHE: built packages will be placed here
SDK_PATH: Path where sdk's can be found (used by macOS)
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
NO_QT: Don't download/build/cache qt and its dependencies
NO_QR: Don't download/build/cache packages needed for enabling qrencode
NO_ZMQ: Don't download/build/cache packages needed for enabling zeromq
NO_WALLET: Don't download/build/cache libs needed to enable the wallet
NO_UPNP: Don't download/build/cache packages needed for enabling upnp
MULTIPROCESS: build libmultiprocess (experimental, requires cmake)
DEBUG: disable some optimizations and enable more runtime checking
HOST_ID_SALT: Optional salt to use when generating host package ids
BUILD_ID_SALT: Optional salt to use when generating build package ids

If some packages are not built, for example make NO_WALLET=1, the appropriate options will be passed to bitcoin's configure. In this case, --disable-wallet.

Additional targets

download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for macOS builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

Android

Before proceeding with an Android build one needs to get the Android SDK and use the "SDK Manager" tool to download the NDK and one or more "Platform packages" (these are Android versions and have a corresponding API level). In order to build ANDROID_API_LEVEL (API level corresponding to the Android version targeted, e.g. Android 9.0 Pie is 28 and its "Platform package" needs to be available) and ANDROID_TOOLCHAIN_BIN (path to toolchain binaries depending on the platform the build is being performed on) need to be set.

API levels from 24 to 29 have been tested to work.

If the build includes Qt, environment variables ANDROID_SDK and ANDROID_NDK need to be set as well but can otherwise be omitted. This is an example command for a default build with no disabled dependencies:

ANDROID_SDK=/home/user/Android/Sdk ANDROID_NDK=/home/user/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/user/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin