870f0cd2a0 build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift)
Pull request description:
Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory.
First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :)
Some historical context:
* 2017: Continuous compilation with Clang Thread Safety analysis enabled (#10866, #10923)
* 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (#12686)
* 2018: Continuous testing of use of locale dependent functions (#13041)
* 2018: Continuous testing of format strings (#13705)
* 2018: Continuous compilation with MSVC `TreatWarningAsError` (#14151)
* 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (#14252, #14673, #17006)
* 2018: Continuous testing under AddressSanitizer – ASan (#14794, #17205, #17674)
* 2018: Continuous testing under ThreadSanitizer – TSan (#14829)
* 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (#15134)
* 2019: Continuous compile-time testing of assumptions we're making (#15391)
* 2019: Continuous testing of fuzz test cases under Valgrind (#17633, #18159, #18166)
* 2020: Finally... MemorySanitizer – MSAN! :)
What is the next step? What tools should we add to CI to keep bugs from entering `master`? :)
ACKs for top commit:
MarcoFalke:
ACK 870f0cd2a0
Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578
e2bab2aa16 multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e7 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52b build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa16
Sjors:
tACK e2bab2aa16 on macOS 10.15.4
hebasto:
ACK e2bab2aa16, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
c31cbe7cfe Add C++17 test to Travis (Pieter Wuille)
7829685e27 Add configure option for c++17 (Pieter Wuille)
0fbde488b2 Support conversion between Spans of compatible types (Pieter Wuille)
7cbfebbf3d Update ax_cxx_compile_stdcxx.m4 (Pieter Wuille)
Pull request description:
This adds a `--enable-c++17` option to the configure script, fixes the only C++17 incompatibility (with a commit taken from #18468), and adds a Travis test for it.
This is all off by default, and release builds remain C++11.
It implements the first step of the plan in https://github.com/bitcoin/bitcoin/issues/16684.
ACKs for top commit:
elichai:
tACK c31cbe7cfe
practicalswift:
Tested ACK c31cbe7cfe
hebasto:
ACK c31cbe7cfe, tested on Linux Mint 19.3 both C++11 and C++17 modes. Compiled and passed tests locally.
Tree-SHA512: a4b00776dbceef9c12abbb404c6bcd48f7916ce24c8c7a14116355f64e817578b7fcddbedd5ce435322319d1e4de43429b68553f4d96d970c308fe3e3e59b9d1
b919efadff depends: Use default macos clang compiler (Russell Yanofsky)
d54f64c6c7 Add multiprocess travis configuration (Russell Yanofsky)
787f40668d Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
d630646662 libmultiprocess depends build (Russell Yanofsky)
e6e44eedd5 Multiprocess build changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
This splits autotools, depends build, and travis changes out of #10102, so code changes and build system changes can be reviewed separately.
ACKs for top commit:
hebasto:
re-ACK b919efadff, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-605514556) review.
Tree-SHA512: ebc5e403cc99a0d9629ed7fe1595e01d57e6d1255cbf03968a3196ff6f528f734c78060fdc065724ee1f923bcc5aa2b29470fcb36a7f15957eb57c76d58178a4
Travis is consistently failing on s390 due to out of disk space issues,
which causes false positives. Disable the s390 build for now until
this is fixed some other way.
fa569e1a9c ci: Set LC_ALL=C when running in qemu-s390x (MarcoFalke)
fa3d77623e ci: Use debian to avoid apt install 404 errors (MarcoFalke)
fabb946090 ci: Install needed gcc and qemu-user iff cross-compiling (MarcoFalke)
faba4672b6 ci: Fix QEMU_USER_CMD parse issues (MarcoFalke)
fa5d709fb2 ci: Move wrap-qemu into separate script (MarcoFalke)
Pull request description:
Now that the ci system no longer propagates files from the docker container back to the host, the wrap-qemu script is broken.
To fix it, every statement in the script needs to be executed in the docker (with `DOCKER_EXEC`). Instead of juggling with triple escape sequences like `\\\"`, just move the script to a separate file and call it with `DOCKER_EXEC`.
Also, fix a bunch of other bugs that prevent running the ci system in qemu
See the `ci/README.md` on how to test. TLDR: Can be tested with (replace `arm` with `s390x` to run the s390x build):
```
FILE_ENV="./ci/test/00_setup_env_arm.sh" MAKEJOBS="-j9" ./ci/test_run_all.sh
```
ACKs for top commit:
laanwj:
Code review ACK fa569e1a9c
Tree-SHA512: 84ebc44a4f0261ee6c29605a6896a1833ff6c81d729e6d08dd111941f570ce73221422bd3303e1108a266ec5eab2148bd5ee1cf6bc01477d8cc9a6c5bf2b34c2
711e0449cf ci: Remove trusty build (Hennadii Stepanov)
7f3ae22468 ci: Add CentOS 7 build (Hennadii Stepanov)
Pull request description:
Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.
Currently, CentOS 7 has:
- Berkeley DB == 4.8.30
- Boost == 1.53.0
- GCC == 4.8.5
- libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
- MiniUPnPc == 2.0
- Python == 3.6.8
- qrencode == 3.4.1
- Qt == 5.9.7
- ZeroMQ == 4.1.4
~Please note that this PR is based on the bugfix #17634.~
Also trusty build has been removed for the following reasons:
- https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
> Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.
- https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
> Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.
Close#17628
ACKs for top commit:
MarcoFalke:
ACK 711e0449cf🚠
Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)
Pull request description:
Some minor cleanups:
* Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
* Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
* Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
* Fixes#17178 by removing `cd`
Top commit has no ACKs.
Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
MarcoFalke