faceeae49a ci: Disable valgrind functionl tests on forked repos to avoid timeouts (MarcoFalke)
Pull request description:
Allows people to fork our repo and run the tests again
Also print more cache stats
ACKs for top commit:
hebasto:
ACK faceeae49a, tested on my own repo: https://travis-ci.org/github/hebasto/bitcoin/jobs/676257500
Tree-SHA512: 50e44edf94fcb997438eeaf7308b2b58a0854141ecb1fbb0ba7bf5ed662f19b60899f966f579cca90ad5e789234d0e90122d8c2c854da70339058adc3a475fa6
fa4632c417 test: Move boost/stdlib includes last (MarcoFalke)
fa488f131f scripted-diff: Bump copyright headers (MarcoFalke)
fac5c37300 scripted-diff: Sort test includes (MarcoFalke)
Pull request description:
When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.
This pull preempts both issues by just sorting all includes in one commit.
Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.
Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.
ACKs for top commit:
practicalswift:
ACK fa4632c417
jonatack:
ACK fa4632c417, light review and sanity checks with gcc build and clang fuzz build
Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
b919efadff depends: Use default macos clang compiler (Russell Yanofsky)
d54f64c6c7 Add multiprocess travis configuration (Russell Yanofsky)
787f40668d Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
d630646662 libmultiprocess depends build (Russell Yanofsky)
e6e44eedd5 Multiprocess build changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
This splits autotools, depends build, and travis changes out of #10102, so code changes and build system changes can be reviewed separately.
ACKs for top commit:
hebasto:
re-ACK b919efadff, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-605514556) review.
Tree-SHA512: ebc5e403cc99a0d9629ed7fe1595e01d57e6d1255cbf03968a3196ff6f528f734c78060fdc065724ee1f923bcc5aa2b29470fcb36a7f15957eb57c76d58178a4
6136a96cdf ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e ci: Add native s390x (Hennadii Stepanov)
Pull request description:
Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.
This approach makes builds quick and stable (see: #18106).
The excerpt from the Travis log:
```
...
Running on host system without docker wrapper
...
Byte Order: Big Endian
...
```
ACKs for top commit:
MarcoFalke:
ACK 6136a96cdf
Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
Remove inconsistency between functional and unit test environments and make it
possible to substitute bitcoin-qt and bitcoin-node in place of bitcoind in
python tests, or to link bitcoind against shared libraries.
4444edc2e6 ci: Enable all functional tests in valgrind (MarcoFalke)
Pull request description:
The travis timeout for our repo has been bumped to 2h, so we can run all tests in valgrind now
ACKs for top commit:
practicalswift:
ACK 4444edc2e6 -- regarding the three disabled cases (`feature_abortnode`, `feature_block` and `rpc_bind`): not a big deal since MSan will take care of those once #18288 is merged. More is more :)
Tree-SHA512: ea2f798112911b6d1f3d88cfcdf0a7cdb698687248343703d6fe55da144542c961c15d888bffb41672c10aa76765615cb7c7ff93d468bfad3c51f962f24e7abb
fa569e1a9c ci: Set LC_ALL=C when running in qemu-s390x (MarcoFalke)
fa3d77623e ci: Use debian to avoid apt install 404 errors (MarcoFalke)
fabb946090 ci: Install needed gcc and qemu-user iff cross-compiling (MarcoFalke)
faba4672b6 ci: Fix QEMU_USER_CMD parse issues (MarcoFalke)
fa5d709fb2 ci: Move wrap-qemu into separate script (MarcoFalke)
Pull request description:
Now that the ci system no longer propagates files from the docker container back to the host, the wrap-qemu script is broken.
To fix it, every statement in the script needs to be executed in the docker (with `DOCKER_EXEC`). Instead of juggling with triple escape sequences like `\\\"`, just move the script to a separate file and call it with `DOCKER_EXEC`.
Also, fix a bunch of other bugs that prevent running the ci system in qemu
See the `ci/README.md` on how to test. TLDR: Can be tested with (replace `arm` with `s390x` to run the s390x build):
```
FILE_ENV="./ci/test/00_setup_env_arm.sh" MAKEJOBS="-j9" ./ci/test_run_all.sh
```
ACKs for top commit:
laanwj:
Code review ACK fa569e1a9c
Tree-SHA512: 84ebc44a4f0261ee6c29605a6896a1833ff6c81d729e6d08dd111941f570ce73221422bd3303e1108a266ec5eab2148bd5ee1cf6bc01477d8cc9a6c5bf2b34c2
ef63f5fc11 ci: Combine 32-bit build with CentOS 7 build (Sebastian Falbesoner)
Pull request description:
Combines the CentOS build with the 32-bit (i686) build to avoid Travis bottlenecks, as suggested in #17757 by MarcoFalke. This keeps most of the properties of the 32-bit build (dash as config shell, building QT5 GUI) and just builds it with depends inside the CentOS docker container.
Making the depends in `05_before_script.sh` with unset config shell (`CONFIG_SHELL=`)
6196e93001/ci/test/05_before_script.sh (L28)
caused problems for building the library libevent (resulting in a Makefile with no shell set (`SHELL=`)), that's why I set it explicitely to `/bin/bash` if we have a CentOS Docker container.
A Travis output of this 32-bit CentOS build can be seen here: https://travis-ci.org/theStack/bitcoin/jobs/634472394 (has been restarted once due to too long build time and appearance of the `CACHE_ERR_MSG`).
For anyone wanting to verify the outputs, I found these instructions useful to reproduce a Travis build locally: https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine (steps 1-3). In this case it's a bit tricky since you run Docker inside Docker -- within the Travis Docker container, the CentOS Docker container is created. To make this possible, the Docker socket has to be exposed to the Travis container via bind-mounting (`docker run -v /var/run/docker.sock:/var/run/docker.sock ...`), as suggested in https://stackoverflow.com/a/33003273.
Top commit has no ACKs.
Tree-SHA512: af508241cec3a10a66c37673d56691717b78375340e910fcdd3fb3870741eba623a436e1e85b26b54f013375611896f5411c5a7fec2437d367d27172230129fe
* Fix bash syntax by adding missing -
* Disable QEMU_USER_CMD fallback when it is set
Apparently bash can't distinguish an unset variable from a variable
that is set to the empty string
* Export the environment variable to the docker env, otherwise it
couldn't be used there
711e0449cf ci: Remove trusty build (Hennadii Stepanov)
7f3ae22468 ci: Add CentOS 7 build (Hennadii Stepanov)
Pull request description:
Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.
Currently, CentOS 7 has:
- Berkeley DB == 4.8.30
- Boost == 1.53.0
- GCC == 4.8.5
- libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
- MiniUPnPc == 2.0
- Python == 3.6.8
- qrencode == 3.4.1
- Qt == 5.9.7
- ZeroMQ == 4.1.4
~Please note that this PR is based on the bugfix #17634.~
Also trusty build has been removed for the following reasons:
- https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
> Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.
- https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
> Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.
Close#17628
ACKs for top commit:
MarcoFalke:
ACK 711e0449cf🚠
Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)
Pull request description:
Some minor cleanups:
* Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
* Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
* Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
* Fixes#17178 by removing `cd`
Top commit has no ACKs.
Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
fa7523d3aa ci: Extend docs (MarcoFalke)
fa493ef088 ci: Make ci system read-only on the git work tree (MarcoFalke)
fab1333292 ci: Remove git from required packages on host (MarcoFalke)
fa00393bce ci: Make all filesystem operations inside docker (MarcoFalke)
Pull request description:
Running the ci completely in a docker, without leaving any traces on the host system is not possible right now because the ccache and depends dir needs to be propagated back and picked up by the host for caching.
Fixes#17372
ACKs for top commit:
JeremyRubin:
tested ACK fa7523d3aa
Tree-SHA512: 4bce1a0f883bcbdb34abf409bdbc80d420c5da2045d2f9c5536ac433f9e5b490f23df084546c8c049f688b487572bbfc4f9c4029e9e672f4d9279739d066ed2e
30fb598737 Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f4 Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad71548822 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)
Pull request description:
Changes in #12048 cause a compilation error in Arena::walk() when
ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
changed to have a different value type.
Additionally, missing includes cause other compilation errors when
ARENA_DEBUG is defined.
Reproduced with:
make CPPFLAGS=-DARENA_DEBUG
ACKs for top commit:
laanwj:
ACK 30fb598737
fanquake:
ACK 30fb598737 - thanks for following up jkczyz.
Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
The definition and uses of Arena::walk() are compiled only if
ARENA_DEBUG is defined. Configure Travis to define ARENA_DEBUG so
compilation errors do not go unnoticed.
facc0da63a travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a ci: Remove ccache requirement on the host (MarcoFalke)
Pull request description:
This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.
ACKs for top commit:
laanwj:
LGTM ACK facc0da63a
practicalswift:
ACK facc0da63a -- diff looks correct and Travis seems happy
Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
ddddd8961b ci: Use busybox utils for one build (MarcoFalke)
Pull request description:
To make sure Bitcoin Core can be built with BusyBox, see https://github.com/bitcoin/bitcoin/pull/16927#issuecomment-536483706
ACKs for top commit:
laanwj:
ACK ddddd8961b
Tree-SHA512: da3a4654ee7975206d04643675d309b4973a510ca344acaec97fb1ed19c43cf13489bdf236c92c4a90499ec5b3c18c3338fff096110b26abee5ffe955089f267
fa21737ba7 ci: Add environment files for all settings (MarcoFalke)
Pull request description:
This moves all environment settings from travis to files in the ci folder. Now, it is possible to easily run each travis configuration with a single command.
Top commit has no ACKs.
Tree-SHA512: 989c6b62eb3839eb1fa5461e986496e9660167e2438a789c7588a6fee4f9c37b332782c010fe5c7de8f606bcf98dffb2481d2777cbce88f87cc9f0c42fb2d7fc
Hennadii Stepanov