4fe7cf1677 build: Drop unneeded dependencies for bitcoin-wallet tool (Hennadii Stepanov)
Pull request description:
`bitcoin-wallet` is an offline tool, and its code does not depend on networking stuff (ZMQ, UPnP, NAT-PMP, and LIBBITCOIN_SERVER).
Also `bitcoin-wallet` does not interacts with the chainstate, therefore dependency on LevelDB is not needed.
ACKs for top commit:
laanwj:
Code review ACK 4fe7cf1677
Tree-SHA512: 97342d9cdf8670806efe16dc7885a85ec92f3c1ae0819a4c3cc147938fc8642089e303c4432cb1395fc75b852c1af6a6a13fc58e29e027c23f75219fd3bd8cb4
29173d6c6c ubsan: add minisketch exceptions (Cory Fields)
54b5e1aeab Add thin Minisketch wrapper to pick best implementation (Pieter Wuille)
ee9dc71c1b Add basic minisketch tests (Pieter Wuille)
0659f12b13 Add minisketch dependency (Gleb Naumenko)
0eb7928ab8 Add MSVC build configuration for libminisketch (Pieter Wuille)
8bc166d5b1 build: add minisketch build file and include it (Cory Fields)
b2904ceb85 build: add configure checks for minisketch (Cory Fields)
b6487dc4ef Squashed 'src/minisketch/' content from commit 89629eb2c7 (fanquake)
Pull request description:
This takes over #21859, which has [recently switched](https://github.com/bitcoin/bitcoin/pull/21859#issuecomment-921899200) to my integration branch. A few more build issues came up (and have been fixed) since, and after discussing with sipa it was decided I would open a PR to shepherd any final changes through.
> This adds a `src/minisketch` subtree, taken from the master branch of https://github.com/sipa/minisketch, to prepare for Erlay implementation (see #21515). It gets configured for just supporting 32-bit fields (the only ones we're interested in in the context of Erlay), and some code on top is added:
> * A very basic unit test (just to make sure compilation & running works; actual correctness checking is done through minisketch's own tests).
> * A wrapper in `minisketchwrapper.{cpp,h}` that runs a benchmark to determine which field implementation to use.
Only changes since my last update to the branch in the previous PR have been rebasing on master and fixing an issue with a header in an introduced file.
ACKs for top commit:
naumenkogs:
ACK 29173d6c6c
Tree-SHA512: 1217d3228db1dd0de12c2919314e1c3626c18a416cf6291fec99d37e34fb6eec8e28d9e9fb935f8590273b8836cbadac313a15f05b4fd9f9d3024c8ce2c80d02
The testconsensus project is not integral to the Bitcoin Core code. It was originally added as a quick and dirty demo of how to do a consensus check with the msvc build. There are better examples.
PR #23438 made a change that caused a compiler error with the buildmsvc/testconsensus code. Rather than leave it hanging around to incur potential bitrot, or furhter PR build failures, it should be removed.
d5f985e51f multiprocess: Add new bitcoin-gui, bitcoin-qt, bitcoin-wallet init implementations (Russell Yanofsky)
Pull request description:
Add separate `interfaces::Init` subclasses for `bitcoin-wallet`, `bitcoin-gui`, and `bitcoin-qt` binaries instead of sharing `bitcoind` and `bitcoin-node` init subclasses in different binaries. After this, the new init subclasses can be customized in #10102, so node and wallet code is dropped from the `bitcoin-gui` binary and wallet code is dropped from into the `bitcoin-node` binary.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
lsilva01:
reACK d5f985e
hebasto:
re-ACK d5f985e51f, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/23006#pullrequestreview-787537444) review.
Tree-SHA512: 6784210bd9ce3a6fbc66852680d0e9bc513c072dc538aeb7f48cb6a41580d3f567ccef04f975ee767a714a4b05d4d87273e94a79abda1b9c25d5ac4bbe752006
6bc4398937 doc: Suggest using jom instead of nmake (Hennadii Stepanov)
c4139f06a9 doc: Replace a link to Qt precompiled binaries with compile instructions (Hennadii Stepanov)
5e42f2ad26 build: Make <QtBaseDir> default name Qt and VS versions agnostic (Hennadii Stepanov)
Pull request description:
This PR replaces a [link to Qt precompiled binaries](https://github.com/sipsorcery/qt_win_binary/releases) with compile instructions that allow users to self-compile static Qt package which is required for building Bitcoin Core with Visual Studio.
ACKs for top commit:
sipsorcery:
ACK 6bc4398937.
Tree-SHA512: 0c91536e51177ec2ed437614cb37b3fd1dccce856548c22307359da68200433971fd97cd8a537b0856cfccc521ac50d61801a78cb5275b818829ee7b43bc7d6a
Add separate init implementations instead of sharing existing bitcoind
and bitcoin-node ones, so they can start to be differentiated in
upcoming commits with node and wallet code no longer linked into the
bitcoin-gui binary and wallet code no longer linked into the
bitcoin-node binary.
e4709c7b56 Start using init makeNode, makeChain, etc methods (Russell Yanofsky)
Pull request description:
Use `interfaces::Init::make*` methods instead of `interfaces::Make*` functions, so interfaces can be constructed differently in different executable without having to change any code. (So for example `bitcoin-gui` can make an `interfaces::Node` pointer that communicates with a `bitcoin-node` subprocess, while `bitcoin-qt` can make an `interfaces::Node` pointer that controls node code in the same process.)
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). The commit was first part of larger PR #10102.
ACKs for top commit:
jamesob:
reACK e4709c7b56
achow101:
ACK e4709c7b56
benthecarman:
utACK e4709c7b56
Tree-SHA512: 580c1979dbb2ef444157c8e53041e70d15ddeee77e5cbdb34f70b6d228cc2d2fe3843825f172da84e506200c58f7e0932f7cd4c006bb5058c1f4e43259394834
57b3c5bda6 build_msvc: Drop 32-bit build configurations (Hennadii Stepanov)
b3decea7f6 build_msvc: Make bitcoin-util ProjectGuid unique (Hennadii Stepanov)
Pull request description:
A 32-bit application for Windows in 2021 looks outdated. I'm pretty sure no one is going to run Bitcoin Core v23.0 on 32-bit Windows.
Also see #15939.
ACKs for top commit:
sipsorcery:
tACK 57b3c5bda6.
Tree-SHA512: d03dccb7bab9f6694d008c4b1fdade1dbd7e5980b5199cc6648c1a8c0c66f07170ae9cb6a77b4ab54c9195003587051b94217b014f97c215dffeec2bcd8fcd6e
This change allows users to not patch `common.qt.init.vcxproj` to fit
their Qt and Visual Studio versions. Also it simplifies code base
maintaining.
To specify a non-default path, the QTBASEDIR environment variable can be
used.
While not many of these are used, some, like HAVE_DLLEXPORT_ATTRIBUTE, are used
when creating libbitcoinconsensus.
Given this file is static, also just remove everything that is commented out.
Use interfaces::Init::make* methods instead of interfaces::Make*
functions, so interfaces can be constructed differently in different
executables without having to change any code. (So for example
bitcoin-gui can make an interfaces::Node pointer that communicates with
a bitcoin-node subprocess, while bitcoin-qt can make an interfaces::Node
pointer that starts node code in the same process.)
fe6dc76b7c wallet test: Add test for subtract fee from recipient behavior (Russell Yanofsky)
2565478c81 wallet test refactor: add CreateSyncedWallet function (Russell Yanofsky)
Pull request description:
This adds test coverage for wallet subtract from recipient behavior without changing it. Behavior seems to have changed recently in a minor way in #17331 without being noticed.
ACKs for top commit:
achow101:
ACK fe6dc76b7c
glozow:
ACK fe6dc76b7c
promag:
Code review ACK fe6dc76b7c.
Tree-SHA512: e00c5dfe467e4ccef5edb0dd4fff6c53f35a37828a4327bea2e166751e5ef971d519ffca7b8f735b12912bb4a547980626356bc1855981005aed1a6c2a57be0b
8169fc4e73 qt, refactor: Fix code styling of moved InitExecutor class (Hennadii Stepanov)
c82165a557 qt, refactor: Move InitExecutor class into its own module (Hennadii Stepanov)
dbcf56b6c6 scripted-diff: Rename BitcoinCore class to InitExecutor (Hennadii Stepanov)
19a1d00831 qt: Add BitcoinCore::m_thread member (Hennadii Stepanov)
Pull request description:
This PR makes the `BitcoinCore` class reusable, i.e., it can be used by the widget-based GUI or by the [QML-based](https://github.com/bitcoin-core/gui-qml/tree/main/src/qml) one, and it makes the divergence between these two repos minimal.
The small benefit to the current branch is more structured code.
Actually, this PR is ported from https://github.com/bitcoin-core/gui-qml/pull/10.
The example of the re-using of the `BitcoinCore` class is https://github.com/bitcoin-core/gui-qml/pull/11.
ACKs for top commit:
laanwj:
ACK 8169fc4e73
ryanofsky:
Code review ACK 8169fc4e73. Only change is switching from `m_executor` from pointer to optional type (thanks for update!)
Tree-SHA512: a0552c32d26d9acf42921eb12bcdf68f02d52f7183c688c43257b1a58679f64e45f193ee2d316850c7f0f516561e17abe989fe545bfa05e158ad3f4c66d19bca
9edd713c18 build: Fix MSVC linker /SubSystem option for bitcoin-qt.exe (Hennadii Stepanov)
Pull request description:
On master (6f3fbc062f), running `bitcoin-qt.exe`, which was built with MSVC, causes a terminal window open along with the GUI.
This PR fixes such behavior. See Microsoft [docs](https://docs.microsoft.com/en-us/cpp/build/reference/subsystem-specify-subsystem?view=msvc-160).
It is still possible to use the `-printtoconsole` option for debug builds.
ACKs for top commit:
sipsorcery:
tACK 9edd713c18.
Tree-SHA512: 02f2874b13e484f98344f6a7e3b01fa82a78a39865787c77bd674ead22a84a7f98a1849ccad26bd2b8c8603b3e29dcc1633b0ad731ce7d61be2d6b1f9584839c
No change in behavior. This just moves some code from the ListCoins test
setup to a reusable util function, so it can be reused in a new test in
the next commit.
5c7ee1b2da libsecp256k1 no longer has --with-bignum= configure option (Pieter Wuille)
bdca9bcb6c Squashed 'src/secp256k1/' changes from 3967d96bf1..efad3506a8 (Pieter Wuille)
cabb566123 Disable certain false positive warnings for libsecp256k1 msvc build (Pieter Wuille)
Pull request description:
This updates our src/secp256k1 subtree to the latest upstream master. The changes include:
* The introduction of safegcd-based modular inverses, reducing ECDSA signing time by 25%-30% and ECDSA verification time by 15%-17%.
* [Original paper](https://gcd.cr.yp.to/papers.html) by Daniel J. Bernstein and Bo-Yin Yang
* [Implementation](https://github.com/bitcoin-core/secp256k1/pull/767) by Peter Dettman; [final](https://github.com/bitcoin-core/secp256k1/pull/831) version
* [Explanation](https://github.com/bitcoin-core/secp256k1/blob/master/doc/safegcd_implementation.md) of the algorithm using Python snippets
* [Analysis](https://github.com/sipa/safegcd-bounds) of the maximum number of iterations the algorithm needs
* [Formal proof in Coq](https://medium.com/blockstream/a-formal-proof-of-safegcd-bounds-695e1735a348) by Russell O'Connor, for a high-level equivalent algorithm
* Removal of libgmp as an (optional) dependency (which wasn't used in the Bitcoin Core build)
* CI changes (Travis -> Cirrus)
* Build system improvements
ACKs for top commit:
laanwj:
Tested ACK 5c7ee1b2da
Tree-SHA512: ad8ac3746264d279556a4aa7efdde3733e114fdba8856dd53218588521f04d83950366f5c1ea8fd56329b4c7fe08eedf8e206f8f26dbe3f0f81852e138655431
fac30eec42 refactor: Replace &foo[0] with foo.data() (MarcoFalke)
faece47c47 refactor: Avoid &foo[0] on C-Style arrays (MarcoFalke)
face961109 refactor: Use only one temporary buffer in CreateObfuscateKey (MarcoFalke)
fa05dddc42 refactor: Use CPubKey vector constructor where possible (MarcoFalke)
fabb6dfe6e script: Replace address-of idiom with vector data() method (Guido Vranken)
Pull request description:
The main theme of this refactor is to replace `&foo[0]` with `foo.data()`.
The first commit is taken from #21781 with the rationale:
* In CSignatureCache::ComputeEntryECDSA, change the way a vector pointer is resolved to prevent invoking undefined behavior if the vector is empty.
The other commits aim to remove all `&foo[0]`, where `foo` is any kind of byte representation. The rationale:
* Sometimes alternative code without any raw data pointers is easier to read (refer to the respective commit message for details)
* If the raw data pointer is needed, `foo.data()` should be preferred, as pointed out in the developer notes. This addresses the instances that have been missed in commit 592404f03f, and https://github.com/bitcoin/bitcoin/pull/9804
ACKs for top commit:
laanwj:
Code review ACK fac30eec42
practicalswift:
cr ACK fac30eec42c486ec1bfd696293040a7aa0f04625: patch looks correct
promag:
Code review ACK fac30eec42.
Tree-SHA512: e7e73146edbc78911a8e8c728b0a1c6b0ed9a88a008e650aa5dbffe72425bd42c76df70199a9cf7e02637448d7593e0eac52fd0f91f59240283e1390ee21bfa5
46b025e00d test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7 test: fix file permissions on various scripts (windsok)
Pull request description:
Adds a new python linter test which tests for correct filenames and file permissions in the repository.
Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050
Summary of tests:
* Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.
* Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)
* Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.
* Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)
* Checks every file that contains a shebang line to ensure it has an executable permission
Additionally updates the permissions on various files to comply with the new tests.
Fixes#21729
ACKs for top commit:
practicalswift:
cr re-ACK 46b025e00df40724175735eb5606ac73067cb3b8: patch still looks correct
kiminuo:
code review ACK 46b025e00d if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
laanwj:
Code review ACK 46b025e00d
Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
This PR was motivated by a comment in GUI PR (257) regarding a suggested improvement not being supported by VS2017.
When checking whether master can still be built with the VS2017 toolset ABI issues were encountered. Most likely due to the pre-compiled Qt binaries that are used.
It does not seem worth the effort to try and support VS2017, which would most likely require additional Qt binaries, or lengthy instructions on how to build static Qt binaries on Windows (which is very error prone and tedious).
Added advisory note about build not working with earlier Visual Studio versions.
Fixed grammar.
9bd3f35003 build: adds switch for disabling random base addresses in MSVC (Ethan Heilman)
Pull request description:
In m4 builds we have the --disable-hardening switch that can be given in `./configure` to turn off randomized addresses. This PR provides a simple way of turning off randomized addresses in MSVC builds.
This PR:
* Adds this option the common-init project file so that it can be globally set across the project
* Documents this switch in msvc build readme
I have run the following test to verify this works
I ran the msvc build with `<RandomizedBaseAddress>true</RandomizedBaseAddress>` then checked `bitcoind.exe` and `bitcoin-cli.exe` with `dumpbin.exe`:
bitcoind
```
> .\dumpbin.exe /headers src/bitcoind.exe
Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
...
OPTIONAL HEADER VALUES
20B magic # (PE32+)
14.28 linker version
AE4600 size of code
345C00 size of initialized data
0 size of uninitialized data
6BED74 entry point (00000001406BED74) mainCRTStartup
1000 base of code
140000000 image base (0000000140000000 to 0000000140E2DFFF)
1000 section alignment
200 file alignment
6.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
E2E000 size of image
400 size of headers
0 checksum
3 subsystem (Windows CUI)
8160 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
Terminal Server Aware
```
bitcoin-cli
```
> .\dumpbin.exe /headers src/bitcoin-cli.exe
Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
...
OPTIONAL HEADER VALUES
20B magic # (PE32+)
14.28 linker version
1E3E00 size of code
92C00 size of initialized data
0 size of uninitialized data
104384 entry point (0000000140104384) mainCRTStartup
1000 base of code
140000000 image base (0000000140000000 to 0000000140279FFF)
1000 section alignment
200 file alignment
6.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
27A000 size of image
400 size of headers
0 checksum
3 subsystem (Windows CUI)
8160 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
Terminal Server Aware
```
Then I built with `<RandomizedBaseAddress>false</RandomizedBaseAddress>` then checked `bitcoind.exe` and `bitcoin-cli.exe` with `dumpbin.exe` and observed that `Dynamic base` was longer listed in `OPTIONAL HEADER VALUES`
bitcoind
```
PS C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64> .\dumpbin.exe /headers C:\Users\e0\Documents\GitHub\bitcoin021noaslr/src/bitcoind.exe
Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
...
OPTIONAL HEADER VALUES
20B magic # (PE32+)
14.28 linker version
AE4600 size of code
33FE00 size of initialized data
0 size of uninitialized data
6BED74 entry point (00000001406BED74) mainCRTStartup
1000 base of code
140000000 image base (0000000140000000 to 0000000140E27FFF)
1000 section alignment
200 file alignment
6.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
E28000 size of image
400 size of headers
0 checksum
3 subsystem (Windows CUI)
8120 DLL characteristics
High Entropy Virtual Addresses
NX compatible
Terminal Server Aware
```
bitcoin-cli
```
> .\dumpbin.exe /headers src/bitcoin-cli.exe
Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
...
OPTIONAL HEADER VALUES
20B magic # (PE32+)
14.28 linker version
1E3E00 size of code
90C00 size of initialized data
0 size of uninitialized data
104384 entry point (0000000140104384) mainCRTStartup
1000 base of code
140000000 image base (0000000140000000 to 0000000140277FFF)
1000 section alignment
200 file alignment
6.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
278000 size of image
400 size of headers
0 checksum
3 subsystem (Windows CUI)
8120 DLL characteristics
High Entropy Virtual Addresses
NX compatible
Terminal Server Aware
```
ACKs for top commit:
sipsorcery:
ACK 9bd3f35003.
practicalswift:
cr ACK 9bd3f35003c2e9eff74766d57a71d6b391ab602a: patch looks correct
Tree-SHA512: ddffdb4ff8a09c7cfef61c07a5db2a2828e9e3aa795ad8e5a1bf51ab489a68b40f87f6694518c5e0b8858c0fad4f93bb947b052e6b9d5e55eb38e764b746fc02
`vcpkg integrate install` must be executed so that msbuild will automatically install external dependencies.
It was removed in 712f95d332
It was originally added in 7644567758
7c543bc27b build: Enable safe warnings for msvc builds (Hennadii Stepanov)
Pull request description:
Disabling of some specific warnings no longer needed.
ACKs for top commit:
MarcoFalke:
ACK 7c543bc27b if green
practicalswift:
ACK 7c543bc27b if green: compiler warnings are good, compiler safety warnings are great
sipsorcery:
tACK 7c543bc27b.
Tree-SHA512: a8253a5a0ccc22b8622ae66efae77a163c54bbd6dbd7e28b043ad4eb474b2c8e5c7b22d3870b15a2b9b91bade0a5ae5bfc5ee765d132dc32ddbbe690fb0146d9
These were added as part of #9366 to fix issues with Protobuf.
Now that we no-longer use Protobuf, there's no reason to maintain a
duplicate set of byteswap tests for qt.
Hennadii Stepanov
