History

laanwj 7008087548 Merge bitcoin/bitcoin#24410 : [kernel 2a/n] Split hashing/index `GetUTXOStats` codepaths, decouple from `coinstatsindex` `664a14ba7c` coinstats: Move GetUTXOStats to rpc/blockchain (Carl Dong) `f100687566` kernel: Use ComputeUTXOStats in validation (Carl Dong) `faa52387e8` style-only: Rearrange using decls after scripted-diff (Carl Dong) `f329a9298c` scripted-diff: Move src/kernel/coinstats to kernel:: (Carl Dong) `0e54456f04` Use only kernel/coinstats.h in index/coinstatsindex.h (Carl Dong) `80970985c9` coinstats: Split node/coinstats.h to kernel/coinstats.h (Carl Dong) `35f73ce4b2` coinstats: Move hasher codepath to kernel/coinstats (Carl Dong) `b7634fe02b` Move logic from LookupUTXOStatsWithIndex to CoinStatsIndex::LookUpStats (Carl Dong) `1352e410a5` coinstats: Separate hasher/index lookup codepaths (Carl Dong) `524463daf6` coinstats: Return purely out-param CCoinsStats (Carl Dong) `46eb9fc56a` coinstats: Extract index_requested in-member to in-param (Carl Dong) `a789f3f2b8` coinstats: Extract hash_type in-member to in-param (Carl Dong) `102294898d` includes: Remove rpc/util.h -> node/coinstats.h (Carl Dong) `0848db9c35` fuzz: Remove useless GetUTXOStats fuzz case (Carl Dong) `52b1939993` kernel: Remove unnecessary blockfilter{index,}.cpp (Carl Dong) Pull request description: Part of: #24303 Depends on: #24322 The `GetUTXOStats` function has 2 codepaths: - One which queries the `CoinStatsIndex` for the UTXO hash - One which actually performs the hashing For `libbitcoinkernel`, the only place where we call `GetUTXOStats` is in `PopulateAndValidateSnapshots`, which uses the `SHA256D` hash, and is therefore unable to use the `CoinStatsIndex` since that only provides `MuHash` hashes. Not that I think indices necessarily belong in `libbitcoinkernel` anyway. This PR separates these 2 aforementioned codepaths of `GetUTXOStats`, uses the hashing codepath in `PopulateAndValidateSnapshots`, and removes the need to link in `index/coinstatsindex.cpp` and `node/coinstats.cpp`. ----- Logistically, this PR: - Extracts out the `index_requested` and `hash_type` members of `CoinStats`, which served as "in-params" to `GetUTXOStats` embedded within the `CoinStats` struct. This allows `CoinStats` to only consist of "out-param" members, and be returned by `GetUTXOStats` without needing to be an "in-out" param - Introduce the purely virtual `UTXOHashers` class, with 3 implementations: `SHA256DHasher`, `MuHashHasher`, and `NullHasher`. These replace the existing template-based polymorphism. - Split `GetUTXOStats` into: - `CalculateUTXOStatsWithHasher(UTXOHasher&, ...)`, and - `LookupUTXOStatsWithIndex(CoinStatsIndex&, ...)` - Use `CalculateUTXOStatsWithHasher` directly where appropriate (`src/validation.cpp` and `src/fuzz`) - Move `GetUTXOStats` to `rpc/blockchain`, which is the only place that depends on `GetUTXOStats`'s weird fallback behaviour - Move `LookupUTXOStatsWithIndex` to `index/coinstatsindex` Code organization: - `src/` - `kernel/` → only contains the hashing codepath - `coinstats.cpp` → hashing codepath implementations - `coinstats.h` → header for `kernel/coinstats.cpp` - `index/` → only contains the index codepath - `coinstatsindex.cpp` → index codepath implementations - `coinstatsindex.h` - `validation.cpp` → only uses the hashing codepath - `rpc/blockchain.cpp` → uses both the hashing and index codepath, old `GetUTXOStats` fallback logic moved here as static - `test/fuzz/coins_view.cpp` → only uses the hashing codepath TODOs: - [x] Commit messages could be fleshed out more Would love any feedback! ACKs for top commit: laanwj: Code review ACK `664a14ba7c` Tree-SHA512: 18722c7bd279174d2d1881fec33ea04a9b261aae1c12e998cf434ef297d8ded47de69c526c8033a2ba7abc93ba3d2ff5faf4ce05e8888c725c31cf885ce3ef73		3 years ago
..
README.md	Converting `lint-all.sh` to `lint-all.py`.	3 years ago
check-doc.py	…
commit-script-check.sh	…
git-subtree-check.sh	…
lint-all.py	Converting `lint-all.sh` to `lint-all.py`.	3 years ago
lint-assertions.py	lint: Mention NONFATAL_UNREACHABLE in lint-assertions.py	3 years ago
lint-circular-dependencies.py	kernel: Use ComputeUTXOStats in validation	3 years ago
lint-files.py	test: Use permissions from git in `lint-files.py`	3 years ago
lint-format-strings.py	lint: Fix lint-circular-dependencies.py file list	3 years ago
lint-git-commit-check.py	lint: Convert lint-git-commit-check.sh to Python	3 years ago
lint-include-guards.py	Convert lint-include-guards.sh to python	3 years ago
lint-includes.py	http: replace boost::split with SplitString	3 years ago
lint-locale-dependence.py	test: Add `strerror` to locale-dependence linter	3 years ago
lint-logs.py	lint: Convert lint-logs.sh to Python	3 years ago
lint-python-dead-code.py	lint: Convert Python dead code linter to Python	3 years ago
lint-python-mutable-default-parameters.py	Converted lint-python-mutable-default-parameters.sh to python	3 years ago
lint-python-utf8-encoding.py	lint: Convert lint-python-utf8-encoding.sh to Python	3 years ago
lint-python.py	lint: Convert Python linter to Python	3 years ago
lint-shell-locale.py	lint: convert shell locale linter test to Python	3 years ago
lint-shell.py	test: port 'lint-shell.sh' to python	3 years ago
lint-spelling.py	lint: convert spell check lint test to python	3 years ago
lint-submodule.py	lint: convert submodule linter test to Python	3 years ago
lint-tests.py	Converted lint-tests.sh to python	3 years ago
lint-whitespace.py	lint: Convert lint-whitespace.sh to Python	3 years ago
run-lint-format-strings.py	…
spelling.ignore-words.txt	…

README.md

This folder contains lint scripts.

check-doc.py

Check for missing documentation of command line options.

commit-script-check.sh

Verification of scripted diffs. Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems might require installing GNU tools, such as GNU sed.

git-subtree-check.sh

Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.

Usage: test/lint/git-subtree-check.sh [-r] DIR [COMMIT]
       test/lint/git-subtree-check.sh -?

DIR is the prefix within the repository to check.
COMMIT is the commit to check, if it is not provided, HEAD will be used.
-r checks that subtree commit is present in repository.

To do a full check with -r, make sure that you have fetched the upstream repository branch in which the subtree is maintained:

for src/secp256k1: https://github.com/bitcoin-core/secp256k1.git (branch master)
for src/leveldb: https://github.com/bitcoin-core/leveldb-subtree.git (branch bitcoin-fork)
for src/univalue: https://github.com/bitcoin-core/univalue-subtree.git (branch master)
for src/crypto/ctaes: https://github.com/bitcoin-core/ctaes.git (branch master)
for src/crc32c: https://github.com/bitcoin-core/crc32c-subtree.git (branch bitcoin-fork)
for src/minisketch: https://github.com/sipa/minisketch.git (branch master)

To do so, add the upstream repository as remote:

git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git

lint-all.py

Calls other scripts with the lint- prefix.