Litecoin Core integration/staging tree

Go to file

MarcoFalke 7da9e3a817 Merge #17050 : tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32) `a1308b7e12` tests: Add fuzzing harnesses for various JSON/univalue parsing functions (practicalswift) `e3d2bcf5cf` tests: Add fuzzing harnesses for various number parsing functions (practicalswift) `fb8c12093a` tests: Add ParseScript(...) (core_io) fuzzing harness (practicalswift) `074cb6451b` tests: Add ParseHDKeypath(...) (bip32) fuzzing harness (practicalswift) `0dc5907d0f` tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift) Pull request description: Add fuzzing harnesses for `DecodeRawPSBT(...)`, `ParseHDKeypath(...)`, `ParseScript(...)`, various number parsing functions and various JSON/univalue parsing functions. Testing this PR As usual the best way to test proposed fuzzing harnesses is to use `test_fuzzing_harnesses.sh` (#17000) to quickly verify that the relevant code regions are triggered, that the fuzzing throughput seems reasonable, etc. `test_fuzzing_harnesses.sh 'psbt\|hd_keypath\|numbers\|parse_script\|univalue' 10` runs all fuzzers matching the regexp and gives them ten seconds of runtime each. ``` $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined $ make $ contrib/devtools/test_fuzzing_harnesses.sh 'psbt\|hd_keypath\|numbers\|parse_script\|univalue' 10 Testing fuzzer parse_hd_keypath during 10 second(s) A subset of reached functions: NEW_FUNC[0/2]: 0x55bc23a76940 in ParsePrechecks(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:267 NEW_FUNC[1/2]: 0x55bc23a77300 in ParseUInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int) src/util/strencodings.cpp:309 stat::number_of_executed_units: 34237 stat::average_exec_per_sec: 3112 stat::new_units_added: 113 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 282 Number of unique code paths taken during fuzzing round: 30 Testing fuzzer parse_numbers during 10 second(s) A subset of reached functions: stat::number_of_executed_units: 31309 stat::average_exec_per_sec: 2846 stat::new_units_added: 688 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 234 Number of unique code paths taken during fuzzing round: 149 Testing fuzzer parse_script during 10 second(s) A subset of reached functions: NEW_FUNC[1/11]: 0x5636ff61ba00 in IsDigit(char) src/./util/strencodings.h:70 NEW_FUNC[0/14]: 0x5636fe6c6280 in CScript::operator<<(opcodetype) src/./script/script.h:448 NEW_FUNC[1/14]: 0x5636fe6e0290 in prevector<28u, unsigned char, unsigned int, int>::insert(prevector<28u, unsigned char, unsigned int, int>::iterator, unsigned char const&) src/./prevector.h:342 NEW_FUNC[2/14]: 0x5636fe6e1040 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277 NEW_FUNC[3/14]: 0x5636fe6e1250 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295 NEW_FUNC[4/14]: 0x5636fe6e1cb0 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196 NEW_FUNC[0/10]: 0x5636fe6c5650 in CScript::operator<<(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/./script/script.h:462 NEW_FUNC[2/10]: 0x5636fe6e0a20 in void prevector<28u, unsigned char, unsigned int, int>::insert<__gnu_cxx::__normal_iterator<unsigned char const, std::vector<unsigned char, std::allocator<unsigned char> > > >(prevector<28u, unsigned char, unsigned int, int>::iterator, __gnu_cxx::__normal_iterator<unsigned char const, std::vector<unsigned char, std::allocator<[32/1902] char> > >, __gnu_cxx::__normal_iterator<unsigned char const, std::vector<unsigned char, std::allocator<unsigned char> > >) src/./prevector.h:368 NEW_FUNC[5/10]: 0x5636fe6e2350 in void prevector<28u, unsigned char, unsigned int, int>::fill<__gnu_cxx::__normal_iterator<unsigned char const, std::vector<unsigned char, std::allocator<unsigned char> > > >(unsigned char, __gnu_cxx::__normal_iterator<unsigned char const, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsign ed char const, std::vector<unsigned char, std::allocator<unsigned char> > >) src/./prevector.h:204 NEW_FUNC[0/1]: 0x5636ff8e48b0 in IsHex(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:61 NEW_FUNC[0/2]: 0x5636fe6e1410 in prevector<28u, unsigned char, unsigned int, int>::change_capacity(unsigned int) src/./prevector.h:165 NEW_FUNC[1/2]: 0x5636fe6e1f00 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) src/./prevector.h:161 NEW_FUNC[0/1]: 0x5636fe6e0580 in void prevector<28u, unsigned char, unsigned int, int>::insert<unsigned char>(prevector<28u, unsigned char, unsigned int, int>::iterator, unsigned char, unsigned char) src/./prevector.h:368 NEW_FUNC[0/3]: 0x5636fe85f0d0 in CScript::push_int64(long) src/./script/script.h:394 NEW_FUNC[1/3]: 0x5636fe85f520 in prevector<28u, unsigned char, unsigned int, int>::push_back(unsigned char const&) src/./prevector.h:422 NEW_FUNC[2/3]: 0x5636ff8ed730 in atoi64(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:417 stat::number_of_executed_units: 8153 stat::average_exec_per_sec: 741 stat::new_units_added: 296 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 237 Number of unique code paths taken during fuzzing round: 98 Testing fuzzer parse_univalue during 10 second(s) A subset of reached functions: NEW_FUNC[0/19]: 0x560db8655950 in tinyformat::detail::formatImpl(std::ostream&, char const, tinyformat::detail::FormatArg const, int) src/./tinyformat.h:791 NEW_FUNC[4/19]: 0x560db86582b0 in tinyformat::detail::printFormatStringLiteral(std::ostream&, char const) src/./tinyformat.h:564 NEW_FUNC[5/19]: 0x560db8658690 in tinyformat::detail::streamStateFromFormat(std::ostream&, bool&, int&, char const, tinyformat::detail::FormatArg const, int&, int) src/./tinyformat.h:601 NEW_FUNC[6/19]: 0x560db865f090 in tinyformat::detail::FormatArg::format(std::ostream&, char const, char const, int) const src/./tinyformat.h:513 NEW_FUNC[12/19]: 0x560db8661ba0 in void tinyformat::detail::FormatArg::formatImpl<int>(std::ostream&, char const, char const, int, void const) src/./tinyformat.h:530 NEW_FUNC[13/19]: 0x560db8661d90 in void tinyformat::formatValue<int>(std::ostream&, char const, char const, int, int const&) src/./tinyformat.h:317 NEW_FUNC[14/19]: 0x560db875c8b0 in void tinyformat::detail::FormatArg::formatImpl<unsigned int>(std::ostream&, char const, char const, int, void const) src/./tinyformat.h:530 NEW_FUNC[15/19]: 0x560db875caa0 in void tinyformat::formatValue<unsigned int>(std::ostream&, char const, char const, int, unsigned int const&) src/./tinyformat.h:317 NEW_FUNC[16/19]: 0x560db9473ef0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<int, unsigned int>(char const, int const&, unsigned int const&) src/./tinyformat.h:976 NEW_FUNC[17/19]: 0x560db94749a0 in void tinyformat::format<int, unsigned int>(std::ostream&, char const, int const&, unsigned int const&) src/./tinyformat.h:968 NEW_FUNC[18/19]: 0x560db9474cf0 in tinyformat::detail::FormatListN<2>::FormatListN<int, unsigned int>(int const&, unsigned int const&) src/./tinyformat.h:885 stat::number_of_executed_units: 14089 stat::average_exec_per_sec: 1280 stat::new_units_added: 135 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 356 Number of unique code paths taken during fuzzing round: 62 Testing fuzzer psbt_input_deserialize during 10 second(s) A subset of reached functions: NEW_FUNC[0/46]: 0x557847ce3530 in prevector<28u, unsigned char, unsigned int, int>::~prevector() src/./prevector.h:456 NEW_FUNC[3/46]: 0x557847cfdcf0 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277 NEW_FUNC[4/46]: 0x557847cfe0c0 in prevector<28u, unsigned char, unsigned int, int>::change_capacity(unsigned int) src/./prevector.h:165 NEW_FUNC[13/46]: 0x557847d3c890 in unsigned long ReadCompactSize<CDataStream>(CDataStream&) src/./serialize.h:290 NEW_FUNC[14/46]: 0x557847d47b60 in prevector<28u, unsigned char, unsigned int, int>::resize(unsigned int) src/./prevector.h:311 NEW_FUNC[16/46]: 0x557847d48800 in CTxOut::CTxOut() src/./primitives/transaction.h:140 NEW_FUNC[17/46]: 0x557847d4b050 in CTxOut::SetNull() src/./primitives/transaction.h:155 NEW_FUNC[18/46]: 0x557847d4b140 in CScript::clear() src/./script/script.h:563 NEW_FUNC[19/46]: 0x557847d4ead0 in void Unserialize_impl<CDataStream, unsigned char, std::allocator<unsigned char> >(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> >&, unsigned char const&) src/./serialize.h:746 NEW_FUNC[0/58]: 0x557847cfdf00 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295 NEW_FUNC[1/58]: 0x557847cfe960 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196 NEW_FUNC[2/58]: 0x557847cfebb0 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) src/./prevector.h:161 NEW_FUNC[3/58]: 0x557847d03990 in uint256::uint256() src/./uint256.h:123 NEW_FUNC[0/3]: 0x557847d47430 in void CScript::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./script/script.h:418 NEW_FUNC[1/3]: 0x557847d47730 in void Unserialize_impl<CDataStream, 28u, unsigned char>(CDataStream&, prevector<28u, unsigned char, unsigned int, int>&, unsigned char const&) src/./serialize.h:666 NEW_FUNC[2/3]: 0x557847d60dd0 in CDataStream& CDataStream::operator>><CScript&>(CScript&) src/./streams.h:460 NEW_FUNC[1/78]: 0x557847cffae0 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) const src/./prevector.h:197 NEW_FUNC[2/78]: 0x557847cffd30 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) const src/./prevector.h:162 NEW_FUNC[0/1]: 0x557847d65f90 in OverrideStream<CDataStream>& OverrideStream<CDataStream>::operator>><unsigned char&>(unsigned char&) src/./streams.h:46 NEW_FUNC[0/3]: 0x557847d470e0 in void SerReadWriteMany<CDataStream, CScript&>(CDataStream&, CSerActionUnserialize, CScript&) src/./serialize.h:989 NEW_FUNC[1/3]: 0x557847d4ac50 in void CTxOut::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./primitives/transaction.h:149 NEW_FUNC[2/3]: 0x557847d5f860 in void UnserializeFromVector<CDataStream, CTxOut>(CDataStream&, CTxOut&) src/./script/sign.h:90 NEW_FUNC[0/1]: 0x557847d60840 in void UnserializeFromVector<CDataStream, int>(CDataStream&, int&) src/./script/sign.h:90 NEW_FUNC[0/1]: 0x557847d41010 in CMutableTransaction::HasWitness() const src/./primitives/transaction.h:398 stat::number_of_executed_units: 13615 stat::average_exec_per_sec: 1237 stat::new_units_added: 357 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 446 Number of unique code paths taken during fuzzing round: 152 Testing fuzzer psbt_output_deserialize during 10 second(s) A subset of reached functions: NEW_FUNC[0/27]: 0x55c9347e5940 in prevector<28u, unsigned char, unsigned int, int>::~prevector() src/./prevector.h:456 NEW_FUNC[5/27]: 0x55c93483eca0 in unsigned long ReadCompactSize<CDataStream>(CDataStream&) src/./serialize.h:290 NEW_FUNC[6/27]: 0x55c934850ee0 in void Unserialize_impl<CDataStream, unsigned char, std::allocator<unsigned char> >(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> >&, unsigned char const&) src/./serialize.h:746 NEW_FUNC[14/27]: 0x55c934858500 in PSBTOutput::PSBTOutput() src/./psbt.h:281 NEW_FUNC[15/27]: 0x55c934858870 in CDataStream& CDataStream::operator>><PSBTOutput&>(PSBTOutput&) src/./streams.h:460 NEW_FUNC[0/1]: 0x55c934800100 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277 NEW_FUNC[0/4]: 0x55c934849840 in void CScript::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./script/script.h:418 NEW_FUNC[1/4]: 0x55c934849b40 in void Unserialize_impl<CDataStream, 28u, unsigned char>(CDataStream&, prevector<28u, unsigned char, unsigned int, int>&, unsigned char const&) src/./serialize.h:666 NEW_FUNC[2/4]: 0x55c934849f70 in prevector<28u, unsigned char, unsigned int, int>::resize(unsigned int) src/./prevector.h:311 NEW_FUNC[3/4]: 0x55c93485dc60 in CDataStream& CDataStream::operator>><CScript&>(CScript&) src/./streams.h:460 NEW_FUNC[0/3]: 0x55c934800310 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295 NEW_FUNC[1/3]: 0x55c934800d70 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196 NEW_FUNC[2/3]: 0x55c934849d40 in prevector<28u, unsigned char, unsigned int, int>::resize_uninitialized(unsigned int) src/./prevector.h:381 NEW_FUNC[0/1]: 0x55c93485ddd0 in void DeserializeHDKeypaths<CDataStream>(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> > const&, std::map<CPubKey, KeyOriginInfo, std::less<CPubKey>, std::allocator<std::pair<CPubKey const, KeyOriginInfo> > >&) src/./script/sign.h:103 stat::number_of_executed_units: 19130 stat::average_exec_per_sec: 1739 stat::new_units_added: 195 stat::slowest_unit_time_sec: 0 stat::peak_rss_mb: 411 Number of unique code paths taken during fuzzing round: 64 Tested fuzz harnesses seem to work as expected. ``` Top commit has no ACKs. Tree-SHA512: baf1630a6e438d02d33c77b9e602c99546b9e8d83705e67c2749e0600039c37707cdf419cee19282f069e8d787c536ed4960f9c47e93bd0f0251495b83780ada		5 years ago
.github	Adds GitHub Action workflow which duplicates AppVeyor job.	5 years ago
.tx	gui: Update transifex slug for 0.19	5 years ago
build-aux/m4	Merge #16110 : depends: Add Android NDK support	5 years ago
build_msvc	Appveyor install libevent[thread] vcpkg	5 years ago
ci	Merge #17635 : ci: Add CentOS 7 build	5 years ago
contrib	Merge #17680 : gitian: fixed SC2001 regex	5 years ago
depends	Merge #17698 : depends: don't configure xcb_proto	5 years ago
doc	Merge #17713 : doc: Add release notes for 17447	5 years ago
share	build: set minimum supported macOS to 10.12	5 years ago
src	Merge #17050 : tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32)	5 years ago
test	Merge #17050 : tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32)	5 years ago
.appveyor.yml	Moves vcpkg list to a text file and updates the appveyor job and readme to use it.	5 years ago
.cirrus.yml	ci: remove OpenSSL installation	5 years ago
.gitattributes	Separate protocol versioning from clientversion	10 years ago
.gitignore	Merge #16371 : build: ignore macOS make deploy artefacts & add them to clean-local	5 years ago
.python-version	.python-version: Specify full version 3.5.6	6 years ago
.style.yapf	test: .style.yapf: Set column_limit=160	6 years ago
.travis.yml	Merge #17635 : ci: Add CentOS 7 build	5 years ago
CONTRIBUTING.md	doc: Added instructions for how to add an upsteam to forked repo	5 years ago
COPYING	[Trivial] Update license year range to 2019	6 years ago
INSTALL.md	Update INSTALL landing redirection notice for build instructions.	8 years ago
Makefile.am	lcov: filter /usr/lib64 from coverage report	5 years ago
README.md	doc: Fix some misspellings	5 years ago
SECURITY.md	doc: Remove explicit mention of version from SECURITY.md	5 years ago
autogen.sh	Added double quotes	5 years ago
configure.ac	build: Use dnl for all comments in configure.ac, rather than #	5 years ago
libbitcoinconsensus.pc.in	build: remove libcrypto as internal dependency in libbitcoinconsensus.pc	5 years ago

README.md

Bitcoin Core integration/staging tree

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately usable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built and tested, but is not guaranteed to be completely stable. Tags are created regularly to indicate new official, stable release versions of Bitcoin Core.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.