History

Samuel Dobson 8ed37f6c84 Merge #19077 : wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets `c4a29d0a90` Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky) `310b0fde04` Run dumpwallet for legacy wallets only in wallet_backup.py (Andrew Chow) `6c6639ac9f` Include sqlite3 in documentation (Andrew Chow) `f023b7cac0` wallet: Enforce sqlite serialized threading mode (Andrew Chow) `6173269866` Set and check the sqlite user version (Andrew Chow) `9d3d2d263c` Use network magic as sqlite wallet application ID (Andrew Chow) `9af5de3798` Use SQLite for descriptor wallets (Andrew Chow) `9b78f3ce8e` walletutil: Wallets can also be sqlite (Andrew Chow) `ac38a87225` Determine wallet file type based on file magic (Andrew Chow) `6045f77003` Implement SQLiteDatabase::MakeBatch (Andrew Chow) `727e6b2a4e` Implement SQLiteDatabase::Verify (Andrew Chow) `b4df8fdb19` Implement SQLiteDatabase::Rewrite (Andrew Chow) `010e365906` Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow) `ac5c1617e7` Implement SQLiteDatabase::Backup (Andrew Chow) `f6f9cd6a64` Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow) `bf90e033f4` Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow) `7aa45620e2` Add SetupSQLStatements (Andrew Chow) `6636a2608a` Implement SQLiteBatch::Close (Andrew Chow) `93825352a3` Implement SQLiteDatabase::Close (Andrew Chow) `a0de83372b` Implement SQLiteDatabase::Open (Andrew Chow) `3bfa0fe125` Initialize and Shutdown sqlite3 globals (Andrew Chow) `5a488b3d77` Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow) `ca8b7e04ab` Implement SQLiteDatabaseVersion (Andrew Chow) `7577b6e1c8` Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow) `e87df82580` Add sqlite to travis and depends (Andrew Chow) `54729f3f4e` Add libsqlite3 (Andrew Chow) Pull request description: This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`. For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite. We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use. I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier. ACKs for top commit: Sjors: re-utACK `c4a29d0a90` promag: Tested ACK `c4a29d0a90`. fjahr: reACK `c4a29d0a90` S3RK: Re-review ACK `c4a29d0a90` meshcollider: re-utACK `c4a29d0a90` hebasto: re-ACK `c4a29d0a90`, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`. ryanofsky: Code review ACK `c4a29d0a90`. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction. jonatack: ACK `c4a29d0a90`, debug builds and test runs after rebase to latest master @ `c2c4dbaebd`, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns. Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c		4 years ago
..
man	doc: remove OpenSSL from build instructions and licensing info	5 years ago
release-notes	doc: Add historical release notes for 0.20.1	4 years ago
.gitignore	…
Doxyfile.in	[doc] include Doxygen comments for HAVE_BOOST_PROCESS	4 years ago
JSON-RPC-interface.md	Replace `hidden service` with `onion service`	4 years ago
README.md	doc: Update webchat URLs in README.md	5 years ago
README_doxygen.md	doc: Improve doxygen readme navigation section	5 years ago
README_windows.txt	…
REST-interface.md	Merge #19390 : doc/REST-interface: Remove stale info	4 years ago
assets-attribution.md	…
benchmarking.md	Replace current benchmarking framework with nanobench	4 years ago
bips.md	Mention BIP155 in doc/bips.md	4 years ago
bitcoin-conf.md	doc: add default bitcoin.conf locations	5 years ago
bitcoin_logo_doxygen.png	…
build-freebsd.md	doc: mention MAKE=gmake workaround when building on a BSD	5 years ago
build-netbsd.md	doc: mention MAKE=gmake workaround when building on a BSD	5 years ago
build-openbsd.md	Add OpenBSD instructions for building the Qt GUI	4 years ago
build-osx.md	Include sqlite3 in documentation	4 years ago
build-unix.md	Include sqlite3 in documentation	4 years ago
build-windows.md	Add Windows WSL build recommendation to temporarily disable Win32 PE support.	4 years ago
dependencies.md	Include sqlite3 in documentation	4 years ago
descriptors.md	Output a descriptor in createmultisig and addmultisigaddress	5 years ago
developer-notes.md	Remove unused LockAssertion struct	4 years ago
dnsseed-policy.md	…
files.md	Include sqlite3 in documentation	4 years ago
fuzzing.md	doc: afl fuzzing comment about afl-gcc and afl-g++	4 years ago
gitian-building.md	…
init.md	fixing documentation to not require rpcpassword	5 years ago
multiprocess.md	build: multiprocess autotools changes	5 years ago
productivity.md	doc: Drop protobuf stuff	5 years ago
psbt.md	doc: fix/improve analyzepsbt in doc/psbt.md	6 years ago
reduce-memory.md	doc: block-relay-only is not blocksonly	5 years ago
reduce-traffic.md	doc: Use precise permission flags where possible	4 years ago
release-notes-19988.md	Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers	4 years ago
release-notes.md	Merge #20109 : Release notes and followups from 19339	4 years ago
release-process.md	[doc] fix git add argument	5 years ago
shared-libraries.md	doc: correct bitcoinconsensus_version in shared-libraries	6 years ago
tor.md	doc: Update onion service target port numbers in tor.md	4 years ago
translation_process.md	build: Factor out qt translations from build system	5 years ago
translation_strings_policy.md	doc: Do not translate technical or extremely rare errors	5 years ago
zmq.md	doc: Adjust ZMQ usage to support multiple interfaces	4 years ago

README.md

Bitcoin Core

Setup

Bitcoin Core is the original Bitcoin client and it builds the backbone of the network. It downloads and, by default, stores the entire history of Bitcoin transactions, which requires a few hundred gigabytes of disk space. Depending on the speed of your computer and network connection, the synchronization process can take anywhere from a few hours to a day or more.

To download Bitcoin Core, visit bitcoincore.org.

Running

The following are some helpful notes on how to run Bitcoin Core on your native platform.

Unix

Unpack the files into a directory and run:

bin/bitcoin-qt (GUI) or
bin/bitcoind (headless)

Windows

Unpack the files into a directory, and then run bitcoin-qt.exe.

macOS

Drag Bitcoin Core to your applications folder, and then run Bitcoin Core.

Need Help?

See the documentation at the Bitcoin Wiki for help and more information.
Ask for help on #bitcoin on Freenode. If you don't have an IRC client, use webchat here.
Ask for help on the BitcoinTalk forums, in the Technical Support board.

Building

The following are developer notes on how to build Bitcoin Core on your native platform. They are not complete guides, but include notes on the necessary libraries, compile flags, etc.

Development

The Bitcoin repo's root README contains relevant information on the development process and automated testing.

Resources

Discuss on the BitcoinTalk forums, in the Development & Technical Discussion board.
Discuss project-specific development on #bitcoin-core-dev on Freenode. If you don't have an IRC client, use webchat here.
Discuss general Bitcoin development on #bitcoin-dev on Freenode. If you don't have an IRC client, use webchat here.

Miscellaneous

License

Distributed under the MIT software license.