David Burkett
2 years ago
committed by
Loshan T
parent
a14c94f8d2
commit
1049218e66
@ -0,0 +1,21 @@
|
||||
# Light Clients
|
||||
|
||||
Since addresses are not stored on MWEB outputs, identifying coins belonging to a wallet is no longer as simple as sharing a [bloom filter](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki) with a full node, and letting them check for outputs that _probabilistically_ belong to the wallet.
|
||||
|
||||
There are 2 ways that existing light clients identify coins belonging to a wallet:
|
||||
|
||||
## 1. Server-Side Filtering
|
||||
|
||||
[BIP37 - Bloom Filters](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki): Light wallets build a bloom filter for a list of addresses belonging to them, and share it with a full node. That full node then does the work of checking for outputs that _probabilistically_ belong to the wallet.
|
||||
|
||||
#### MWEB Equaivalent
|
||||
|
||||
Since addresses are not stored on MWEB outputs, we cannot identify a wallet's transactions using server-side filtering through probabilistic data structures like bloom filters. The only server-side filtering currently supported for MWEB is filtering using knowledge of private view keys. Light wallets would share their private view keys with a full node, which would then churn through all outputs to find any belonging to the wallet. Unfortunately, this is not a probabilistic process, so full nodes will know with 100% certainty outputs belong to the wallet. To make matters worse, the node will also learn the _value_ of the wallet's outputs. This method is ideal for those who want to use MWEB from a mobile wallet connected to _their own_ full node on a desktop or server, but due to the weakened privacy, you should avoid using this method when connected to untrusted nodes.
|
||||
|
||||
## 2. Client-Side Filtering
|
||||
|
||||
[BIP157 - Client Side Block Filtering](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki): The reverse, where full nodes build a compact filter for each block, and light wallets use those filters to identify blocks that may contain the wallet's transactions.
|
||||
|
||||
#### MWEB Equivalent
|
||||
|
||||
For those who don't have a trusted full node to connect to, we could use something similar to client side block filtering. More precisely, we could perform client side _output_ filtering. Though no method has yet been designed for building filters as compact as those defined in [BIP158 - Compact Block Filters for Light Clients](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki), nodes can provide light clients with _compact_ MWEB outputs, allowing them to avoid downloading the large rangeproofs associated to each output. In fact, we can be even more data efficient, and download only outputs that are verifiably unspent. Following the process defined in [LIP-0006](https://github.com/DavidBurkett/lips/blob/LIP0006/LIP-0006.mediawiki), light clients can download and check all UTXOs from multiple nodes _in parallel_, avoiding the privacy and verifiability shortcomings of the server-side filtering approaches.
|
||||
@ -0,0 +1,175 @@
|
||||
# Prunable Merkle Mountain Ranges (PMMRs)
|
||||
|
||||
Merkle Mountain Ranges (MMRs), an invention of Peter Todd for use in [Open Timestamps](https://github.com/opentimestamps/opentimestamps-server/blob/master/doc/merkle-mountain-range.md), are append-only binary trees that can be used to commit to a collection of elements. In addition to supporting efficient insertion, they also come in a prunable form,called Prunable Merkle Mountain Ranges (PMMRs), which support pruning of entire subtrees. They are used in Litecoin\'s MWEB to build TXO commitments.
|
||||
|
||||
## Example 1
|
||||
|
||||
```
|
||||
Height 4: 30
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
Height 3: 14 29 45
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
/ \ / \ / \
|
||||
Height 2: 06 13 21 28 37 44 52
|
||||
/ \ / \ / \ / \ / \ / \ / \
|
||||
/ \ / \ / \ / \ / \ / \ / \
|
||||
/ \ / \ / \ / \ / \ / \ / \
|
||||
/ \ / \ / \ / \ / \ / \ / \
|
||||
Height 1: 02 05 09 12 17 20 24 27 33 36 40 43 48 51 55
|
||||
/ \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
|
||||
/ \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
|
||||
Height 0: 00 01 03 04 07 08 10 11 15 16 18 19 22 23 25 26 31 32 34 35 38 39 41 42 46 47 49 50 53 54 56
|
||||
|| || || || || || || || || || || || || || || || || || || || || || || || || || || || || || ||
|
||||
Leaves: 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
|
||||
|
||||
Peaks: 30, 45, 52, 55, 56
|
||||
```
|
||||
|
||||
## Glossary
|
||||
|
||||
* Leaf - An item to commit to
|
||||
* Leaf Index - 0-based insertion position of leaves, represented as a 64-bit unsigned integer
|
||||
* Node - A leaf or the parent of leaves or nodes
|
||||
* Position - 0-based index of nodes, represented as a 64-bit unsigned integer
|
||||
* Size - The number of nodes in an unpruned MMR. In our example with 31 leaves (L0-L30), there are 57 nodes (N0-N56)
|
||||
* Sibling - The other leaf or node that shares the same parent (e.g. N14 & N29)
|
||||
* Prune - When a leaf is spent, it shall be marked as pruned. A parent node whose children are both pruned shall also be marked as pruned.
|
||||
* Compact - To remove or "forget" a node
|
||||
* Peak - A node with no parent
|
||||
* HASH(x || y) - The blake3 hash of `x` concatenated with `y`
|
||||
|
||||
## Insertion
|
||||
|
||||
To append a new leaf `L` to an MMR, add a node at height 0 to the right of the last leaf. If the leaf to the left of it has no parent, add a parent node to the 2 leaves. If the node to the left of the newly-added parent node also has no parent, add a parent node for them. Repeat until the last node added does not have a parent-less sibling.
|
||||
|
||||
Let\'s walk through an example. Start with an MMR with 3 leaves (L00, L01, and L02):
|
||||
|
||||
```
|
||||
N02
|
||||
/ \
|
||||
/ \
|
||||
N00 N01 N03
|
||||
| | |
|
||||
L00 L01 L02
|
||||
```
|
||||
|
||||
To add a 4th leaf (L03), append a node (N04) to right of the last leaf node (N03):
|
||||
|
||||
```
|
||||
N02
|
||||
/ \
|
||||
/ \
|
||||
N00 N01 N03 N04
|
||||
| | | |
|
||||
L00 L01 L02 L03
|
||||
```
|
||||
|
||||
The node to the left (N03) of our added node (N04) does not have a parent. So let\'s give them a parent node (N05).
|
||||
|
||||
```
|
||||
N02 N05
|
||||
/ \ / \
|
||||
/ \ / \
|
||||
N00 N01 N03 N04
|
||||
| | | |
|
||||
L00 L01 L02 L03
|
||||
```
|
||||
|
||||
The node to the left (N02) of our added parent node (N05) also does not have a parent. So let\'s add another parent node (N06).
|
||||
|
||||
```
|
||||
N06
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
/ \
|
||||
N02 N05
|
||||
/ \ / \
|
||||
/ \ / \
|
||||
N00 N01 N03 N04
|
||||
| | | |
|
||||
L00 L01 L02 L03
|
||||
```
|
||||
|
||||
There is no node to the left of node our added parent node (N06), so we are finished updating the MMR.
|
||||
|
||||
## Hashing
|
||||
|
||||
All nodes in an MMR are just hashes that commit to an item or a tree of items (in the case of parent nodes).
|
||||
|
||||
To calculate the hash of a leaf `L` containing data `D`:
|
||||
|
||||
```
|
||||
Node(L) = HASH(L.position || D)
|
||||
```
|
||||
|
||||
For leaf `L03` with data `0x0102030405`, the hash would be:
|
||||
|
||||
```
|
||||
HASH(uint64_t(4) || 0x0102030405) == blake3(0x00000000000000040102030405)
|
||||
```
|
||||
|
||||
To calculate the hash for a parent node `P` with left child `C_L` and right child `C_R`:
|
||||
|
||||
```
|
||||
Node(P) = HASH(P.position || Node(C_L) || Node(C_R))
|
||||
```
|
||||
|
||||
For parent node `N05` of left child `N03` with data `0x0102030405` and right child `N04` with data `0xaabbccdd`, the hash would be:
|
||||
|
||||
```
|
||||
HASH(uint64_t(05) || N03 || N04) == blake3(0x0000000000000005 || blake3(0x00000000000000030102030405) || blake3(0x0000000000000004aabbccdd))
|
||||
```
|
||||
|
||||
## PMMR Root
|
||||
|
||||
Similar to how a single hash, the merkle root, can commit to a merkle tree, a single hash, the PMMR root, can be calculated that commits to an entire PMMR. Since there's not a single peak for MMRs the way there is with merkle trees, calculating the root is a little more complicated.
|
||||
|
||||
To calculate the root hash, we start with the rightmost (lowest) peak. We\'ll call this `root_hash`. Then, we work up the mountain range until we reach the highest peak, each step calculating `root_hash = HASH(mmr_size || higher_peak_hash || root_hash)`, where `||` is just the concatenate symbol. This process is called "bagging the peaks."
|
||||
|
||||
In *example 1*, we have 5 disparate trees, and therefore 5 different peaks. So we start by calculating the `mmr_size`, which in our case would be 57 (N0-N56). Then, we assign `root_hash` as the lowest peak (56), and then bag peak 55, then 52, then 45, and finally peak 30:
|
||||
|
||||
```
|
||||
root_hash = N56.hash;
|
||||
root_hash = HASH(57 || N55.hash || root_hash);
|
||||
root_hash = HASH(57 || N52.hash || root_hash);
|
||||
root_hash = HASH(57 || N45.hash || root_hash);
|
||||
root_hash = HASH(57 || N30.hash || root_hash);
|
||||
```
|
||||
|
||||
Without temporary variables, this would be:
|
||||
```
|
||||
root_hash = HASH(57 || N30.hash || HASH(57 || N45.hash || HASH(57 || N52.hash || HASH(57 || N55.hash || N56.hash))));
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
1. Describe difference between pruning and compacting
|
||||
2. Document PruneList
|
||||
3. Explain merkle proofs
|
||||
4. Document `Segment`s and how they\'re built & verified
|
||||
@ -0,0 +1,81 @@
|
||||
# Stealth Addresses
|
||||
|
||||
Transacting on the MWEB happens via stealth addresses. Stealth addresses are supported using the *Dual-Key Stealth Address Protocol (DKSAP)*. These addresses consist of 2 secp256k1 public keys (**A<sub>i</sub>**, **B<sub>i</sub>**) where **A<sub>i</sub>** is the scan pubkey used for identifying outputs, and **B<sub>i</sub>** is the spend pubkey.
|
||||
|
||||
### Notation
|
||||
* **G** = Curve generator point
|
||||
* Bold uppercase letter (**A**, **A<sub>i</sub>**, etc.) represents a secp256k1 public key
|
||||
* Bold lowercase letter (**a**, **a<sub>i</sub>**, etc.) represents a scalar
|
||||
* A letter in single quotes ('K') represents the ASCII value of the character literal
|
||||
* HASH32(x | y | z) represents the standard 32-byte `BLAKE3` hash of the conjoined serializations of `x`, `y`, and `z`
|
||||
* HASH64(m) represents the `BLAKE3` hash of `m` using a 64-byte output digest
|
||||
* SWITCH(**v**, **r**) represents the pedersen commitment generated using a blind switch with value **v** and key **r**.
|
||||
|
||||
### Address Generation
|
||||
|
||||
Unique stealth addresses should be generated deterministically from a wallet's seed using the following formula (using **G** = generator point):
|
||||
|
||||
1. Generate master scan keypair (**a**, **A**) using HD keychain path `m/1/0/100'`
|
||||
2. Generate master spend keypair (**b**, **B**) using HD keychain path `m/1/0/101'`
|
||||
3. Choose the lowest unused address index **i**
|
||||
4. Calculate one-time spend keypair (**b<sub>i</sub>**, **B<sub>i</sub>**) as:<br/>
|
||||
**b<sub>i</sub>** = **b** + HASH32(**A** | **i** | **a**)<br/>
|
||||
**B<sub>i</sub>** = **b<sub>i</sub>\*G** where **G** refers to the curve generator point
|
||||
5. Calculate one-time scan keypair (**a<sub>i</sub>**, **A<sub>i</sub>**) as:<br/>
|
||||
**a<sub>i</sub>** = **a\*b<sub>i</sub>**<br/>
|
||||
**A<sub>i</sub>** = **a<sub>i</sub>\*G** where **G** refers to the curve generator point
|
||||
|
||||
### Outputs
|
||||
|
||||
Outputs consist of the following data:
|
||||
|
||||
* **C<sub>o</sub>** - The pedersen commitment to the value.
|
||||
* Output message, **m**, consisting of:
|
||||
* **K<sub>o</sub>** - The receiver's one-time public key.
|
||||
* **K<sub>e</sub>** - The key exchange public key.
|
||||
* **K<sub>s</sub>** - The sender's public key.
|
||||
* **t** - The view tag. This is the first byte of the shared secret.
|
||||
* **v'** - The masked value.
|
||||
* **n'** - The masked nonce.
|
||||
* A signature of the **m** using the sender's key **k<sub>s</sub>**.
|
||||
* A rangeproof of **C<sub>o</sub>** that also commits to the **m**.
|
||||
|
||||
### Output Construction
|
||||
|
||||
To create an output for value **v** to a receiver's stealth address (**A<sub>i</sub>**,**B<sub>i</sub>**):
|
||||
|
||||
1. Generate a random sender keypair (**k<sub>s</sub>**, **K<sub>s</sub>**). # TODO: Can we generate this deterministically?
|
||||
2. Derive the nonce **n** as the first 16 bytes of HASH32('N'|**k<sub>s</sub>**)
|
||||
3. Derive the sending key **s** = HASH32('S'|**A<sub>i</sub>**|**B<sub>i</sub>**|**v**|**n**)
|
||||
4. Derive the shared secret **e** = HASH32('D'|**s*A<sub>i</sub>**). The view tag **t** is the first byte of **e**.
|
||||
5. Calculate the receiver's one-time public key **K<sub>o</sub>** = **B<sub>i</sub>** + HASH32('O'|**e**)***G**
|
||||
6. Calculate the key exchange pubkey **K<sub>e</sub>** = **s*B<sub>i</sub>**
|
||||
7. Derive the 64-byte mask **m** = HASH64(**e**)
|
||||
8. Encrypt the value using **v'** = **v** ⊕ **m[32,39]**
|
||||
9. Encrypt the nonce using **n'** = **n** ⊕ **m[40,55]**
|
||||
10. Calculate the commitment **C<sub>o</sub>** = SWITCH(**v**, **m[0,31]**).
|
||||
11. Generate the rangeproof for **C<sub>o</sub>**, committing also to **m**.
|
||||
12. Sign **m** using the sender key **k<sub>s</sub>**.
|
||||
|
||||
|
||||
### Output Identification
|
||||
|
||||
NOTE: the wallet must keep a map **B<sub>i</sub>**->**i** of all used spend pubkeys and the next few unused ones.
|
||||
|
||||
To check if an output belongs to a wallet:
|
||||
|
||||
1. Calculate the ECDHE shared secret **e** = HASH32('D'|**a*K<sub>e</sub>**)
|
||||
2. If the first byte of **e** does not match the view tag **t**, the output does not belong to the wallet.
|
||||
3. Calculate the one-time spend pubkey: **B<sub>i</sub>** = **K<sub>o</sub>** - HASH('O'|**e**)***G**
|
||||
4. Lookup the index **i** that generates **B<sub>i</sub>** from the wallet's map **B<sub>i</sub>**->**i**. If not found, the output does not belong to the wallet.
|
||||
5. Derive the 64-byte mask **m** = HASH64(**e**)
|
||||
6. Decrypt the value **v** = **v'** ⊕ **m[32,39]** where **m[32,39]** refers to bytes 32-39 (0-based big-endian) of the 64 byte mask **m**.
|
||||
7. Verify that SWITCH(**v**,**m[0-31]**) ?= **C<sub>o</sub>**
|
||||
8. Decrypt the nonce **n** = **n'** ⊕ **m[40,55]**.
|
||||
9. Calculate the send key **s** = HASH32('S'|**A<sub>i</sub>**|**B<sub>i</sub>**|**v**|**n**)
|
||||
10. Verify that **K<sub>e</sub>** ?= **s*B<sub>i</sub>**.
|
||||
|
||||
If all verifications succeed, the output belongs to the wallet, and is safe to use.
|
||||
|
||||
The spend key can be recovered by **k<sub>o</sub>** = HASH32('O'|**e**) + **a<sub>i</sub>**.
|
||||
|
||||
@ -0,0 +1,73 @@
|
||||
# Data Storage
|
||||
|
||||
In addition to extending the existing `CBlock` and `CTransaction` objects already used in Litecoin, the following data stores are created or modified for MWEB.
|
||||
|
||||
### CBlockUndo
|
||||
|
||||
After a new block is connected, a `rev<index>.dat` file is created that describes how to remove or "undo" the block from the chain state.
|
||||
When performing a reorg, the undo file can be processed, which will revert the UTXO set back to the way it was just before the block.
|
||||
|
||||
To support undo-ing MWEB blocks, we''ve added a new `mw::BlockUndo` object to `CBlockUndo`, which gets serialized at the end.
|
||||
This contains the following fields:
|
||||
|
||||
* `prev_header` - the MWEB header in the previous block
|
||||
* `utxos_spent` - vector of `UTXO`s that were spent in the block
|
||||
* `utxos_added` - vector of coin IDs (hashes) that were added in the block
|
||||
|
||||
To revert the block from the chain state, the node adds back the `UTXO`s in `utxos_spent`, removes the matching `UTXO`s in `utxos_added`, and sets `prev_header` as the MWEB chain tip.
|
||||
|
||||
NOTE: For backward compatibility, when deserializing a `CBlockUndo`, we first must look up the size of the `CBlockUndo` object.
|
||||
After deserializing vtxundo (the vector of `CTxUndo`s), if more data remains, assume it\'s the `mw::BlockUndo` object.
|
||||
If no more data remains, assume the `CBlockUndo` does not have MWEB data.
|
||||
An `UnserializeBlockUndo` function was added to handle this.
|
||||
|
||||
### UTXOs
|
||||
##### CoinDB (leveldb)
|
||||
|
||||
Litecoin's leveldb instance is used to maintain a UTXO table (prefix: 'U') with `UTXO` objects, consisting of the following data fields:
|
||||
|
||||
* output_hash (key) - The hash of the output.
|
||||
* block_height - The block height the UTXO was included.
|
||||
* leaf_index - The index of the leaf in the output PMMR.
|
||||
* output - The full `Output` object, including the rangeproof and owner data.
|
||||
|
||||
### PMMRs
|
||||
##### MMR Info (leveldb)
|
||||
Litecoin's leveldb instance is used to maintain an MMR Info table (prefix: "M") with `MMRInfo` objects consisting of the following data fields:
|
||||
|
||||
* version - A version byte that allows for future schema upgrades.
|
||||
* index (key) - File number of the PMMR files.
|
||||
* pruned_hash - Hash of latest header this PMMR represents.
|
||||
* compact_index - File number of the PruneList bitset.
|
||||
* compacted_hash - Hash of the header this MMR was compacted for. You cannot rewind beyond this point.
|
||||
|
||||
Each time the PMMRs are flushed to disk, a new MMRInfo object is written to the DB and marked as the latest.
|
||||
|
||||
##### Leaves (leveldb)
|
||||
Litecoin's leveldb instance is used to maintain MMR leaf tables (prefix: 'O' for outputs) to store uncompacted PMMR leaves consisting of the following data fields:
|
||||
|
||||
* leaf_index (key) - The zero-based leaf position.
|
||||
* leaf - The raw leaf data committed to by the PMMR.
|
||||
|
||||
Leaves spent before the horizon will be removed during compaction.
|
||||
|
||||
##### MMR Hashes (file)
|
||||
|
||||
Stored in file `<prefix><index>.dat` where `<prefix>` refers to 'O' for the output PMMR, and `<index>` is a 6-digit number that matches the `index` value of the latest `MMRInfo` object.
|
||||
Example: If the latest `MMRInfo` object has an `index` of 123, the matching output PMMR hash file will be named `O000123.dat`.
|
||||
|
||||
The hash file consists of un-compacted leaf hashes and their parent hashes.
|
||||
|
||||
##### Leafset (file)
|
||||
|
||||
Stored in file `leaf<index>.dat`.
|
||||
|
||||
The leafset file consists of a bitset indicating which leaf indices of the output PMMR are unspent.
|
||||
Example: If the PMMR contains 5 leaves where leaf indices 0, 1, and 2 are spent, but 3 and 4 are unspent, the file will contain a single byte of 00011000 = 0x18.
|
||||
|
||||
##### PruneList (file)
|
||||
|
||||
Stored in file `prun<index>.dat`.
|
||||
|
||||
The prunelist file consists of a bitset indicating which nodes of the output PMMR are not included in the output PMMR hash file.
|
||||
Example: If nodes 0, 1, 3, and 4 are compacted (not included in PMMR), then the first byte of the prune list bitset will be 11011000 = 0xD8.
|
||||
Loading…
Reference in new issue