Skip to content

Commit d2658c7

Browse files
committed
remove callbacks
We want to migrate this crate into a direction where we provide out-of-the-box all of the ways in which a header and / or block could be analyzed. The user can then switch these analyses on or off. For this functionality, we don't need the overkill flexibility of trait objects. The goal is to provide a much more authoritative block analysis tool, closer to an enhanced block explorer than the ad-hoc analyses implemented before.
1 parent 5d73fb5 commit d2658c7

File tree

10 files changed

+28
-793
lines changed

10 files changed

+28
-793
lines changed

README.md

Lines changed: 1 addition & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,7 @@ If something doesn't match the parser exits.
2121

2222
## Usage
2323
```
24-
Usage: bitcoin-blockparser [OPTIONS] [COMMAND]
25-
26-
Commands:
27-
unspentcsvdump Dumps the unspent outputs to CSV file
28-
simplestats Shows various Blockchain stats
29-
balances Dumps all addresses with non-zero balance to CSV file
30-
opreturn Shows embedded OP_RETURN data that is representable as UTF8
31-
help Print this message or the help of the given subcommand(s)
24+
Usage: bitcoin-blockparser [OPTIONS]
3225
3326
Options:
3427
--verify
@@ -48,31 +41,6 @@ Options:
4841
-V, --version
4942
Print version
5043
```
51-
### Example
52-
53-
To make a `unspentcsvdump` of the Bitcoin blockchain your command would look like this:
54-
```
55-
# ./blockparser unspentcsvdump /path/to/dump/
56-
[6:02:53] INFO - main: Starting bitcoin-blockparser v0.7.0 ...
57-
[6:02:53] INFO - index: Reading index from ~/.bitcoin/blocks/index ...
58-
[6:02:54] INFO - index: Got longest chain with 639626 blocks ...
59-
[6:02:54] INFO - blkfile: Reading files from ~/.bitcoin/blocks ...
60-
[6:02:54] INFO - parser: Parsing Bitcoin blockchain (range=0..) ...
61-
[6:02:54] INFO - callback: Using `unspentcsvdump` with dump folder: /path/to/dump ...
62-
[6:03:04] INFO - parser: Status: 130885 Blocks processed. (left: 508741, avg: 13088 blocks/sec)
63-
...
64-
[10:28:47] INFO - parser: Status: 639163 Blocks processed. (left: 463, avg: 40 blocks/sec)
65-
[10:28:57] INFO - parser: Status: 639311 Blocks processed. (left: 315, avg: 40 blocks/sec)
66-
[10:29:07] INFO - parser: Status: 639452 Blocks processed. (left: 174, avg: 40 blocks/sec)
67-
[10:29:17] INFO - parser: Status: 639596 Blocks processed. (left: 30, avg: 40 blocks/sec)
68-
[10:29:19] INFO - parser: Done. Processed 639626 blocks in 266.43 minutes. (avg: 40 blocks/sec)
69-
[10:32:01] INFO - callback: Done.
70-
Dumped all 639626 blocks:
71-
-> transactions: 549390991
72-
-> inputs: 1347165535
73-
-> outputs: 1359449320
74-
[10:32:01] INFO - main: Fin.
75-
```
7644

7745

7846
## Installing
@@ -108,63 +76,6 @@ It is important to build with `--release`, otherwise you will get a horrible per
10876
Bitcoin and Bitcoin Testnet transactions are parsed using [rust-bitcoin](https://github.com/rust-bitcoin/rust-bitcoin),
10977
this includes transactions of type P2SH, P2PKH, P2PK, P2WSH, P2WPKH, P2TR, OP_RETURN and SegWit.
11078

111-
112-
## Memory Usage
113-
The required memory usage depends on the used callback:
114-
115-
* simplestats: ~100MB
116-
* unspentcsvdump: ~18GB
117-
* balances: ~18GB
118-
119-
NOTE: Those values are taken from parsing to block height 639631 (17.07.2020).
120-
121-
## Callbacks
122-
123-
Callbacks are built on top of the core parser. They can be implemented to extract specific types of information.
124-
125-
* `balances`: dumps all addresses with a non-zero balance.
126-
The csv file is in the following format:
127-
```
128-
balances.csv
129-
address ; balance
130-
```
131-
132-
* `unspentcsvdump`: dumps all UTXOs along with the address balance.
133-
The csv file is in the following format:
134-
```
135-
unspent.csv
136-
txid ; indexOut ; height ; value ; address
137-
```
138-
NOTE: The total size of the csv dump is at least 8 GiB (height 635000).
139-
140-
* `opreturn`: shows transactions with embedded OP_RETURN data that is representable as UTF8.
141-
142-
See [Usage](#Usage) for an example. I chose CSV dumps instead of an active db-connection because `LOAD DATA INFILE` is the most performant way for bulk inserts.
143-
The files are in the following format:
144-
```
145-
blocks.csv
146-
block_hash ; height ; version ; blocksize ; hashPrev ; hashMerkleRoot ; nTime ; nBits ; nNonce
147-
```
148-
```
149-
transactions.csv
150-
txid ; hashBlock ; version ; lockTime
151-
```
152-
```
153-
tx_in.csv
154-
txid ; hashPrevOut ; indexPrevOut ; scriptSig ; sequence
155-
```
156-
```
157-
tx_out.csv
158-
txid ; indexOut ; height ; value ; scriptPubKey ; address
159-
```
160-
If unclear what some of these fields are, see the [block](https://en.bitcoin.it/wiki/Protocol_documentation#block) and [transaction](https://en.bitcoin.it/wiki/Protocol_documentation#tx) specifications.
161-
162-
163-
* `simplestats`: prints some blockchain statistics like block count, transaction count, avg transactions per block, largest transaction, transaction types etc.
164-
165-
You can also define custom callbacks. A callback gets called at startup, on each block and at the end. See [src/callbacks/mod.rs](src/callbacks/mod.rs) for more information.
166-
167-
16879
## Contributing
16980

17081
Use the issue tracker to report problems, suggestions and questions. You may also contribute by submitting pull requests.

src/callbacks/balances.rs

Lines changed: 0 additions & 91 deletions
This file was deleted.

src/callbacks/mod.rs

Lines changed: 0 additions & 97 deletions
This file was deleted.

src/callbacks/opreturn.rs

Lines changed: 0 additions & 46 deletions
This file was deleted.

0 commit comments

Comments
 (0)