You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Python SDK for developing indexers of [Tezos](https://tezos.com/) smart contracts inspired by [The Graph](https://thegraph.com/).
10
9
11
-
## Installation
10
+
## Quickstart
12
11
13
-
Python 3.8+ is required for dipdup to run.
12
+
Python 3.9+ is required for dipdup to run.
14
13
15
14
```shell
16
15
$ pip install dipdup
17
16
```
18
17
19
-
## Creating indexer
20
-
21
-
If you want to see dipdup in action before diving into details you can run a demo project and use it as reference. Clone this repo and run the following command in it's root directory:
22
-
23
-
```shell
24
-
$ dipdup -c src/demo_hic_et_nunc/dipdup.yml run
25
-
```
26
-
27
-
Examples in this guide are simplified Hic Et Nunc demo.
28
-
29
-
### Write configuration file
30
-
31
-
Create a new YAML file and adapt the following example to your needs:
Each handler in index config matches an operation group based on operations' entrypoints and destination addresses in pattern. Matched operation groups will be passed to handlers you define.
72
-
73
-
### Initialize project structure
74
-
75
-
Run the following command replacing `config.yml` with path to YAML file you just created:
76
-
77
-
```shell
78
-
$ dipdup -c config.yml init
79
-
```
80
-
81
-
This command will create a new package with the following structure (some lines were omitted for readability):
82
-
83
-
```
84
-
demo_hic_et_nunc/
85
-
├── handlers
86
-
│ ├── on_mint.py
87
-
│ └── on_rollback.py
88
-
├── hasura-metadata.json
89
-
├── models.py
90
-
└── types
91
-
├── hen_minter
92
-
│ ├── storage.py
93
-
│ └── parameter
94
-
│ └── mint_OBJKT.py
95
-
└── hen_objkts
96
-
├── storage.py
97
-
└── parameter
98
-
└── mint.py
99
-
```
100
-
101
-
`types`directory is Pydantic dataclasses of contract storage and parameter. This directory is autogenerated, you shouldn't modify any files in it. `models` and `handlers` modules are placeholders for your future code and will be discussed later.
102
-
103
-
You could invoke `init` command on existing project (must be in your `PYTHONPATH`. Do it each time you update contract addresses or models. Code you've wrote won't be overwritten.
104
-
105
-
### Define models
106
-
107
-
Dipdup uses [Tortoise](https://tortoise-orm.readthedocs.io/en/latest/) under the hood, fast asynchronous ORM supporting all major database engines. Check out [examples](https://tortoise-orm.readthedocs.io/en/latest/examples.html) to learn how to use is.
108
-
109
-
Now open `models.py` file in your project and define some models:
Now take a look at `handlers` module generated by `init` command. When operation group matching `pattern` block of corresponding handler at config will arrive callback will be fired. This example will simply save minted Hic Et Nunc tokens and their owners to the database:
129
-
130
-
```python
131
-
import demo_hic_et_nunc.models as models
132
-
from demo_hic_et_nunc.types.hen_minter.parameter.mint_objkt import MintOBJKTParameter
133
-
from demo_hic_et_nunc.types.hen_minter.storage import HenMinterStorage
134
-
from demo_hic_et_nunc.types.hen_objkts.parameter.mint import MintParameter
135
-
from demo_hic_et_nunc.types.hen_objkts.storage import HenObjktsStorage
136
-
from dipdup.models import TransactionContext, OperationHandlerContext
Handler name `on_rollback` is reserved by dipdup, this special handler will be discussed later.
156
-
157
-
### Atomicity and persistency
158
-
159
-
Here's a few important things to know before running your indexer:
160
-
161
-
* __WARNING!__ Make sure that database you're connecting to is used by dipdup exclusively. When index configuration or models change the whole database will be dropped and indexing will start from scratch.
162
-
* Do not rename existing indexes in config file without cleaning up database first, didpup won't handle this renaming automatically and will consider renamed index as a new one.
163
-
* Multiple indexes pointing to different contracts must not reuse the same models because synchronization is performed by index first and then by block.
164
-
* Reorg messages signal about chain reorganizations, when some blocks, including all operations, are rolled back in favor of blocks with higher weight. Chain reorgs happen quite often, so it's not something you can ignore. You have to handle such messages correctly, otherwise you will likely accumulate duplicate data or, worse, invalid data. By default Dipdup will start indexing from scratch on such messages. To implement your own rollback logic edit generated `on_rollback` handler.
165
-
166
-
### Run your dapp
167
-
168
-
Now everything is ready to run your indexer:
169
-
170
-
```shell
171
-
$ dipdup -c config.yml run
172
-
```
173
-
174
-
Parameters wrapped with `${VARIABLE:-default_value}` in config could be set from corresponding environment variables. For example if you want to use another TzKT instance:
175
-
176
-
```shell
177
-
$ TZKT_URL=https://api.tzkt.io dipdup -c config.yml run
178
-
```
179
-
180
-
You can interrupt indexing at any moment, it will start from last processed block next time you run your app again.
181
-
182
-
Use `docker-compose.yml` included in this repo if you prefer to run dipdup in Docker:
183
-
184
-
```shell
185
-
$ docker-compose build
186
-
$ # example target, edit volumes section to change dipdup config
187
-
$ docker-compose up hic_et_nunc
188
-
```
189
-
190
-
For debugging purposes you can index specific block range only and skip realtime indexing. To do this set `first_block` and `last_block` fields in index config.
191
-
192
-
### Index templates
193
-
194
-
Sometimes you need to run multiple indexes with similar configs whose only difference is contract addresses. In this case you can use index templates like this:
195
-
196
-
```yaml
197
-
templates:
198
-
trades:
199
-
kind: operation
200
-
datasource: tzkt_staging
201
-
contracts:
202
-
- <dex>
203
-
handlers:
204
-
- callback: on_fa12_token_to_tez
205
-
pattern:
206
-
- type: transaction
207
-
destination: <dex>
208
-
entrypoint: tokenToTezPayment
209
-
- type: transaction
210
-
destination: <token>
211
-
entrypoint: transfer
212
-
- callback: on_fa20_tez_to_token
213
-
pattern:
214
-
- type: transaction
215
-
destination: <dex>
216
-
entrypoint: tezToTokenPayment
217
-
- type: transaction
218
-
destination: <token>
219
-
entrypoint: transfer
220
-
221
-
indexes:
222
-
trades_fa12:
223
-
template: trades
224
-
values:
225
-
dex: FA12_dex
226
-
token: FA12_token
227
-
228
-
trades_fa20:
229
-
template: trades
230
-
values:
231
-
dex: FA20_dex
232
-
token: FA20_token
233
-
```
234
-
235
-
Template values mapping could be accessed from within handlers at `ctx.template_values`.
236
-
237
-
### Optional: configure Hasura GraphQL Engine
238
-
239
-
When using PostgreSQL as a storage solution you can use Hasura integration to get GraphQL API out-of-the-box. Add the following section to your config, Hasura will be configured automatically when you run your indexer.
240
-
241
-
```yaml
242
-
hasura:
243
-
url: http://hasura:8080
244
-
admin_secret: changeme
245
-
```
246
-
247
-
When using included docker-compose example make sure you run Hasura first:
248
-
249
-
```shell
250
-
$ docker-compose up -d hasura
251
-
```
252
-
253
-
Then run your indexer and navigate to `127.0.0.1:8080`.
254
-
255
-
### Optional: configure logging
256
-
257
-
You may want to tune logging to get notifications on errors or enable debug messages. Specify path to Python logging config in YAML format at `--logging-config` argument. Default config to start with:
258
-
259
-
```yml
260
-
version: 1
261
-
disable_existing_loggers: false
262
-
formatters:
263
-
brief:
264
-
format: "%(levelname)-8s %(name)-35s %(message)s"
265
-
handlers:
266
-
console:
267
-
level: INFO
268
-
formatter: brief
269
-
class: logging.StreamHandler
270
-
stream : ext://sys.stdout
271
-
loggers:
272
-
SignalRCoreClient:
273
-
formatter: brief
274
-
dipdup.datasources.tzkt.datasource:
275
-
level: INFO
276
-
dipdup.datasources.tzkt.cache:
277
-
level: INFO
278
-
root:
279
-
level: INFO
280
-
handlers:
281
-
- console
282
-
```
18
+
* Read the rest of the tutorial: [docs.dipdup.net](https://docs.dipdup.net/)
19
+
* Check out [demo projects](https://github.com/dipdup-net/dipdup-py/tree/master/src)
0 commit comments