|
| 1 | +--- |
| 2 | +title: "Quickstart on EVM" |
| 3 | +description: "This page will guide you through the steps to get your first selective indexer up and running in a few minutes without getting too deep into the details." |
| 4 | +navigation.icon: "stars" |
| 5 | +network: "ethereum" |
| 6 | +--- |
| 7 | + |
| 8 | +# Quickstart |
| 9 | + |
| 10 | +This page will guide you through the steps to get your first selective indexer up and running in a few minutes without getting too deep into the details. |
| 11 | + |
| 12 | +Let's create an indexer for the [USDt token contract](https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7). Our goal is to save all token transfers to the database and then calculate some statistics of its holders' activity. |
| 13 | + |
| 14 | +## Install DipDup |
| 15 | + |
| 16 | +A modern Linux/macOS distribution with Python 3.11 installed is required to run DipDup. |
| 17 | + |
| 18 | +We have a [convenient installer](https://dipdup.io/install.py) that will install DipDup and all the required dependencies for you. It uses pipx to install DipDup as a CLI application and make it available everywhere for the current user. Run the following command in your terminal: |
| 19 | + |
| 20 | +```shell [Terminal] |
| 21 | +curl -Lsf https://dipdup.io/install.py | python3 |
| 22 | +``` |
| 23 | + |
| 24 | +If you don't want to use our script, install DipDup manually using commands from the snippet below. You can use any Python package manager you like, but we recommend [PDM](https://pdm.fming.dev/latest/) and provide some useful scripts for it. Here are some spells to get you started: |
| 25 | + |
| 26 | +```shell [Terminal] |
| 27 | +# pipx |
| 28 | +pipx install dipdup datamodel-code-generator |
| 29 | + |
| 30 | +# PDM |
| 31 | +pdm init --python 3.11 --lib # use "">=3.11,<3.12" for requires-python |
| 32 | +pdm venv create |
| 33 | +pdm add "dipdup>=7.0.0rc4,<8" --venv |
| 34 | +$(pdm venv activate) |
| 35 | + |
| 36 | +# Poetry |
| 37 | +poetry init --python ">=3.11,<3.12" |
| 38 | +poetry add "dipdup>=7.0.0rc4,<8" |
| 39 | +poetry shell |
| 40 | + |
| 41 | +# pip + venv |
| 42 | +python -m venv .venv |
| 43 | +. .venv/bin/activate |
| 44 | +echo "dipdup>=7.0.0rc4,<8" > requirements.txt |
| 45 | +pip install -r requirements.txt -e . |
| 46 | +``` |
| 47 | + |
| 48 | +## Create a project |
| 49 | + |
| 50 | +DipDup CLI has a built-in project generator. Run the following command in your terminal: |
| 51 | + |
| 52 | +```shell [Terminal] |
| 53 | +dipdup new |
| 54 | +``` |
| 55 | + |
| 56 | +For educational purposes, we'll create a project from scratch, so choose `[none]` network and `demo_blank` template. |
| 57 | + |
| 58 | +::banner{type="note"} |
| 59 | +Want to skip a tutorial? Choose `EVM-compatible` and `demo_evm_events` instead! |
| 60 | +:: |
| 61 | + |
| 62 | +Follow the instructions; the project will be created in the new directory. |
| 63 | + |
| 64 | +## Write a configuration file |
| 65 | + |
| 66 | +In the project root, you'll find a file named `dipdup.yaml`. It's the main configuration file of your indexer. We will discuss it in detail in the [Config](1.getting_started/3.config.md) section; for now just replace its content with the following: |
| 67 | + |
| 68 | +```yaml [dipdup.yaml] |
| 69 | +{{ #include ../src/demo_evm_events/dipdup.yaml }} |
| 70 | +``` |
| 71 | + |
| 72 | +## Generate types and stubs |
| 73 | + |
| 74 | +Now it's time to generate typeclasses and callback stubs based on definitions from config. Examples below use `demo_evm_events` as a package name; yours may differ. |
| 75 | + |
| 76 | +Run the following command: |
| 77 | + |
| 78 | +```shell [Terminal] |
| 79 | +dipdup init |
| 80 | +``` |
| 81 | + |
| 82 | +DipDup will create a Python package `demo_evm_events` with everything you need to start writing your indexer. Use `package tree` command to see the generated structure: |
| 83 | + |
| 84 | +```shell [Terminal] |
| 85 | +$ dipdup package tree |
| 86 | +demo_evm_events [.] |
| 87 | +├── abi |
| 88 | +│ ├── eth_usdt/abi.json |
| 89 | +│ └── eth_usdt/events.json |
| 90 | +├── configs |
| 91 | +│ ├── dipdup.compose.yaml |
| 92 | +│ ├── dipdup.sqlite.yaml |
| 93 | +│ ├── dipdup.swarm.yaml |
| 94 | +│ └── replay.yaml |
| 95 | +├── deploy |
| 96 | +│ ├── .env.default |
| 97 | +│ ├── Dockerfile |
| 98 | +│ ├── compose.sqlite.yaml |
| 99 | +│ ├── compose.swarm.yaml |
| 100 | +│ ├── compose.yaml |
| 101 | +│ ├── sqlite.env.default |
| 102 | +│ └── swarm.env.default |
| 103 | +├── graphql |
| 104 | +├── handlers |
| 105 | +│ └── on_transfer.py |
| 106 | +├── hasura |
| 107 | +├── hooks |
| 108 | +│ ├── on_index_rollback.py |
| 109 | +│ ├── on_reindex.py |
| 110 | +│ ├── on_restart.py |
| 111 | +│ └── on_synchronized.py |
| 112 | +├── models |
| 113 | +│ └── __init__.py |
| 114 | +├── sql |
| 115 | +├── types |
| 116 | +│ └── eth_usdt/evm_events/transfer.py |
| 117 | +└── py.typed |
| 118 | +``` |
| 119 | + |
| 120 | +That's a lot of files and directories! But don't worry, we will need only `models` and `handlers` sections in this guide. |
| 121 | + |
| 122 | +## Define data models |
| 123 | + |
| 124 | +DipDup supports storing data in SQLite, PostgreSQL and TimescaleDB databases. We use custom ORM based on Tortoise ORM as an abstraction layer. |
| 125 | + |
| 126 | +First, you need to define a model class. Our schema will consist of a single model `Holder` with the following fields: |
| 127 | + |
| 128 | +| | | |
| 129 | +| ----------- | ----------------------------------- | |
| 130 | +| `address` | account address | |
| 131 | +| `balance` | token amount held by the account | |
| 132 | +| `turnover` | total amount of transfer/mint calls | |
| 133 | +| `tx_count` | number of transfers/mints | |
| 134 | +| `last_seen` | time of the last transfer/mint | |
| 135 | + |
| 136 | +Here's how to implement this model in DipDup: |
| 137 | + |
| 138 | +```python [models/__init__.py] |
| 139 | +{{ #include ../src/demo_evm_events/models/__init__.py }} |
| 140 | +``` |
| 141 | + |
| 142 | +## Implement handlers |
| 143 | + |
| 144 | +Everything's ready to implement an actual indexer logic. |
| 145 | + |
| 146 | +Our task is to index all the balance updates. Put some code to the `on_transfer` handler callback to process matched logs: |
| 147 | + |
| 148 | +```python [handlers/on_transfer.py] |
| 149 | +{{ #include ../src/demo_evm_events/handlers/on_transfer.py }} |
| 150 | +``` |
| 151 | + |
| 152 | +And that's all! We can run the indexer now. |
| 153 | + |
| 154 | +## Next steps |
| 155 | + |
| 156 | +Run the indexer in-memory: |
| 157 | + |
| 158 | +```bash |
| 159 | +dipdup run |
| 160 | +``` |
| 161 | + |
| 162 | +Store data in SQLite database: |
| 163 | + |
| 164 | +```bash |
| 165 | +dipdup -c . -c configs/dipdup.sqlite.yaml run |
| 166 | +``` |
| 167 | + |
| 168 | +Or spawn a docker-compose stack: |
| 169 | + |
| 170 | +```bash |
| 171 | +cd deploy |
| 172 | +cp .env.default .env |
| 173 | +# Edit .env before running |
| 174 | +docker-compose up |
| 175 | +``` |
| 176 | + |
| 177 | +DipDup will fetch all the historical data and then switch to realtime updates. You can check the progress in the logs. |
| 178 | + |
| 179 | +If you use SQLite, run a query to check the data: |
| 180 | + |
| 181 | +```bash |
| 182 | +sqlite3 demo_evm_events.sqlite 'SELECT * FROM holder LIMIT 10' |
| 183 | +``` |
| 184 | + |
| 185 | +If you run a Compose stack, check open `http://127.0.0.1:8080` in your browser to see the Hasura console (exposed port may differ). You can use it to explore the database and build GraphQL queries. |
| 186 | + |
| 187 | +Congratulations! You've just created your first DipDup indexer. Proceed to the Getting Started section to learn more about DipDup configuration and features. |
0 commit comments