6.0 docs final touches (#441)

Author: droserasprout

CHANGELOG.md

@@ -507,7 +507,7 @@ This release contains no changes except for the version number.
 
 ## [4.0.0-rc2] - 2021-12-11
 
-### ⚠ Migration
+### Migration
 
 - Run `dipdup init` command to generate `on_synchronized` hook stubs.
 
@@ -529,7 +529,7 @@ This release contains no changes except for the version number.
 
 ## [4.0.0-rc1] - 2021-12-02
 
-### ⚠ Migration
+### Migration
 
 - Run `dipdup schema approve` command on every database you want to use with 4.0.0-rc1. Running `dipdup migrate` is not necessary since `spec_version` hasn't changed in this release.
 

CONTRIBUTING.md

@@ -9,11 +9,18 @@
 * To set up the development environment, you need to install [Poetry](https://python-poetry.org/docs/#installation) package manager and GNU Make.
 * Run `make help` to get available commands.
 
+## Git
+
+* Branch names MUST follow `prefix/short-descriptions` format. Prefixes we currently use: `feat` for features, `fix` for bugfixes, `docs` for documentation, `aux` for miscellaneous, `exp` for experiments.
+
 ## Codestyle
 
+* We use the following combo of linters and formatters: `isort`, `black`, `flake8`, `mypy`. All linter checks MUST pass before merging code to master.
+
 ## Releases
 
 * Release versions SHOULD conform to [Semantic Versioning](https://semver.org/). Releases that introduce breaking changes MUST be major ones.
+* Only the latest major version is supported in general. Critical fixes CAN be backported to the previous major release. To do so, git checkout from the latest stable tag, bump DipDup version manually and add a new tag.
 
 ## Documentation
 

README.md

@@ -18,7 +18,7 @@
              /_/                  /_/      
 ```
 
-DipDup is a Python framework for building indexers of [Tezos](https://tezos.com/) smart-contracts. It helps developers focus on the business logic instead of writing data storing and serving boilerplate. DipDup-based indexers are selective, which means only required data is requested. This approach allows to achieve faster indexing times and decreased load on APIs DipDup uses.
+DipDup is a Python framework for building indexers of [Tezos](https://tezos.com/) smart contracts. It helps developers focus on the business logic instead of writing data storing and serving boilerplate. DipDup-based indexers are selective, which means only required data is requested. This approach allows to achieve faster indexing times and decreased load on APIs DipDup uses.
 
 * **Ready to build your first indexer?** Head to [Quickstart](https://dipdup.net/docs/quickstart).
 
@@ -29,4 +29,5 @@ DipDup is a Python framework for building indexers of [Tezos](https://tezos.com/
 * **Have a question?** Contact us on [Discord](https://discord.com/invite/RcPGSdcVSx), [Telegram](https://t.me/baking_bad_chat), or [Slack](https://tezos-dev.slack.com/archives/CV5NX7F2L)!
 
 This project is maintained by the [Baking Bad](https://baking-bad.org/) team.
+<br>
 Development is supported by [Tezos Foundation](https://tezos.foundation/).

docs/Makefile

@@ -23,7 +23,7 @@ docs:
 	poetry run make html
 	tail -n '+34' _build/html/cli-reference.html | head -n '-63' > cli-reference.md
 	tail -n '+34' _build/html/config-reference.html | head -n '-63' > config/reference.md
-	tail -n '+34' _build/html/context-reference.html | head -n '-63' > advanced/context/reference.md
+	tail -n '+34' _build/html/context-reference.html | head -n '-63' > include/context-reference.md
 	mdbook build  --dest-dir=dist/docs
 
 homepage:
@@ -46,6 +46,6 @@ orphans:
 	cat SUMMARY.md | grep ".md" > mentioned.tmp
 	sed -i -E 's@^.*\[(.*)\]\((.*)\).*$$@\2@p' mentioned.tmp
 	cat mentioned.tmp | uniq | sort -o mentioned.tmp
-	find . | grep ".md" | grep -v -e "node_modules/" -e " book/" -e "placeholders.md" -e "SUMMARY.md" -e "mdbook-summary" | cut -c "3-" | sort -o existing.tmp
+	find . | grep ".md" | grep -v -e "node_modules/" -e " book/" -e "placeholders.md" -e "SUMMARY.md" -e "mdbook-summary" -e "include" | cut -c "3-" | sort -o existing.tmp
 	diff --color mentioned.tmp existing.tmp
 	rm mentioned.tmp existing.tmp

docs/SUMMARY.md

@@ -22,7 +22,7 @@
   * [Feature flags](advanced/feature-flags.md)
   * [SQL scripts](advanced/sql.md)
   * [Improving performance](advanced/performance.md)
-  * [Callback context (ctx)](advanced/context/README.md)
+  * [Callback context (ctx)](advanced/context.md)
   * [Internal models](advanced/internal-models.md)
   * [Index factories](advanced/index-factories.md)
   * [Metadata interface](advanced/metadata-interface.md)
@@ -81,7 +81,7 @@
 
 * [Changelog](CHANGELOG.md)
 * [Release notes](release-notes/README.md)
-  * [6.0.0 (unreleased)](release-notes/6.0.0.md)
+  * [6.0.0](release-notes/6.0.0.md)
   * [5.1.0](release-notes/5.1.0.md)
   * [5.0.0](release-notes/5.0.0.md)
   * [4.2.0](release-notes/4.2.0.md)

docs/advanced/context/README.md renamed to docs/advanced/context.md

@@ -4,4 +4,4 @@ An instance of the `HandlerContext` class is passed to every handler providing a
 
 ## Reference
 
-{{ #include reference.md}}
+{{ #include ../include/context-reference.md}}

docs/advanced/datasources.md

@@ -38,8 +38,7 @@ datasources:
 
 The rest HTTP tunables are the same as for other datasources.
 
-<!-- FIXME: Not needed since we have DB rollbacks? -->
-Also, you can wait for several block confirmations before processing the operations, e.g. to mitigate chain reorgs:
+Also, you can wait for several block confirmations before processing the operations:
 
 ```yaml
 datasources:
@@ -48,6 +47,8 @@ datasources:
     buffer_size: 1  # indexing with a single block lag
 ```
 
+Since 6.0 chain reorgs are processed automatically, but you may find this feature useful for other cases.
+
 ## Tezos node
 
 Tezos RPC is a standard interface provided by the Tezos node. This datasource is used solely by `mempool` and `metadata` standalone services; you can't use it in regular DipDup indexes.
@@ -83,7 +84,12 @@ datasources:
     network: mainnet | ithacanet
 ```
 
-<!-- FIXME: No usage example -->
+Then, in your hook or handler code:
+
+```python
+datasource = ctx.get_metadata_datasource('metadata')
+token_metadata = await datasource.get_token_metadata('KT1...', '0')
+```
 
 ## IPFS
 

docs/advanced/event-hooks.md

@@ -16,13 +16,12 @@ This hook fires when every active index reaches a realtime state. Here you can c
 
 ## `on_index_rollback`
 
-Fires when TzKT datasource has received a chain reorg message which can't be processed automatically.
+Fires when TzKT datasource has received a chain reorg message which can't be processed by dropping buffered messages (`buffer_size` option).
 
-<!-- FIXME -->
-If your indexer is stateless, you can just drop DB data saved after `to_level` and continue indexing. Otherwise, implement more complex logic. By default, this hook triggers full reindexing.
+Since version 6.0 this hook performs a database-level rollback by default. If it doesn't work for you for some reason remove `ctx.rollback` call and implement your own rollback logic.
 
 > 💡 **SEE ALSO**
 >
 > * {{ #summary advanced/reindexing.md}}
 > * {{ #summary advanced/sql.md}}
-> * {{ #summary advanced/context/README.md}}
+> * {{ #summary advanced/context.md}}

docs/advanced/feature-flags.md

@@ -11,6 +11,7 @@ advanced:
   postpone_jobs: False
   metadata_interface: False
   skip_version_check: False
+  crash_reporting: False
 ```
 
 ## Early realtime
@@ -35,8 +36,12 @@ Do not start the job scheduler until all indexes are synchronized. If your jobs
 
 ## Metadata interface
 
-Without this flag calling `ctx.update_contract_metadata` and `ctx.update_token_metadata` will make no effect. Corresponding internal tables are created on reindex in any way.
+Without this flag calling `ctx.update_contract_metadata` and `ctx.update_token_metadata` will make no effect. Corresponding internal tables are created on reindexing in any way.
 
 ## Skip version check
 
 Disables warning about running unstable or out-of-date DipDup version.
+
+## Crash reporting
+
+Enables sending crash reports to the Baking Bad team. This is **disabled by default**. You can inspect crash dumps saved as `/tmp/dipdup-tombstone_XXXXXXX.json` before enabling this option.

docs/advanced/index-factories.md

@@ -28,5 +28,5 @@ Another solution is to implement custom logic in `on_restart` hook (see {{ #summ
 
 > 💡 **SEE ALSO**
 >
-> * {{ #summary advanced/context/README.md}}
+> * {{ #summary advanced/context.md}}
 > * {{ #summary config/templates.md}}