Hooks, typed callbacks, more builtin tables, no more tempstates (#119)

* Keep more details about index state

* WIP

* Hooks WIP

* WIP

* WIP

* callback codegen WIP

* Finish callback codegen

* Update demos

* Cleanup

* Better template substitution

* CallbackManager, rewrite config initialization

* Rewrite config module

* Refactor codegen mixin, add locate

* Finish and integrate CallbackManager

* Codegen fixes

* Fix

* Dipdup models state wip

* `run` refactoring

* Create context in callback manager

* DB states wip

* State initialization

* Fix head updating, state

* No more temporary states!

* Bump mypy

* Lint

* Split execute_sql

* Fix codegen, revert SQL execution logic, 20% cooler exceptions

* Fix rollback

* Bump hasura, skip warning for missing default hooks

* Fix graceful shutdown 🎉🎉🎉

* Fix PArentMixin generic

* DatabaseConfigurationError, refactoring

* Docs

* dipdup.config refactoring

* Spawn indexes early

* useless initialize

* Resolve missing table names before Hasura does

* Another jobs fix

* Fix saving index states

* Hook fixes

* Less logs

* Migrations

* Migrate demos

* Cleanup

* Finish migrations :(

* Logging regression

* pass reindex reason

* Fix spawning indexes, refactor formattedlogger creation

* Fix SQL hooks

* Failed jobs raise exception in main loop

* Bump

* Bump

* Bump

* Force public schema

Author: droserasprout

poetry.lock

@@ -47,12 +47,14 @@ black = "^20.8b1"
 bump2version = "^1.0.1"
 diff-cover = "^5.0.1"
 isort = "^5.7.0"
-mypy = "^0.812"
+mypy = "^0.900"
 pytest = "^3.0"
 pytest-cov = "^2.4"
 flake8 = "3.9.0"
 flakehell = "^0.9.0"
 testcontainers = "^3.4.1"
+types-tabulate = "^0.8.2"
+types-pytz = "^2021.1.2"
 
 [tool.poetry.extras]
 pytezos = ["pytezos"]

pyproject.toml

@@ -5,7 +5,6 @@ database:
   user: ${POSTGRES_USER:-dipdup}
   password: ${POSTGRES_PASSWORD:-changeme}
   database: ${POSTGRES_DB:-dipdup}
-  schema_name: hic_et_nunc
 
 hasura:
   url: http://hasura:8080

src/demo_hic_et_nunc/dipdup-docker.yml

@@ -5,7 +5,6 @@ database:
   user: ${POSTGRES_USER:-dipdup}
   password: ${POSTGRES_PASSWORD:-changeme}
   database: ${POSTGRES_DB:-dipdup}
-  schema_name: hic_et_nunc
 
 hasura:
   url: http://127.0.0.1:8080

src/demo_hic_et_nunc/dipdup-local.yml

@@ -1,4 +1,4 @@
-spec_version: 1.1
+spec_version: 1.2
 package: demo_hic_et_nunc
 
 database:

src/demo_hic_et_nunc/dipdup.yml

@@ -30,7 +30,7 @@ services:
       retries: 5
 
   hasura:
-    image: hasura/graphql-engine:v2.0.4
+    image: hasura/graphql-engine:v2.0.8
     ports:
       - 127.0.0.1:8080:8080
     depends_on:

src/demo_hic_et_nunc/docker/docker-compose.yml

@@ -0,0 +1,7 @@
+from dipdup.context import HookContext
+
+
+async def on_reindex(
+    ctx: HookContext,
+) -> None:
+    await ctx.execute_sql('on_reindex')