4.1.0

4.1.0 - 2022-01-24

Added

• cli: Added schema init command to initialize database schema.
• cli: Added --force flag to hasura configure command.
• codegen: Added support for subpackages inside callback directories.
• hasura: Added dipdup_head_status view and REST endpoint.
• index: Added an ability to skip historical data while synchronizing big_map indexes.
• metadata: Added metadata datasource.
• tzkt: Added get_big_map and get_contract_big_maps datasource methods.

4.0.5

4.0.5 - 2022-01-20

Fixed

• index: Fixed deserializing manually modified typeclasses.

4.0.4

4.0.4 - 2022-01-17

Added

• cli: Added --keep-schemas flag to init command to preserve JSONSchemas along with generated types.

Fixed

• demos: Tezos Domains and Homebase DAO demos were updated from edo2net to mainnet contracts.
• hasura: Fixed missing relations for models with ManyToManyField fields.
• tzkt: Fixed parsing storage with nested structures.

Performance

• dipdup: Minor overall performance improvements.

Other

• ci: Cache virtual environment in GitHub Actions.
• ci: Detect CI environment and skip tests that fail in GitHub Actions.
• ci: Execute tests in parallel with pytest-xdist when possible.
• ci: More strict linting rules of flake8.

4.0.3

4.0.3 - 2022-01-09

Fixed

• tzkt: Fixed parsing parameter with an optional value.

4.0.2

4.0.2 - 2022-01-06

Added

• tzkt: Added optional delegate_address and delegate_alias fields to OperationData.

Fixed

• tzkt: Fixed crash due to unprocessed pysignalr exception.
• tzkt: Fixed parsing OperationData.amount field.
• tzkt: Fixed parsing storage with top-level boolean fields.

4.0.1

4.0.1 - 2021-12-30

Fixed

• codegen: Fixed generating storage typeclasses with Union fields.
• codegen: Fixed preprocessing contract JSONSchema.
• index: Fixed processing reindexing reason saved in the database.
• tzkt: Fixed processing operations with default entrypoint and empty parameter.
• tzkt: Fixed crash while recursively applying bigmap diffs to the storage.

Performance

• tzkt: Increased speed of applying bigmap diffs to operation storage.

4.0.0

4.0.0 - 2021-12-24

This release contains no changes except for the version number.

4.0.0-rc3

4.0.0-rc3 - 2021-12-20

Fixed

• cli: Fixed missing schema approve --hashes argument.
• codegen: Fixed contract address used instead of an alias when typename is not set.
• tzkt: Fixed processing operations with entrypoint default.
• tzkt: Fixed regression in processing migration originations.
• tzkt: Fixed filtering of big map diffs by the path.

Removed

• cli: Removed deprecated run --oneshot argument and clear-cache command.

4.0.0-rc2

4.0.0-rc2 - 2021-12-11

⚠ Migration

• Run dipdup init command to generate on_synchronized hook stubs.

Added

• hooks: Added on_synchronized hook, which fires each time all indexes reach realtime state.

Fixed

• cli: Fixed config not being verified when invoking some commands.
• codegen: Fixed generating callback arguments for untyped operations.
• index: Fixed incorrect log messages, remove duplicate ones.
• index: Fixed crash while processing storage of some contracts.
• index: Fixed matching of untyped operations filtered by source field (@pravin-d).

Performance

• index: Checks performed on each iteration of the main DipDup loop are slightly faster now.

4.0.0-rc1

4.0.0-rc1 - 2021-12-02

⚠ Migration

• Run dipdup schema approve command on every database you want to use with 4.0.0-rc1. Running dipdup migrate is not necessary since spec_version hasn't changed in this release.

Added

• cli: Added run --early-realtime flag to establish a realtime connection before all indexes are synchronized.
• cli: Addedrun --merge-subscriptions flag to subscribe to all operations/big map diffs during realtime indexing. This flag helps to avoid reaching TzKT subscriptions limit (currently 10000 channels).
• cli: Added status command to print the current status of indexes from the database.
• cli: Added config export [--unsafe] command to print config after resolving all links and variables. Add --unsafe option to substitute environment variables.
• cli: Added cache show command to get information about file caches used by DipDup.
• config: Added first_level and last_level optional fields to TemplateIndexConfig. These limits are applied after ones from the template itself.
• config: Added daemon boolean field to JobConfig to run a single callback indefinitely. Conflicts with crontab and interval fields.
• config: Added advanced top-level section with following fields:

advanced:
  early_realtime: False
  merge_subscriptions: False
  oneshot: False
  postpone_jobs: False
  reindex:
    manual: exception
    migration: wipe
    rollback: ignore
    config_modified: exception
    schema_modified: wipe

ReindexingRequiredError exception is raised by default when reindexing is triggered. CLI flags have priority over self-titled AdvancedConfig fields.

Fixed

• cli: Fixed crashes and output inconsistency when piping DipDup commands.
• cli: Fixed schema wipe --immune flag being ignored.
• codegen: Fixed missing imports in handlers generated during init.
• coinbase: Fixed possible data inconsistency caused by caching enabled for method get_candles.
• hasura: Fixed unnecessary reconfiguration in restart.
• http: Fixed increasing sleep time between failed request attempts.
• index: Fixed invocation of head index callback.
• index: Fixed CallbackError raised instead of ReindexingRequiredError in some cases.
• tzkt: Fixed resubscribing when realtime connectivity is lost for a long time.
• tzkt: Fixed sending useless subscription requests when adding indexes in runtime.
• tzkt: Fixed get_originated_contracts and get_similar_contracts methods whose output was limited to HTTPConfig.batch_size field.
• tzkt: Fixed lots of SignalR bugs by replacing aiosignalrcore library with pysignalr.

Changed

• cli: schema wipe command now requires confirmation when invoked in the interactive shell.
• cli: schema approve command now also causes a recalculation of schema and index config hashes.
• index: DipDup will recalculate respective hashes if reindexing is triggered with config_modified: ignore or schema_modified: ignore in advanced config.

Deprecated

• cli: run --oneshot option is deprecated and will be removed in the next major release. The oneshot mode applies automatically when last_level field is set in the index config.
• cli: clear-cache command is deprecated and will be removed in the next major release. Use cache clear command instead.

Performance

• config: Configuration files are loaded 10x times faster.
• index: Number of operations processed by matcher reduced by 40%-95% depending on the number of addresses and entrypoints used.
• tzkt: Rate limit was increased. Try to set connection_timeout to a higher value if requests fail with ConnectionTimeout exception.
• tzkt: Improved performance of response deserialization.