Kathmandu events support (#497)

Author: droserasprout

.github/FUNDING.yml

@@ -0,0 +1 @@
+github: dipdup-net

.github/workflows/docs.yml

@@ -6,6 +6,8 @@ on:
       - '*.*.*'
     branches:
       - master
+      # FIXME
+      - feat/events
     paths:
       - 'docs/**'
       - 'scripts/**'

.github/workflows/release.yml

@@ -70,7 +70,7 @@ jobs:
       - name: Set up Poetry
         uses: snok/install-poetry@v1
         with:
-          version: '1.2.1'
+          version: '1.2.2'
 
       - name: Install project
         run: make install

CHANGELOG.md

@@ -2,31 +2,47 @@
 
 All notable changes to this project will be documented in this file.
 
-The format is based on [Keep a Changelog],
-and this project adheres to [Semantic Versioning].
+The format is based on [Keep a Changelog], and this project adheres to [Semantic Versioning].
 
-## [Unreleased]
+## [6.2.0] - 2022-10-12
 
 ### Added
 
-- install: New install script based on pipx.
-- cli: `dipdup new` command to create a new project (ex. cookiecutter).
-- cli: `dipdup update` command to update local pipx and poetry installations.
+- cli: `new` command to create a new project interactively.
+- cli: `install/update/uninstall` commands to manage local DipDup installation.
+- index: New index kind `event` to process contract events.
+- install: New interactive installer based on pipx (`install.py` or `dipdup-install`).
 
 ### Fixed
 
-- cli: Commands that only print help pages no longer require a valid config.
-- codegen: Fail lately when datamodel-codegen is not available.
-- config: Allow `dsn` field to be empty.
+- cli: Fixed commands that don't require a valid config yet crash with `ConfigurationError`.
+- codegen: Fail on demand when `datamodel-codegen` is not available.
+- codegen: Fixed Jinja2 template caching.
+- config: Allow `sentry.dsn` field to be empty.
+- config: Fixed greedy environment variable regex.
 - hooks: Raise a `FeatureAvailabilityHook` instead of a warning when trying to execute hooks on SQLite.
 
-### Removed
+### Improved
 
-- install: Cookiecutter template is no longer supported.
+- cli: Detect `src/` layout when guessing package path.
+- codegen: Improved cross-platform compatibility.
+- config: `sentry.user_id` option to set user ID for Sentry (affects release adoption data).
+- sentry: Detect environment when not set in config (docker/gha/tests/local)
+- sentry: Expose more tags under the `dipdup` namespace.
 
 ### Performance
 
-- cli: Up to 5x decrease in startup time.
+- cli: Up to 5x faster startup for some commands.
+
+### Security
+
+- sentry: Prevent Sentry from leaking hostname if `server_name` is not set.
+- sentry: Notify about using Sentry when DSN is set or crash reporting is enabled.
+
+### Other
+
+- ci: A significantly faster execution of GitHub Actions.
+- docs: Updated "Contributing Guide" page.
 
 ## [6.1.3] - 2022-09-21
 

CONTRIBUTING.md

@@ -1,28 +1,52 @@
 # DipDup contribution guide
 
-> 🚧 **UNDER CONSTRUCTION**
->
-> This page or paragraph is yet to be written. Come back later.
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
 
-## Preparation
+## General
 
-* To set up the development environment, you need to install [Poetry](https://python-poetry.org/docs/#installation) package manager and GNU Make.
-* Run `make help` to get available commands.
+* All code in this repository MUST be licensed under the [MIT License](LICENSE).
+* Python code in this repository MUST run on Python 3.10. It also SHOULD run on Python 3.11. Using modern language features is encouraged.
+* Python code in this repository MUST run in Linux, macOS, Docker, and `amd64`/`arm64` environments. Windows SHOULD be supported as well.
+* We use the [Poetry](https://python-poetry.org/docs/#installation) package manager and GNU Make to set up the development environment. You SHOULD install both tools and run `make help` to see available shortcuts.
+* Developers SHOULD have fun while contributing to the project.
 
-## Git
+## GitHub
 
-* Branch names MUST follow `prefix/short-descriptions` format. Prefixes we currently use: `feat` for features, `fix` for bugfixes, `docs` for documentation, `aux` for miscellaneous, `exp` for experiments.
+* Branch names MUST follow `prefix/short-description` format. Prefixes currently in use: `feat` for features, `fix` for bugfixes, `docs` for documentation, `aux` for miscellaneous, `exp` for experiments.
+* Commits in pull requests MUST be squashed when merging to `master`.
+* Issues and pull requests MUST have a descriptive title; they SHOULD be linked to each other, appropriately labeled, and assigned to maintainers while in progress.
 
 ## Codestyle
 
-* We use the following combo of linters and formatters: `isort`, `black`, `flake8`, `mypy`. All linter checks MUST pass before merging code to master.
+* We use the following combo of linters and formatters: `isort`, `black`, `flake8`, `mypy`. All linter checks MUST pass before merging code to `master` (CI will fail otherwise).
+* Single quotes are RECOMMENDED for string literals.
+* Meaningful comments are highly RECOMMENDED to begin with `# NOTE:`, `# TODO:`, or `# FIXME:`.
+* f-string formatting is RECOMMENDED over other methods. Logging is an exception to this rule.
 
 ## Releases
 
-* Release versions SHOULD conform to [Semantic Versioning](https://semver.org/). Releases that introduce breaking changes MUST be major ones.
-* Only the latest major version is supported in general. Critical fixes CAN be backported to the previous major release. To do so, git checkout from the latest stable tag, bump DipDup version manually and add a new tag.
+* Release versions MUST conform to [Semantic Versioning](https://semver.org/). Releases that introduce breaking changes MUST be major ones.
+* Only the latest major version is supported in general. Critical fixes MAY be backported to the previous major release. To do so, create an `aux/X.Y.Z` branch from the latest stable tag, bump the DipDup version manually, and add a new tag.
 
 ## Documentation
 
-* All changes that affect user experience MUST be documented in CHANGELOG.md file.
-* Changelog formatting SHOULD stick to GitLab changelog [guidelines](https://docs.gitlab.com/ee/development/changelog.html).
+* All changes that affect user experience MUST be documented in the CHANGELOG.md file.
+* Changes that significantly affect DipDup maintainers' experience MAY be documented in the CHANGELOG.md file.
+* The changelog MUST conform to the "Keep a Changelog" specification (CI will break otherwise).
+* A page in Release Notes SHOULD accompany all major releases.
+* All internal links MUST be created with `{{ #summary ...` shortcodes.
+* All values used in project templates MUST be replaced with `{{ #cookiecutter ...` shortcodes.
+
+> 🚧 **UNDER CONSTRUCTION**
+>
+> This page or paragraph is yet to be written. Come back later.
+
+## Demo Projects
+
+## Installer
+
+## Docker Images
+
+## Tests
+
+## Code Review

Dockerfile

@@ -8,7 +8,7 @@ RUN <<eot
     apt update
     apt install -y gcc make git `if [[ $PYTEZOS = "1" ]]; then echo build-essential pkg-config libsodium-dev libsecp256k1-dev libgmp-dev; fi`
 
-    pip install --no-cache-dir poetry==1.2.1
+    pip install --no-cache-dir poetry==1.2.2
 
     mkdir -p /opt/dipdup
 
@@ -37,7 +37,7 @@ SHELL ["/bin/bash", "-c"]
 
 RUN <<eot
     useradd -ms /bin/bash dipdup
-    pip install --no-cache-dir poetry==1.2.1 setuptools
+    pip install --no-cache-dir poetry==1.2.2 setuptools
 
     apt update
     apt install -y --no-install-recommends git `if [[ $PYTEZOS = "1" ]]; then echo libsodium-dev libsecp256k1-dev libgmp-dev; fi`

Makefile

@@ -27,7 +27,7 @@ lint:           ## Lint with all tools
 	make isort black flake mypy
 
 test:           ## Run test suite
-	poetry run pytest --cov-report=term-missing --cov=dipdup --cov-report=xml -n auto --dist loadscope -s -v tests
+	poetry run pytest --cov-report=term-missing --cov=dipdup --cov-report=xml -n auto -s -v tests
 
 docs:           ## Build docs
 	scripts/update_cookiecutter.py

README.md

@@ -1,12 +1,13 @@
+[![GitHub stars](https://img.shields.io/github/stars/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup)
+[![Latest stable release](https://img.shields.io/github/v/release/dipdup-net/dipdup?label=stable%20release&color=2c2c2c)](https://github.com/dipdup-net/dipdup/releases)
+[![Latest pre-release)](https://img.shields.io/github/v/release/dipdup-net/dipdup?include_prereleases&label=latest%20release&color=2c2c2c)](https://github.com/dipdup-net/dipdup/releases)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dipdup?color=2c2c2c)](https://www.python.org)
-[![Latest stable release](https://img.shields.io/github/v/release/dipdup-net/dipdup?label=stable&color=2c2c2c)](https://github.com/dipdup-net/dipdup/releases)
-[![Latest pre-release)](https://img.shields.io/github/v/release/dipdup-net/dipdup?include_prereleases&label=latest&color=2c2c2c)](https://github.com/dipdup-net/dipdup/releases)
 [![License: MIT](https://img.shields.io/github/license/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup/blob/master/LICENSE)
 <br>
-[![GitHub stars](https://img.shields.io/github/stars/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup)
 [![PyPI monthly downloads](https://img.shields.io/pypi/dm/dipdup?color=2c2c2c)](https://pypi.org/project/dipdup/)
 [![GitHub issues](https://img.shields.io/github/issues/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup/issues)
 [![GitHub pull requests](https://img.shields.io/github/issues-pr/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup/pulls)
+[![GitHub Discussions](https://img.shields.io/github/discussions/dipdup-net/dipdup?color=2c2c2c)](https://github.com/dipdup-net/dipdup/discussions)
 
 ```text
         ____   _         ____              
@@ -17,13 +18,13 @@
              /_/                  /_/      
 ```
 
-DipDup is a Python framework for building indexers of [Tezos](https://tezos.com/) smart contracts. It helps developers focus on the business logic instead of writing data storing and serving boilerplate. DipDup-based indexers are selective, which means only required data is requested. This approach allows to achieve faster indexing times and decreased load on APIs DipDup uses.
+DipDup is a Python framework for building smart contract indexers. It helps developers focus on business logic instead of writing a boilerplate to store and serve data. DipDup-based indexers are selective, which means only required data is requested. This approach allows to achieve faster indexing times and decreased load on underlying APIs.
 
 * **Ready to build your first indexer?** Head to [Quickstart](https://docs.dipdup.io/quickstart).
 
 * **Looking for examples?** Check out [Demo Projects](https://docs.dipdup.io/examples/demo-projects) and [Built with DipDup](https://docs.dipdup.io/examples/built-with-dipdup) pages.
 
-* **Want to participate?** Vote for issues on [GitHub](https://github.com/dipdup-net/dipdup/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc) or [become a sponsor](https://github.com/sponsors/dipdup-net).
+* **Want to participate?** Vote for [open issues](https://github.com/dipdup-net/dipdup/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc), join [discussions](https://github.com/dipdup-net/dipdup/discussions) or [become a sponsor](https://github.com/sponsors/dipdup-net).
 
 * **Have a question?** Contact us on [Discord](https://discord.com/invite/RcPGSdcVSx), [Telegram](https://t.me/baking_bad_chat), or [Slack](https://tezos-dev.slack.com/archives/CV5NX7F2L)!
 

demos/demo-domains-big-map/.dockerignore

@@ -0,0 +1,20 @@
+# Ignore all
+*
+
+# Add build files
+!Makefile
+!pyproject.toml
+!poetry.lock
+!requirements**
+!README.md
+
+# Add code
+!src
+
+# Add configs
+!*.yml
+
+# Ignore caches
+**/.mypy_cache
+**/.pytest_cache
+**/__pycache__

demos/demo-domains-big-map/.gitignore

@@ -0,0 +1,150 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+*.ipynb
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+.vscode
+.idea
+
+*sqlite3*
+
+.noseids
+
+secrets.env
+
+*.out*