Merged
Conversation
Inserting updates for a block that already has updates in the db should now fail: this is to avoid inserting duplicates.
Otherwise we will skip data within the current batch if it's updating values within the same block
The new versioning rules expect block-scoped inserts for dynamic/versioned data. Therefore it is important to group updates to each table into one gateway call each. If a gateway fn handles both a static value insert and a dynamic value insert this causes complications with subsequent updates applied to the dynamic values inserted. Therefore it is cleanest to maintain that each gateway fn insert/update only static OR dynamic values and not both.
This commit introduces a new `EntryPointTracer` trait for tracing smart contract entry points, along with related types such as `EntryPoint` and `TracedEntryPoint`. The `trace` method allows for analyzing execution behavior at a specific block. Additionally, the `blockchain` module has been updated to include necessary structures for this functionality.
This commit introduces new tables and types for entry point tracing in the PostgreSQL schema, including `entry_point`, `traced_entry_point`, `entry_point_tracing_data`, and `entry_point_calls_account`. It also includes the necessary joinable relationships and updates to the ORM to support these new structures.
This commit refactor the tycho-common structs and trait for entry-point. This is needed to match better with the design in postgres.
This commit refactors our Postgres design: we're now linking protocol components to `entry_point_tracing_data` instead of `entry_point`. This is to keep track of what component is linked to what tracing data (previously lost during inserts). Also refactor the schema.patch, the patch data is now packed together at the top make it more robust to conflicts on new updates
…entry point Following previous commit refactor on the db design, this commit adapts the gateway functions and common structs
This commit batch some SQL queries together to optimize, clean the code and address a todo to make JSON serialization consistent for tracing data.
This commit moves the EntryPointGateway trait into tycho-common to stay consitent with its siblings traits. This trait is now implemented on the cache gateway. Also miscellaneous code cleaning and comments
This commit renames the `get_entry_points` method to `get_entry_points_with_data` across the relevant traits and implementations, reflecting the inclusion of tracing data in the results.
In SQL, when inserting data with ON CONFLICT DO NOTHING and RETURNING, it will ignore the conflict rows, so you can't expect this to return the full set of data for each row. We address this by first inserting and then reading, it comes with a small overhead but ensure we get the complete data.
…al id. Before this commit we were using `entry_point_tracing_data_id` as primary key for this table. This works now but it's not future proof if we want to allow a one-to-many relation between the two tables (for example if we want to add some kind of versioning). This commit fixes this by adding a unique internal ID as primary key. The `entry_point_tracing_data_id` stays constrained to UNIQUE for now but this can be easily removed later if needed.
- Remove unnecessary explicity clone implementation on DirectGateway - Remove extra api_key setter in ServicesBuilder - Remove registration of empty list of extractors
using a helper function. For less maintenance.
This is needed by DCI to avoid indexing the whole contract, instead we will just index some slots that were used during the tracing (read or write)
The latest version on clippy complains about this "error: lifetime flowing from input to output with different syntax can be confusing ". This commit fixes it for CI
This used to be done automatically in the previous version because `ref_into_account` uses `self.balance.clone().unwrap_or_default()`. Then these values used to be inserted together with the new contract. Now it's done in two steps, we still need to ensure that balance is set to default if `None`. This commit fixes this.
The string was misinterpreted as both the tracing type and the rpc url. It's now interpreted as the tracing type only and the rpc_url comes from the args.
There was a bug with the `ids_by_entry_point_with_tracing_params` function that was wrongly retrieving tracing params internal ids. This commits makes it correctly retrieve the internal id attached to the given entry point
This commit fixes a bug which omitted some entrypoints if more than one was pushed by a single transaction.
In may places in our code we expect these transactions to be sorted by transaction index. DCI inserts transaction at the end of the vector, to keep it sorted we need to sort it again after inserting new transactions.
We got some cases in the testing suite where we where inserting many contracts with this. But the db transaction size became too big and it crashed. This commit address this by splitting inserts in many transactions. It makes it slightly slower but this is used for testing purposes only.
In a previous commit we refactored this logic, but during this refactor we removed the conflict handling logic. A contract can be inserted many times, we should just ignore the conflicts. This commit addresses this
After rebasing on the release branch this is needed.
…ization This is requested by Clippy.
louise-poole
force-pushed
the
release/dci
branch
from
6ef7c1f to
3fd0d50
Compare
kayibal
approved these changes
Jun 19, 2025
propellerci bot
pushed a commit
that referenced
this pull request
Jun 19, 2025
## [0.71.0](0.70.9...0.71.0) (2025-06-19) ### Features * (WIP) Add get_traced_entrypoints endpoint. ([0528c3c](0528c3c)) * (WIP) get_traced_entrypoints pagination + caching ([2af5359](2af5359)) * access control on add_entry_points ([9e9a7a4](9e9a7a4)) * Add BatchAccountExtractor with balance and code get logic ([e3b6566](e3b6566)) * add block storage changes to BlockChanges extractor model ([fc96b8e](fc96b8e)) * add dci plugin to ProtocolExtractor ([5edf7e8](5edf7e8)) * add entry point tables and types to PostgreSQL schema ([3e53c46](3e53c46)) * add entry point tracing functionality ([1d64d12](1d64d12)) * add entrypoint and entrypoint params to TxWithChanges ([3237e0f](3237e0f)) * add entrypoint tracking to component tracker ([87839a0](87839a0)) * add get_traced_entry_points endpoint to RPCClient ([d4a3238](d4a3238)) * add get_valid_from to row versioning control ([003df54](003df54)) * add method to fetch account storage ([d1f2b4f](d1f2b4f)) * add pagination to EntryPointGateway ([9df2e5a](9df2e5a)) * add slots tracing to the RPC tracer ([21b98e5](21b98e5)) * add trace results to BlockChanges and BlockAggregatedChanges ([5b4e56e](5b4e56e)) * add triggers for updating modification timestamps on entry point tables ([5d0d72b](5d0d72b)) * Add Unknown RpcError type ([f0caeee](f0caeee)) * add UpsertEntryPoints and UpsertTracedEntryPoints operations to WriteOp ([a71add2](a71add2)) * AddEntrypointRequestBody improvements ([405499c](405499c)) * address rebase changes ([3fd0d50](3fd0d50)) * Create DirectGateway and use in RPCHandler ([fc349ce](fc349ce)) * create DynamicContractIndexerTrait for simpler generic handling ([df48540](df48540)) * DCI Release ([#595](#595)) ([d33afb3](d33afb3)) * Do not error if tracing result not found + small fixes ([79d16e0](79d16e0)) * enhance EntryPointFilter to support component id filtering ([4694a18](4694a18)) * extend BlockAggregatedChanges and BlockChanhes dto to include DCI data ([cc6e8ed](cc6e8ed)) * extend TxWithChanges to include EntryPoints ([475e253](475e253)) * get rpc url from global args, get api key from env ([06debe1](06debe1)) * handle dci data on tycho-client synchronizer ([5196938](5196938)) * implement entry-points functions on PostgresGateway ([1a374db](1a374db)) * Manually paginate when getting traced entrypoint ([8dafb0d](8dafb0d)) * merge traits and finish EVMBatchAccountExtractor ([69ef0f8](69ef0f8)) * move setting dci plugin to extractor config ([be06549](be06549)) * register add_entry_points endpoint ([6fdf540](6fdf540)) * Return tracing results to user after adding entry points ([6a3153a](6a3153a)) * sketch DCI behavior ([efa87c7](efa87c7)) * split inserting static and dynamic parts of a new contract ([76d8f71](76d8f71)) * Take RPC_URL as input to ServicesBuilder ([b775185](b775185)) * Take tracer as input to RpcHandler ([28ed1b4](28ed1b4)) * **tycho-common:** create a dedicated MergeError for merge issues ([169fe65](169fe65)) * Update DCI to use new parameters ([2ecd88c](2ecd88c)) * update tycho-substreams to 0.3.1 ([5336a53](5336a53)) * Use generic over EntryPointTracer for tracer ([3c62f1b](3c62f1b)) * **versioning:** drop versions older than what already exists in the db ([1c0c764](1c0c764)) ### Bug Fixes * address a bug with `.returning` not returning on conflicts. ([db11bd2](db11bd2)) * correctly add default balances and code on account creations ([6e64424](6e64424)) * correctly handle contract inserts ([9f35ee8](9f35ee8)) * correctly handle many new entrypoints for a single transaction ([7ffdf5a](7ffdf5a)) * correctly pass entrypoint data when parsing Substreams message ([f36cfdf](f36cfdf)) * correctly retrieve db ids when inserting tracing results. ([ae22943](ae22943)) * correctly retrives the rpc url from CLI args ([6731078](6731078)) * correctly serve entrypoint and tracing params in the PostgresGateway ([fbdab08](fbdab08)) * correctly setup "rpc" DCI on startup ([7b31a19](7b31a19)) * correctly sort `txs_with_update` after DCI inserts ([98f6ace](98f6ace)) * do not update in insert contract gateway fn ([6cdcf8a](6cdcf8a)) * fix broken sql down migrations ([e5d41ab](e5d41ab)) * get_traced_entrypoints pagination + caching ([f6e1742](f6e1742)) * handle outdated deletes ([730e55b](730e55b)) * handle skipping multiple updates for same entity within a block ([e9ef60b](e9ef60b)) * Keep OpenAPI clean with proper dto macros ([a69ddcc](a69ddcc)) * Make clippy happy ([67d725c](67d725c)) * Properly match params to result in get_traced_entry_points_inner ([ddd6cf5](ddd6cf5)) * Remove add_entry_points from swagger UI ([29b9be1](29b9be1)) * Remove unnecessary clones ([8ad2c6c](8ad2c6c)) * Remove unnecessary to_string (equivalent of cloning) ([20ac1f4](20ac1f4)) * Uncomment test ignore ([8cd9214](8cd9214)) * use latest db versions for outdated data check ([feecf18](feecf18)) * wrap the DCI plugin in an Arc Mutex ([09f0d51](09f0d51))
|
This PR is included in version 0.71.0 🎉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.