feat(node): gRPC interface #9703
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for GitHub. 6 Skipped Deployments
|
github-actions
bot
added
ci
muXxer
force-pushed
the
feat/grpc-implementation
branch
3 times, most recently
from
a43fbe9 to
e366d4f
Compare
- Ported field masking mechanism, including - The `field` module - The `iota-proto-build` crate to generate fields, so we can use `FieldMaskUtil::validate` to verify the client-requested field paths. - Change `EventID` to `EventId` to meet `prost`'s naming convention, so the generated Rust code from `protobuf` can be used without additional conversion (like change `ID` to `Id`). Part of #8688 - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change This PR adds a new transaction filter and a new event filter for gRPC (not used yet by the API). It combines the two existing json-rpc filters into one, but also adds the possibility to filter for transactions that match a certain event filter. Also, the possibility to chain the filters was added (`Any`, `All`, `Or`, `And`) similar to the event filters. ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes
- Initialize the gRPC proto files, which addressed APIs documented in - Note that the design of gRPC request/response in is that we focus on returning low-level BCS iota-type data, instead of extracting fields from the iota-type data and put them in response in most cases (to discuss). Part of #8688 - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change - Upstream range: [v1.46.3, 1.47.1) - Port commit: - MystenLabs/sui@343906c - MystenLabs/sui@1a496a5 - MystenLabs/sui@aec90b4 - MystenLabs/sui@24f9ea2 - Description: - Avoid std::thread::sleep() in safe_drop_db() - Add backoff to safe_drop_db - Mitigates the race between the destructor and metric threads. Reason: `rocksdb::DB::Destroy` returns an error immediately if any other reference is still alive - Reworks the 'rest-index' database to: - Add support for indexing information about epochs ## Links to any relevant issues None ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [x] Nodes (Validators and Full nodes): Add support for indexing epochs, object type information for transactions, type information for dynamic fields. Hint: This will trigger re-indexation of the REST indexes. --------- Co-authored-by: muXxer <[email protected]>
- Implemented`GetEpoch` in the gRPC API. - Verified the `get_epoch` simtest by using `cargo simtest --package iota-e2e-tests --test grpc -- get_epoch` Part of #8688 - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
…types (#9355) This PR refactors the `iota-grpc-types` file / folder structure, adds auto-generated accessors and google types for error details. - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes
# Description of change - Implemented`GetServiceInfo` in the gRPC API. - Make `/crates/iota-e2e-tests/tests/grpc` owned by node team. - Verified the `get_service_info` simtest by using `cargo simtest --package iota-e2e-tests --test grpc -- get_service_info` ## Links to any relevant issues Part of #8688 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change - Implemented `GetObjects` in the gRPC API. - Verified the `get_objects` simtest by using `cargo simtest --package iota-e2e-tests --test grpc -- get_objects` ## Links to any relevant issues Closes #9390 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change This PR fixes the compile errors in the gRPC feature branch introduced by the new `iota-sdk-types` in #9360. ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes
…PC (#9437) # Description of change - Implemented `SimulateTransaction` and `ExecuteTransaction` in the gRPC API. ## Links to any relevant issues Fixes #9392 #9393 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change - Implemented`GetTransactions` in the gRPC API. - Verified the `get_transactions` simtest by using `cargo simtest --package iota-e2e-tests --test grpc -- get_transactions` - Note that #9437 should be merged before this PR. ## Links to any relevant issues Closes #9391 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
# Description of change This pull request introduces IOTA-specific metadata headers to all gRPC responses in both the Ledger and Transaction Execution services. It does so by implementing a new helper function and macro for appending these headers, then updating all relevant service endpoints to use this mechanism. Since tonic does not support interceptor to modify a response, we need to go with macro and apply it to all methods. ## Links to any relevant issues Close #9623 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have checked that new and existing unit tests pass locally with my changes --------- Co-authored-by: muXxer <[email protected]>
- Upstream range: [1.46.3, 1.47.1) - Related commit: MystenLabs/sui@0316d74 - Description: We port the TLS support for gRPC, but not for original http servers --------- Co-authored-by: muXxer <[email protected]>
# Description of change Add high-level gRPC API calls, which makes the types of input parameters and responses be all from `iota-sdk-types` for ease of usage. ## Links to any relevant issues Fixes #9620 ## How the change has been tested - [ ] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Add high-level client API calls. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
This PR add a simulacrum binary, that hosts a gRPC interface to interact with the fullnode API, and a REST API to control the simulacrum. This is useful to run simulacrum in the CI in non-rust projects. fixes #7844 - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Add simulacrum-server binary to simulate an IOTA blockchain - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: jkrvivian <[email protected]>
# Description of change - Add e2e tests for gRPC clients. - We cover edge cases but try to make the test set minimal, like if they are covered by Rust's type system, or similar error conditions (e.g., gas_budget=1 and gas_budget=0), we don't include them because they are redundant. - Run `cargo simtest -p iota-e2e-tests -- client::` for all the e2e tests. - Fixes the signatures field mask (changing signatures.bcs to signatures) in the gRPC client - Implements the previously unimplemented `get_checkpoint_data` endpoint in the server for ease our e2e tests - Note that we might need change after checkpoint streaming is completed - Corrects signature deserialization to use BCS (found by our e2e tests) ## Links to any relevant issues Fixes #9621 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [x] Patch-specific tests (correctness, functionality coverage) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Add e2e gRPC tests - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
muXxer
force-pushed
the
feat/grpc-implementation
branch
from
cbcf661 to
a91285e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
TBD
Links to any relevant issues
TBD
How the change has been tested
Release Notes