-
Notifications
You must be signed in to change notification settings - Fork 0
Feature/node 4771 add test chains for non prod naga deploys #79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/node 4771 add test chains for non prod naga deploys #79
Conversation
|
PASS [ 44.885s] (3/3) lit_node::test toxiproxy::perf_tests::load_with_no_latency |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for running integration tests against live Naga testnet networks, in addition to the existing local Anvil/Hardhat test chains. The implementation introduces a new chain type called "Naga" that reads configuration from a live_testnet.toml file and connects to existing deployed networks rather than spinning up local chains.
Changes:
- Introduced a new
Nagachain implementation that reads network configuration fromlive_testnet.tomland connects to existing live testnets - Renamed
WhichTestnetenum toTestNetNameand addedNagavariant to support the new testnet type - Refactored PKP creation to use a unified
new_pkp(key_set_name)method that automatically routes to Datil or mainnet implementations based on the key set name - Updated test infrastructure to dynamically determine HTTP/HTTPS based on socket addresses and support live network connections
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| rust/lit-node/lit-sdk/src/common.rs | Added from_socket_address method to determine URL prefix (http/https) based on whether address is localhost |
| rust/lit-node/lit-node-testnet/src/testnet/chain/naga.rs | New Naga chain implementation that reads live testnet configuration from TOML file |
| rust/lit-node/lit-node-testnet/src/testnet/mod.rs | Renamed WhichTestnet to TestNetName, added Naga support, changed process field to Optional |
| rust/lit-node/lit-node-testnet/src/validator.rs | Added new_from_testnet to create validators from existing live network nodes |
| rust/lit-node/lit-node-testnet/src/end_user/mod.rs | Changed wallet funding to use actual transactions instead of RPC calls for live networks |
| rust/lit-node/lit-node-testnet/src/end_user/pkp/mod.rs | Refactored PKP creation to route between Datil and mainnet based on key set name |
| rust/lit-node/lit-node-testnet/src/lib.rs | Auto-detect live_testnet.toml and configure for Naga network, increased default timeout |
| rust/lit-node/lit-node/live_testnet_sample.toml | Sample configuration file for live testnet connections |
| rust/lit-node/lit-node/.gitignore | Added live_testnet.toml to gitignore to prevent accidental commits |
| rust/lit-node/lit-node/tests/integration/*.rs | Updated tests to use new new_pkp(key_set_name) API |
| rust/lit-node/lit-node/tests/common/*.rs | Updated SDK request builders to use from_socket_address for URL prefix determination |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
* quick fix of the pkphelper contract. * update js/ts contratc tests * simplify and updated abi * Update PKP Helper V2 as well. * Prettier. * Update ABI Staleness
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
DashKash54
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lot of files have been changed so I want to confirm the objective of this PR- run node tests: unit, integration, components, etc against Naga-test?
| fn contract_resolver_address(&self) -> Address { | ||
| Address::zero() | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the point of this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The return value from setting up a "testnet" is a trait - I needed a way to get the contract_resolver address for test/production addresses; given that it's a trait and is never known, this value gets implemented by the naga networks based on their incoming data. For local tests the value is never used, so I put in a default implementation to avoid having to write extra code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about adding a comment that this is "only" referenced in local testing as it might be confusing since Naga is also one of the enums for Testnet?
|
|
||
| for &validator in validators.iter() { | ||
| let public_address = validator.public_address(); | ||
| let socket_address = validator.public_address(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
misleading name: If it gives the socket address instead of the public IP then we should rename the function?
|
| ) -> Result<Contracts> { | ||
| let contracts = Self::new_contracts(ca, provider.clone()).await; | ||
|
|
||
| if testnet.which != WhichTestnet::NoChain { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we even support NoChain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are ( where? haven't checked in a long time! ) a handful of tests that required some of the underlying infra, but no chain.
Yes against any production network really. ( Also allows us to script things like key migration, hopefully! ) |
|
But it only support Naga-test right now? |
No, any production network. You just need to ensure the configuration data ( the chain & resolver contract mainly ) are correct in the configuration file. It's also helpful if the network is compatible with the current code base 🤣 .... though most tests will work with fairly large underlying changes, so long as the interface to the nodes and the PKP related contracts are the same. ( Basically the same restriction as the SDK tests ). |
DashKash54
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Picking up from a missed merge. * Prep for lit-peer * WIP adding a second keyset * prettier * Clippy! * fix node_restarts_without_key_material - and maybe an FSM update? * remove keyset test ( next branch ) * Better handling of shadow splicing. (#26) * Better handling of shadow splicing. * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> * Quick ordering update for shadowOpts struct. * abi staleness update * blockchain staleness ? * Handle datil old peers (#34) * handle old peers Signed-off-by: Michael Lodder <[email protected]> * Open v2.1.7 --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Brendon <[email protected]> * Keysets - handle multiples in DKG. (#15) * Update test * Update tests * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * resync /w keysets feature. --------- Co-authored-by: Copilot <[email protected]> * fix tests * cargo fmt * cargo update / abi staleness * Fixes to merges & clippy updates * update tests * Fix deploy/test JS ( which I broke with a bad merge ) * prettier.... * Feature/node 4848 test datil keys in naga (#24) * Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * fix actions in testnet * cargo fmt * patch epoch change retries. * cargo fmt * Fix failed DKGs when doing restores. * Feature/node 4874 support multiple chains at anvil level (#41) * Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * fix actions in testnet * cargo fmt * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * Update test to interact with ImportedDatilTestnet * remove unneded code * refactor & remove duplications * Refactor & add datil node secrets * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/src/tss/common/restore/eks_and_ds.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/Cargo.toml Co-authored-by: Copilot <[email protected]> * cargo lock cleanup * cargo.lock update + config.rs in the resolver formatting. * patch epoch change retries. * cargo fmt * Fix failed DKGs when doing restores. * cargo fmt * fix tests * Logging info for CI datil instances. * Open ports for a second instance of anvil in the yml files. * Load hex data from test state into anvil. * remove unused file * comment out #if false * take 2? * fix test reference * Use HEX file to load state, rather than a JSON file. * Refactor to insure that for Datil tests we don't rely on naga cache'd chain state. * switch lit-recovery binaries for testing * Remove keyset_identifier as an enum and clean up tests. * fix datil backup selection of binary. * cargo fmt * yml test. * if false yml * comment out version upgrade * yml update * try a var for the upgrade workflow ? * update lit-os cargo.lock * update gitinfo --------- Co-authored-by: Howard Tam <[email protected]> Co-authored-by: Copilot <[email protected]> * Feature/node 4876 - using both chains in some tests ( adapt end user to support pkps in both chains ) (#42) * Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * fix actions in testnet * cargo fmt * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * Update test to interact with ImportedDatilTestnet * remove unneded code * refactor & remove duplications * Refactor & add datil node secrets * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/src/tss/common/restore/eks_and_ds.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/Cargo.toml Co-authored-by: Copilot <[email protected]> * whole lotta shaking going on. ( endpoints changed throughout ) * PKP sign works. * cargo clippy * cargo lock cleanup * git merge fixes * cargo toml update ( to trigger cargo.lock update ) * cargo.lock update + config.rs in the resolver formatting. * cargo fmt * fix panic when node was kicked (realmid = 0) * patch epoch change retries. * cargo fmt * Expose underlying error when peer_connect attestation fails * Fix failed DKGs when doing restores. * cargo fmt * fix tests ! * fix tests * Logging info for CI datil instances. * Open ports for a second instance of anvil in the yml files. * Load hex data from test state into anvil. * Feature/price setting script (#43) * first pass at price setting script * price setting script seems like it will work * script works on dev and prod * Update blockchain/contracts/scripts/generatePriceSettingTransactions.ts Co-authored-by: Copilot <[email protected]> * remove unused map * regen contracts * 100x decrease for testnets when setting prices * divide by 10000 when setting naga-test prices * actually, made naga-test 100000x cheaper than naga prod --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Brendon <[email protected]> * Fix: Add `correlation_id` to `lit-actions` spans (#25) * Add spans for actions * Add correlation_id to TracingMiddleware * Checkout server.rs * Extract correlation_id' --------- Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Brendon <[email protected]> * Refactors OsQuery metrics emission to use OpenTelemetry Gauges (#28) * Add spans for actions * Improve metrics * first pass at adding request id to logs * Update rust/lit-node/lit-node/src/main.rs Co-authored-by: Copilot <[email protected]> * change how we set the request id on the span * clippy * Add correlation_id to TracingMiddleware * Checkout server.rs * Extract correlation_id' * Fix clippy wanrning * separate info from gauge metrics (#44) * Revert "Merge branch 'inf-852-fix-tracing-lit-actions' into inf-753-olly-metrics" This reverts commit b0a6d38, reversing changes made to 4a58992. * Revert "Merge remote-tracking branch 'origin/feature/NODE-4860-correlation-id-in-logs' into inf-753-olly-metrics" This reverts commit 7be7bbe, reversing changes made to b0a6d38. * cargo fmt --------- Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Adam Reif <[email protected]> * remove unused file * comment out #if false * take 2? * fix test reference * Use HEX file to load state, rather than a JSON file. * Refactor to insure that for Datil tests we don't rely on naga cache'd chain state. * switch lit-recovery binaries for testing * Remove keyset_identifier as an enum and clean up tests. * Fix up tests * cargo fmt * fix datil backup selection of binary. * logging, shorten handshake challange value. * fix "default" keyset bug :( * cargo fmt * yml test. * if false yml * comment out version upgrade * yml update * try a var for the upgrade workflow ? * upgrade yml with default if statement. * cargo clippy. * Update lit-os cargo.lock * update lit-os cargo.lock * update gitinfo * fix restore test to use Datil directly. * cargo fmt * patch so both long running restores work :( * Update epoch change long to deal with keysets * small shiva refactor * difference between pkp & root key lookups when getting a keyset. * remove datil from lit action testing. * clean up integration test setup to account for keyset settings. * limit caches to only active states. * Fix Shadow splicing test, make some integration force builds * update abis & lit-os lock file * cargo fmt + minor cleanup * Refactor one of the last timing related tests. * ABI staleness, remove more testing feature flags * Make the StakerAccountSetupMapper Send+Sync to allow Shiva to build. * Clean up complaints. * Update to determine key set for recovery (#58) * update to determine key set for recovery Signed-off-by: Michael Lodder <[email protected]> * add test when no key set is on chain Signed-off-by: Michael Lodder <[email protected]> * update test Signed-off-by: Michael Lodder <[email protected]> --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Brendon <[email protected]> * Better logging. --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Howard Tam <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Abhi... <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Michael Lodder <[email protected]> * abi staleness + cargo fmt. * Remove traces of custom version upgrade testing. (#57) * fix: Invalid signature combination (#65) * fix invalid signature combination Signed-off-by: Michael Lodder <[email protected]> * fix tests Signed-off-by: Michael Lodder <[email protected]> --------- Signed-off-by: Michael Lodder <[email protected]> * re-instate decryption assertion * clippy. * remove unused val * make openapi spec (#66) * make openapi spec Signed-off-by: Michael Lodder <[email protected]> * remove "with same ip and port" test - chck /w Mike? --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: GTC6244 <[email protected]> * Feature/node 4923 refactor upgrade tests to use local configurations (#60) * Remove traces of custom version upgrade testing. * First few changes - start simplification. * restore key_types in staking contract, allow validator cloning. * Update to allow old version to load * update local builds * debugging updates * better logging, disable network checks between epoch changes ( for now ) * clippy! * add a filnal check. * cargo fmt * clippy. * Download an install old binaries as required. * ignore upgrades by default. * re-include Pallas curves in deployment scripts. * add functions to create proper upgrade envrionment for old nodes ( remove curves, change yaml files ). * blockchain abi stales + cargo fmt. * disable test ( with same port ) * re-add >= to epoch change * Update contracts to include keysets_deprecated type in tests. * missing "deprecated" param in test * merge master fixes * try to update docs * force deploy on lit-actions. * force deploy on more lit actions. * Feature/node 4931 implement lit actions for keysets (datil support!) (#62) * Lit Action Server side updates /w Docs * lit-node side updates. * Add a few tests. * Update rust/lit-node/lit-node-testnet/src/testnet/contracts_repo.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-actions/ext/js/02_litActionsSDK.js Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node-testnet/src/testnet/contracts_repo.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node-testnet/src/testnet/contracts_repo.rs Co-authored-by: Copilot <[email protected]> * copilot suggestions * documentation update * cargo fmt * clipppppppyyyyy! * Update docs? * build docs again! * add in missing "keysetId" values from lit action code samples that are stored as string in mods. * force lit-action datil tests to run outside of caches. * test forcing Lit Actions to recompile ( cache with sometimes using Datil seems off ) * regen docs * Version upgrade check * Refactor for explicit endpoint change - include keyset as part for lit-actions, which can be overwritten wtih a js param. * documentation update * use force_deploy for lit actions * error handling on reading anvil file. * read/write hex data as string + better error handling. * better debugging. * revert keyset in session sig issue. ( creates invalid IPFS CID value! ) * cargo fmt --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Adam Reif <[email protected]> * Remerge build.rs! * cargo fmt * more debugging. * remerge build.rs * cargo fmt * Feature/node 4947 set default keyset on contract deployment (#82) * set default realm config in the deploy.ts, and remove from test builder in node code ;-) * prettier! * switch function name.... which makes me wonder if my test executed! * ABI stalensss * add CI debbugging. * Reset all defaults for realm #1 * Feature/node 4940 fix keyset ci times (#76) * Make datil always available, even if not used. * Add debugging. * update build.rs * debugging * Test / Refactor / Discover - anvil cache issues. * rename to edge testing, add a matrix * clippy ! * test splitting out the edge related tests. * cargo fmt * config update (nomenclature, mainly) * Use an updated version of datil contracts that can be properly cleared of root keys. * remove unit test partitioning * add debugging info * Add keyset into ClientBuilder. * force deploy for the child lit action function. * fix backup-long to use only naga keysets. * cargo fmt * Feature/node 4771 add test chains for non prod naga deploys (#79) * Make datil always available, even if not used. * Add debugging. * update build.rs * debugging * Test / Refactor / Discover - anvil cache issues. * rename to edge testing, add a matrix * clippy ! * test splitting out the edge related tests. * cargo fmt * config update (nomenclature, mainly) * Use an updated version of datil contracts that can be properly cleared of root keys. * remove unit test partitioning * add debugging info * Add keyset into ClientBuilder. * force deploy for the child lit action function. * first pass * Updates to make testing easier * Simplifications of datil vs normal PKPs. * Refactor to search for a local test configuration file, instead of duplicating tests. * Remove duplicate tests. * ensure datil wallets get funded. * clean up datil PKP functions. * Revert the proper funding amount :) * Feature/node 4948 fix broken pkppermissions helper lookup (#81) * quick fix of the pkphelper contract. * update js/ts contratc tests * simplify and updated abi * Update PKP Helper V2 as well. * Prettier. * Update ABI Staleness * Update rust/lit-node/lit-node-testnet/src/lib.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/live_testnet_b.toml Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/live_testnet_sample.toml Co-authored-by: Copilot <[email protected]> * fix merge issues --------- Co-authored-by: Copilot <[email protected]> --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Michael Lodder <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Howard Tam <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Abhi... <[email protected]> Co-authored-by: Chris Cassano <[email protected]>
WHAT
A limited subset of tests that work on Naga networks !
HOW / SETUP :
There is a new
live_testnet_sample.tomlthat contains testnet configuration data - rename this by removing the_sampleportion of the file, and all tests will be run against the desired networks.The test suite uses a well known wallet to fund the "users" created during the test; this wallet is the default Anvil/Hardhat wallet #0 - also known as
0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266Prior to running tests, head over to the Yellowstone faucet ( https://chronicle-yellowstone-faucet.getlit.dev/ ) and claim some tokens.
A single claim of tokens should run a few tests without difficulty, though the exact amount depends on pricing, etc.
CAVEATS
Some tests will attempt to mutate our contracts directly - these tests will obviously fail, as this mutation is all attempted using the well-known account mentioned above.
Test now cost "real" coin, so may be subject to running out of funds, or may not get very far if the well-known account above can't send tokens to whatever test wallets ( end user emulation) are created.