feat(scripting): field coverage tests for zod schemas#692
Open
yahgwai wants to merge 31 commits intofeat/zod-scriptingfrom
Open
feat(scripting): field coverage tests for zod schemas#692yahgwai wants to merge 31 commits intofeat/zod-scriptingfrom
yahgwai wants to merge 31 commits intofeat/zod-scriptingfrom
Conversation
Detects dead schema fields by mutating each field's value and verifying the transform output changes. Values are generated from the schema type. Includes test for getValidators schema.
…ansactionRequest Tests nested object fields (coreContracts.upgradeExecutor, coreContracts.sequencerInbox) alongside flat fields and hex schema.
Exports are needed for schema coverage testing. The runScript call remains as a top-level side effect for direct execution.
Simplified assertSchemaCoverage API to take a single schema (with transform baked in via .transform()). Added tuple, array, and never type support to the value generator. The coverage test caught a real bug: createRollup example was not passing parentChainId to findChain when constructing the public client.
assertSchemaCoverage now accepts an optional execute function for full pipeline testing. When provided, it mocks the SDK function and compares its call args across two runs, catching fields that survive the transform but get overwritten in the script body. The createRollup example test now exercises the full pipeline.
Added Proxy-based tracked objects that record all method calls into a shared sideEffects array. When execute is provided, assertSchemaCoverage runs the full pipeline (parse + execute) and compares all side effects across mutations, not just transform outputs. transferOwnership example now exercises the full execute function with mocked SDK functions, viem methods, and client interactions. Uses the override mechanism for nativeToken (conditional branch on zero/non-zero).
Replace separate testTracker.ts module with a vi.hoisted() inline registry. All mock functions (fn, fnSync, trackedObject) are created in one place and shared with vi.mock factories via hoisting. Tracked objects now return valid hex strings instead of embedding names in hex positions.
Remove unused sdkFunction parameter. Make execute required -- callers compose schema.parse + SDK function into a single execute function via withTransform/withExecute helpers. Remove the two-path branching (transform-only vs full pipeline) since all tests now go through mocks.
Each test now inlines the schema.parse + SDK call as a simple lambda. No helpers needed.
Execute now receives parsed output directly. Tuple results (from simple schema transforms) are auto-spread into the execute function. No more wrapper lambdas needed -- callers just pass the SDK function or execute.
Make assertSchemaCoverage generic so override functions receive and return z.input<T> instead of Record<string, unknown>.
Optional/nullable fields are now skipped in auto-generation (tested via overrides instead) to avoid refine conflicts. The keyset field uses an override to set AnyTrust context. Caught another parentChainId bug -- deployNewChain was not passing it to findChain, same issue as createRollup.
Provide required chainId and InitialChainOwner fields in the keyset override's chainConfig object.
Add coverage for getKeysets, getBatchPosters, isAnyTrust, prepareKeysetHash, prepareKeyset, prepareChainConfig, setAnyTrustFastConfirmer, upgradeExecutorPrepareTransactionRequest, setValidKeyset, createRollupFetchCoreContracts, createRollupFetchTransactionHash. Optional/nullable fields are now skipped from auto-generation to avoid refine conflicts (tested via overrides instead). 16 total tests, all passing.
Add coverage for fetchDecimals, fetchAllowance, upgradeExecutorFetchPrivilegedAccounts, getBridgeUiConfig, isTokenBridgeDeployed, createRollupGetRetryablesFees, createSafePrepareTransactionRequest, and all 4 customFeeToken schemas. 27 total tests, all passing. getDefaults deferred to Stage 3 (needs union support in generator).
Add coverage for createRollupPrepareTransactionRequest (default), createRollup (default), createTokenBridge, createTokenBridgePrepareTransactionRequest, createTokenBridgePrepareSetWethGatewayTransactionRequest, prepareDeploymentParamsConfigV21, prepareDeploymentParamsConfigV32, prepareNodeConfig, feeRouterDeployRewardDistributor, feeRouterDeployChildToParentRewardRouter, getDefaults. Added support for union types, z.null(), and nonoptional (from .required()) in the schema walker and value generator. 38 total tests, all passing.
Tests cover both schemas and examples, so they belong at the scripting level rather than inside schemas/.
generateForType now throws with a clear message when it encounters an unhandled zod type, instead of silently producing an invalid value. Added section comments to the test file explaining the ordering: 1. Mock registry (vi.hoisted), 2. Module mocks, 3. Imports, 4. Tests.
Consumers can import createMockRegistry, viemTransformsMocks, and assertSchemaCoverage to test their own schema coverage. The registry must be inlined in vi.hoisted() due to vitest limitations, but the canonical implementation lives in testing.ts.
…ation
vi.mock and vi.hoisted in an imported file get hoisted into the
importing test file's scope. This means testing.ts can own the
registry, all vi.mock calls, and the vi.hoisted block. The test file
just imports { mocks, assertSchemaCoverage } and writes tests.
No more duplicated registry between testing.ts and the test file.
A second test file imports { mocks, assertSchemaCoverage } from
testing.ts and adds its own vi.mock for a custom SDK function. Both
tests pass, proving the pattern works for consumers without vi.hoisted
duplication.
Single file for all test infrastructure: mock registry, vi.mock
setup, schema walker, value generator, and assertSchemaCoverage.
Consumers import { mocks, assertSchemaCoverage } from one place.
Fixed consumer test to use createRollupGetCallValue with proper types
instead of sequencerInboxReadContract with as any cast.
- Fix stale usage comment (referenced ./testing instead of ./schemaCoverage)
- Build base fixture once outside the loop instead of per-leaf
- Pre-compute leaf path keys to avoid repeated join('.')
- Remove unnecessary export on getSchemaLeaves (internal only)
… unions createRollupPrepareTransactionRequest and createRollup have different params schemas per version (v2.1 has extraChallengeTimeBlocks/ genesisBlockNum, v3.2 has feeTokenPricer/customOsp/bufferConfig/etc). Testing only the default variant missed version-specific fields. 42 total tests, all passing.
- Fix missing findChain in deployNewChain toWalletClient call (same bug class as the toPublicClient fix already applied) - Throw if assertSchemaCoverage finds zero testable fields (prevents silently passing on misconfigured schemas) - Fix misleading vi.mock hoisting comment - Change generateChainId mock to use registry for consistency - Document that optional fields are excluded from automatic testing and must use overrides
yahgwai
force-pushed
the
feat/schema-coverage
branch
from
fbd9986 to
6d131b9
Compare
yahgwai
changed the title
yahgwai
changed the title
Run prettier on schemaCoverage files. Add eslint-disable for no-explicit-any in schemaCoverage.ts -- the file introspects zod internals and uses Proxy-based mocks that require any.
yahgwai
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests to make sure that every input field of the schema causes some discernible effect in the sdk function(s) that are called. Basically a check for dead schema fields - especially useful when writing new schemas for scripts that tie multiple sdk functions together