Initial E2E system tests (`set_code, authorize_upgrade`) #411

karolk91 · 2025-09-25T09:55:37Z

Relates to: #298

Will close #456 .

This initial implementation provides

Scenarios for Polkadot and Kusama relays. These scenearios utilize local pallet Scheduler to execute upgrade related calls as Root:

call set_code with currently used WASM expecting error to happen (this path is checking eg specVersion from WASM and will not allow upgrade to pass if it hasn't been incremented)
call set_code_without_checks with currently used WASM expecting success, basically upgrading to the same wasm but version checks are skipped on runtime level thus allowing to pass
two step approach to upgrade -> authorize_upgrade + apply_authorized_upgrade with currently used WASM expecting error to happen
two step approach to upgrade -> authorize_upgrade_without_checks + apply_authorized_upgrade with currently used WASM expecting success
check that subsequent authorize_upgrade and authorize_upgrade_without_checks can override authorized hash

Scenarios for Polkadot/Kusama system chains (AH, BH, People, Coretime, Collectives, Encointer). These scenarios utilize Relay chain's pallet Scheduler to send XCM Transact go gain root on destination chain:

two step approach to upgrade -> authorize_upgrade + apply_authorized_upgrade with currently used WASM expecting error to happen
two step approach to upgrade -> authorize_upgrade_without_checks + apply_authorized_upgrade with currently used WASM expecting success
check that subsequent authorize_upgrade and authorize_upgrade_without_checks can override authorized hash

packages/shared/src/system.ts

github-actions

The changes introduce a comprehensive set of end-to-end tests for system upgrades (set_code, authorize_upgrade) for both relay chains and parachains. The implementation is well-structured, with most of the logic centralized in the new packages/shared/src/system.ts file. However, there is a copy-paste error in one of the test files, a typo in a test description, and inconsistent, hardcoded values are used for XCM weight, which should be addressed.

packages/polkadot/src/bridgeHubPolkadot.system.e2e.test.ts

packages/shared/src/system.ts

packages/polkadot/src/bridgeHubPolkadot.system.e2e.test.ts

packages/shared/src/helpers/index.ts

packages/shared/src/system.ts

Co-authored-by: Alexandre R. Baldé <[email protected]>

rockbmb

The only changes that really need addressing are the scheduleInlineCallWithOrigin calls, otherwise looks good!

rockbmb · 2025-10-24T20:00:58Z

packages/shared/src/system.ts

+  return runAuthorizeUpgradeScenario(client, {
+    call: client.api.tx.system.authorizeUpgrade,
+    expectedAfterApply: (hash) => [
+      { type: client.api.events.system.RejectedInvalidAuthorizedUpgrade, args: { codeHash: hash } },


If it's not too much work, could you also check the error field of RejectedInvalidAuthorizedUpgrade?

A runtime upgrade to the same WASM failing is expected, but the reasons for the failure should be (and stay) the same.

This can also be left for a future PR.

not sure currently how to do better than:

expectedAfterApply: (hash) => [ { type: paraClient.api.events.system.RejectedInvalidAuthorizedUpgrade, args: { codeHash: hash, error: (r: SpRuntimeDispatchError) => r.eq({Module: { index: '0', error: '0x01000000' } }) } }, ],

since RejectedInvalidAuthorizedUpgrade has associated error of type SpRuntimeDispatchError and in this case it is actually SpRuntimeModuleError ... I believe I cannot "cast" this to anything nicer

@rockbmb , does it look ok for you ?

nvm, TIL I can do it like:

paraClient.api.errors.system.SpecVersionNeedsToIncrease.is(r.asModule)

Indeed - I was unaware that was the issue, as otherwise I'd have pointed you to a few instances of this pattern in PEt.

done in eee2357

rockbmb · 2025-10-24T20:15:49Z

packages/shared/src/system.ts

+) {
+  const authorizeHash = async (someHash) => {
+    const call = params.call(someHash).method
+    await scheduleInlineCallWithOrigin(client, call.toHex(), { system: 'Root' })


Every instance of scheduleInlineCallWithOrigin needs to be given a block provider, which the test config type has:

polkadot-ecosystem-tests/packages/shared/src/helpers/index.ts

Lines 466 to 489 in ae7271e

/**

* Configuration for relay chain tests.

*/

export interface RelayTestConfig {

testSuiteName: string

addressEncoding: number

blockProvider: 'Local'

chainEd?: ChainED

}

/**

* Configuration for parachain tests.

* Async backing is relevant due to the step size of `parachainSystem.lastRelayChainBlockNumber`.

*

* Recall that with the AHM, the scheduler pallet's agenda will be keyed by this block number.

* It is, then, relevant for tests to know whether AB is enabled.

*/

export interface ParaTestConfig {

testSuiteName: string

addressEncoding: number

blockProvider: BlockProvider

asyncBacking: AsyncBacking

chainEd?: ChainED

}

The scheduler in Kusama AH (and soon PAH) keyes its agenda via a nonlocal block provider, which means these tests will fail on KAH.

See other E2E tests for how to do this, e.g. scheduler tests for KAH.

Existing test are actually fine, because the scenarios I introduced are using Relay's scheduler pallet - and that is always a "local" block provider

Before AHM I think parachains didn't have their scheduler (maybe Collectives only?), with AHM we are introducing scheduler pallet to Asset Hubs - so I can possibly create one more scenario specifiacally for post AHM Asset Hub such that it could use its own scheduler to execute the runtime upgrade

Best left for a future PR, this one is 👍

What I meant ⬆️ is that the extra scenario is best left for another PR.

scheduleInlineCallWithOrigin still needs to be given a testConfig.blockProvider param, otherwise, in asset hubs (Kusama now, and Polkadot next week), it'll be scheduling tasks with an incorrect agenda key.

testConfig.blockProvider and extra scenario introduced in eee2357

rockbmb · 2025-10-24T20:26:17Z

packages/shared/src/system.ts

+>(relayChain: Chain<TCustomRelay, TInitStoragesRelay>, paraChain: Chain<TCustomPara, TInitStoragesPara>) {
+  const [relayClient, paraClient] = await setupNetworks(relayChain, paraChain)
+  return runSetCodeScenarioViaRelay(relayClient, paraClient, {
+    call: paraClient.api.tx.system.setCodeWithoutChecks,


The comment says Tests 'setCode' flow — ensures runtime upgrade to the same WASM fails, but this uses setCodeWithoutChecks - won't it pass instead?

True, this code was not used actually because this is a scenario where we want to upgrade parachain, but the parachain doesn't have scheduler pallet so we need to use relay's scheduler - and that fails because XCM message (from relay to parachain) would contain WASM blob and its too big for XCM message.

I have skipped this scenario and just left the code but I think I can actually use this scenario to verify that this will indeed fail

done in eee2357

rockbmb · 2025-10-24T20:34:18Z

Will close #456 .

rockbmb · 2025-10-29T16:42:22Z

packages/kusama/src/assetHubKusama.system.e2e.test.ts

+const testConfig: RelayTestConfig = {
+  testSuiteName: 'Kusama AssetHub System',
+  addressEncoding: 0,
+  blockProvider: 'Local',


This needs to be ParaTestConfig, and the block provider is non-local - asset hub scheduler agendas will use the relay chain's block number as their key.

I have added both scenarios:

using scheduler on relay --> local block provider (from relay perspective)

using scheduler on AH (post-AHM) --> NonLocal provider in that case

rockbmb · 2025-10-29T16:42:37Z

packages/polkadot/src/assetHubPolkadot.system.e2e.test.ts

+import { assetHubPolkadot, polkadot } from '@e2e-test/networks/chains'
+import { type RelayTestConfig, registerTestTree, systemE2ETestsViaRelay } from '@e2e-test/shared'
+
+const testConfig: RelayTestConfig = {


Similar as for Kusama, I have added both scenarios:

using scheduler on relay --> Local block provider (from relay perspective)

(to be uncommented post-AHM) using scheduler on AH --> NonLocal provider in that case

karolk91 · 2025-10-29T22:07:00Z

@rockbmb , few tests for Kusama are failing due to a bug where Preimage pallet is still filtered out post migration and PET scenarios here are extensively using Preimage pallet along with Scheduler to inject stuff for execution

This is affecting current live Kusama Relay. It that has been fixed by polkadot-fellows/runtimes#957 but we will need to wait for Kusama Relay update

karolk91 · 2025-10-30T08:40:24Z

I have commented out failing scenarios for Kusama with TODO to enable them after Kusama 2.0 release (due to preimage pallet bug)
added new scenarios for Kusama where Kusama Asset Hub is used as the "scheduler / governance" chain for other parachains and relay (so basically its like saying lets use KAH gov to upgrade other chains)
added commented out post-AHM scenarios for Polkadot - once we merge this to master, I can prepare a PR to target polkadot-ahm branch and we should enable these tests there

rockbmb · 2025-10-30T11:56:18Z

packages/kusama/src/kusama.system.e2e.test.ts

+
+// registerTestTree(systemE2ETests(kusama, testConfig))
+
+const testConfigForAssetHub: ParaTestConfig = {


This was fine as it was, RelayTestConfig

rockbmb · 2025-10-30T11:57:03Z

packages/polkadot/src/polkadot.system.e2e.test.ts

+
+// TODO: Uncomment Post-AHM on Polkadot
+
+// const testConfigForAssetHub: ParaTestConfig = {


This also needs to changed: RelayTestConfig.

I know it's commented out, but still.

karolk91 added 2 commits September 25, 2025 11:47

Initial E2E system tests (set_code and set_code_without_checks)

74caa30

Add scenarios using authorize+apply

000cd7f

karolk91 changed the title ~~Initial E2E system tests (set_code and set_code_without_checks)~~ Initial E2E system tests (set_code, authorize_upgrade) Sep 25, 2025

karolk91 marked this pull request as ready for review September 25, 2025 14:17

rockbmb reviewed Sep 27, 2025

View reviewed changes

packages/shared/src/system.ts Outdated Show resolved Hide resolved

rockbmb reviewed Sep 27, 2025

View reviewed changes

packages/shared/src/system.ts Outdated Show resolved Hide resolved

rockbmb reviewed Sep 27, 2025

View reviewed changes

packages/shared/src/system.ts Show resolved Hide resolved

Upgrade scenarios for Polkadot/Kusama system chains

df85f3d

github-actions bot reviewed Sep 30, 2025

View reviewed changes

karolk91 and others added 2 commits October 1, 2025 08:31

Code-review fixes

cfab73a

Merge branch 'open-web3-stack:master' into system-tests

4132c03

rockbmb reviewed Oct 3, 2025

View reviewed changes

karolk91 and others added 2 commits October 7, 2025 09:03

Update packages/shared/src/system.ts

e15fbb6

Co-authored-by: Alexandre R. Baldé <[email protected]>

Code review improvements

950bc13

rockbmb requested changes Oct 24, 2025

View reviewed changes

rockbmb assigned karolk91 Oct 27, 2025

rockbmb added enhancement New feature or request e2e tests Related to end-to-end tests labels Oct 27, 2025

rockbmb added this to the Polkadot Migration milestone Oct 27, 2025

rockbmb approved these changes Oct 29, 2025

View reviewed changes

rockbmb reviewed Oct 29, 2025

View reviewed changes

karolk91 added 3 commits October 29, 2025 18:18

Merge remote-tracking branch 'origin/master' into system-tests

5dc94f5

Review fixes and improvements

eee2357

Merge remote-tracking branch 'upstream/master' into system-tests

b05e42f

rockbmb changed the title ~~Initial E2E system tests (set_code, authorize_upgrade)~~ Initial E2E system tests (set_code, authorize_upgrade) Oct 30, 2025

Add more post-AHM scenarios

da4dd61

rockbmb reviewed Oct 30, 2025

View reviewed changes

rockbmb merged commit 3123d23 into open-web3-stack:master Oct 30, 2025
99 checks passed

rockbmb mentioned this pull request Nov 10, 2025

Rework test config design #477

Open

	/**
	* Configuration for relay chain tests.
	*/
	export interface RelayTestConfig {
	testSuiteName: string
	addressEncoding: number
	blockProvider: 'Local'
	chainEd?: ChainED
	}

	/**
	* Configuration for parachain tests.
	* Async backing is relevant due to the step size of `parachainSystem.lastRelayChainBlockNumber`.
	*
	* Recall that with the AHM, the scheduler pallet's agenda will be keyed by this block number.
	* It is, then, relevant for tests to know whether AB is enabled.
	*/
	export interface ParaTestConfig {
	testSuiteName: string
	addressEncoding: number
	blockProvider: BlockProvider
	asyncBacking: AsyncBacking
	chainEd?: ChainED
	}


		// registerTestTree(systemE2ETests(kusama, testConfig))

		const testConfigForAssetHub: ParaTestConfig = {


		// TODO: Uncomment Post-AHM on Polkadot

		// const testConfigForAssetHub: ParaTestConfig = {

Initial E2E system tests (set_code, authorize_upgrade) #411

Initial E2E system tests (set_code, authorize_upgrade) #411

Uh oh!

Conversation

karolk91 commented Sep 25, 2025 • edited by rockbmb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rockbmb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karolk91 Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karolk91 Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rockbmb commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karolk91 commented Oct 29, 2025

Uh oh!

karolk91 commented Oct 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Initial E2E system tests (`set_code, authorize_upgrade`) #411

Initial E2E system tests (`set_code, authorize_upgrade`) #411

karolk91 commented Sep 25, 2025 •

edited by rockbmb

Loading

karolk91 Oct 29, 2025 •

edited

Loading

karolk91 Oct 29, 2025 •

edited

Loading

rockbmb commented Oct 24, 2025 •

edited

Loading