Extend governance E2E tests

[rockbmb]

As mentioned in #240, governance E2E tests can be expanded.

In particular, consider the referendum lifecycle test that currently exists in the suite:

polkadot-ecosystem-tests/packages/shared/src/governance.ts

Lines 105 to 677 in 8736152

	export async function referendumLifecycleTest<
	TCustom extends Record<string, unknown> | undefined,
	TInitStorages extends Record<string, Record<string, any>> | undefined,
	>(client: Client<TCustom, TInitStorages>, addressEncoding: number) {
	// Fund test accounts not already provisioned in the test chain spec.
	await client.dev.setStorage({
	System: {
	account: [
	[[devAccounts.bob.address], { providers: 1, data: { free: 10e10 } }],
	[[devAccounts.charlie.address], { providers: 1, data: { free: 10e10 } }],
	[[devAccounts.dave.address], { providers: 1, data: { free: 10e10 } }],
	[[devAccounts.eve.address], { providers: 1, data: { free: 10e10 } }],
	],
	},
	})
	/**
	* Get current referendum count i.e. the next referendum's index
	*/
	const referendumIndex = await client.api.query.referenda.referendumCount()
	/**
	* Submit a new referendum
	*/
	const submissionTx = client.api.tx.referenda.submit(
	{
	Origins: 'SmallTipper',
	} as any,
	{
	Inline: client.api.tx.treasury.spendLocal(1e10, devAccounts.bob.address).method.toHex(),
	},
	{
	After: 1,
	},
	)
	const submissionEvents = await sendTransaction(submissionTx.signAsync(devAccounts.alice))
	await client.dev.newBlock()
	// Fields to be removed, check comment below.
	let unwantedFields = /index/
	await checkEvents(submissionEvents, 'referenda')
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot('referendum submission events')
	/**
	* Check the created referendum's data
	*/
	let referendumDataOpt: Option<PalletReferendaReferendumInfoConvictionVotingTally> =
	await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "submitted referendum's data cannot be `None`")
	let referendumData: PalletReferendaReferendumInfoConvictionVotingTally = referendumDataOpt.unwrap()
	// These fields must be excised from the queried referendum data before being put in the test
	// snapshot.
	// These fields contain epoch-sensitive data, which will cause spurious test failures
	// periodically.
	unwantedFields = /alarm|submitted/
	await check(referendumData)
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot('referendum info before decision deposit')
	assert(referendumData.isOngoing)
	// Ongoing referendum data, prior to the decision deposit.
	const ongoingRefPreDecDep: PalletReferendaReferendumStatusConvictionVotingTally = referendumData.asOngoing
	assert(ongoingRefPreDecDep.alarm.isSome)
	const undecidingTimeoutAlarm = ongoingRefPreDecDep.alarm.unwrap()[0]
	const blocksUntilAlarm = undecidingTimeoutAlarm.sub(ongoingRefPreDecDep.submitted)
	// Check that the referendum's alarm is set to ring after the (globally predetermined) timeout
	// of 14 days, or 201600 blocks.
	assert(blocksUntilAlarm.eq(client.api.consts.referenda.undecidingTimeout))
	// The referendum was above set to be enacted 1 block after its passing.
	assert(ongoingRefPreDecDep.enactment.isAfter)
	assert(ongoingRefPreDecDep.enactment.asAfter.eq(1))
	const referendaTracks = client.api.consts.referenda.tracks
	const smallTipper = referendaTracks.find((track) => track[1].name.eq('small_tipper'))!
	assert(ongoingRefPreDecDep.track.eq(smallTipper[0]))
	await check(ongoingRefPreDecDep.origin).toMatchObject({
	origins: 'SmallTipper',
	})
	// Immediately after a referendum's submission, it will not have a decision deposit,
	// which it will need to begin the decision period.
	assert(ongoingRefPreDecDep.deciding.isNone)
	assert(ongoingRefPreDecDep.decisionDeposit.isNone)
	assert(ongoingRefPreDecDep.submissionDeposit.who.eq(encodeAddress(devAccounts.alice.address, addressEncoding)))
	assert(ongoingRefPreDecDep.submissionDeposit.amount.eq(client.api.consts.referenda.submissionDeposit))
	// Current voting state of the referendum.
	const votes = {
	ayes: 0,
	nays: 0,
	support: 0,
	}
	// Check that voting data is empty
	await check(ongoingRefPreDecDep.tally).toMatchObject(votes)
	/**
	* Place decision deposit
	*/
	const decisionDepTx = client.api.tx.referenda.placeDecisionDeposit(referendumIndex)
	const decisiondepEvents = await sendTransaction(decisionDepTx.signAsync(devAccounts.bob))
	await client.dev.newBlock()
	// Once more, fields containing temporally-contigent information - block numbers - must be excised
	// from test data to avoid spurious failures after updating block numbers.
	unwantedFields = /alarm|index|submitted/
	await checkEvents(decisiondepEvents, 'referenda')
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot("events for bob's decision deposit")
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	referendumData = referendumDataOpt.unwrap()
	await check(referendumData)
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot('referendum info post decision deposit')
	assert(referendumData.isOngoing)
	const ongoingRefPostDecDep = referendumData.asOngoing
	// The referendum can only begin deciding after its track's preparation period has elapsed, even though
	// the decision deposit has been placed.
	assert(ongoingRefPostDecDep.deciding.isNone)
	assert(ongoingRefPostDecDep.decisionDeposit.isSome)
	assert(ongoingRefPostDecDep.decisionDeposit.unwrap().who.eq(encodeAddress(devAccounts.bob.address, addressEncoding)))
	assert(ongoingRefPostDecDep.decisionDeposit.unwrap().amount.eq(smallTipper[1].decisionDeposit))
	// The block at which the referendum's preparation period will end, and its decision period will begin.
	const preparePeriodWithOffset = smallTipper[1].preparePeriod.add(ongoingRefPostDecDep.submitted)
	assert(ongoingRefPostDecDep.alarm.isSome)
	// The decision deposit has been placed, so the referendum's alarm should point to that block, at the
	// end of the decision period.
	assert(ongoingRefPostDecDep.alarm.unwrap()[0].eq(preparePeriodWithOffset))
	// Placing a decision deposit for a referendum should change nothing BUT the referendum's
	// 1. deposit data and
	// 2. alarm.
	referendumCmp(ongoingRefPreDecDep, ongoingRefPostDecDep, ['decisionDeposit', 'alarm'])
	/**
	* Wait for preparation period to elapse
	*/
	let refPre = ongoingRefPostDecDep
	let refPost: PalletReferendaReferendumStatusConvictionVotingTally
	for (let i = 0; i < smallTipper[1].preparePeriod.toNumber() - 2; i++) {
	await client.dev.newBlock()
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	referendumData = referendumDataOpt.unwrap()
	assert(referendumData.isOngoing)
	refPost = referendumData.asOngoing
	referendumCmp(refPre, refPost, [], `Failed on iteration number ${i}.`)
	refPre = refPost
	}
	await client.dev.newBlock()
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	const refNowDeciding = referendumDataOpt.unwrap().asOngoing
	unwantedFields = /alarm|submitted|since/
	await check(refNowDeciding)
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot('referendum upon start of decision period')
	const decisionPeriodStartBlock = ongoingRefPreDecDep.submitted.add(smallTipper[1].preparePeriod)
	assert(refNowDeciding.alarm.unwrap()[0].eq(smallTipper[1].decisionPeriod.add(decisionPeriodStartBlock)))
	assert(
	refNowDeciding.deciding.eq({
	since: decisionPeriodStartBlock,
	confirming: null,
	}),
	)
	referendumCmp(refPost!, refNowDeciding, ['alarm', 'deciding'])
	/**
	* Vote on the referendum
	*/
	// Charlie's vote
	const ayeVote = 5e10
	let voteTx = client.api.tx.convictionVoting.vote(referendumIndex, {
	Standard: {
	vote: {
	aye: true,
	conviction: 'Locked3x',
	},
	balance: ayeVote,
	},
	})
	let voteEvents = await sendTransaction(voteTx.signAsync(devAccounts.charlie))
	await client.dev.newBlock()
	unwantedFields = /alarm|when|since|submitted/
	await checkEvents(voteEvents, 'convictionVoting')
	.redact({ removeKeys: unwantedFields, redactKeys: unwantedFields })
	.toMatchSnapshot("events for charlie's vote")
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	referendumData = referendumDataOpt.unwrap()
	await check(referendumData)
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot("referendum info after charlie's vote")
	assert(referendumData.isOngoing)
	const ongoingRefFirstVote = referendumData.asOngoing
	// Charlie voted with 3x conviction
	votes.ayes += ayeVote * 3
	votes.support += ayeVote
	await check(ongoingRefFirstVote.tally).toMatchObject(votes)
	// Check Charlie's locked funds
	const charlieClassLocks = await client.api.query.convictionVoting.classLocksFor(devAccounts.charlie.address)
	const localCharlieClassLocks = [[smallTipper[0], ayeVote]]
	assert(charlieClassLocks.eq(localCharlieClassLocks))
	// , and overall account's votes
	const votingByCharlie: PalletConvictionVotingVoteVoting = await client.api.query.convictionVoting.votingFor(
	devAccounts.charlie.address,
	smallTipper[0],
	)
	assert(votingByCharlie.isCasting, "charlie's votes are cast, not delegated")
	const charlieCastVotes: PalletConvictionVotingVoteCasting = votingByCharlie.asCasting
	// The information present in the `VotingFor` storage item contains the referendum index,
	// which must be removed.
	const unwantedRefIx = new RegExp(`${referendumIndex},`)
	await check(charlieCastVotes.votes[0][1])
	.redact({ removeKeys: unwantedRefIx })
	.toMatchSnapshot("charlie's votes after casting his")
	assert(charlieCastVotes.votes.length === 1)
	assert(charlieCastVotes.votes[0][0].eq(referendumIndex))
	const charlieVotes = charlieCastVotes.votes[0][1].asStandard
	assert(charlieVotes.vote.conviction.isLocked3x && charlieVotes.vote.isAye)
	// After a vote the referendum's alarm is set to the block following the one the vote tx was
	// included in.
	ongoingRefFirstVote.alarm.unwrap()[0].eq(refNowDeciding.deciding.unwrap().since.add(new BN(1)))
	// Placing a vote for a referendum should change nothing BUT:
	// 1. the tally, and
	// 2. its decision period, which at this point should still be counting down.
	referendumCmp(refNowDeciding, ongoingRefFirstVote, ['tally', 'alarm'])
	// Dave's vote
	const nayVote = 1e10
	voteTx = client.api.tx.convictionVoting.vote(referendumIndex, {
	Split: {
	aye: ayeVote,
	nay: nayVote,
	},
	})
	voteEvents = await sendTransaction(voteTx.signAsync(devAccounts.dave))
	await client.dev.newBlock()
	await checkEvents(voteEvents, 'convictionVoting')
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot("events for dave's vote")
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	referendumData = referendumDataOpt.unwrap()
	await check(referendumData)
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot("referendum info after dave's vote")
	assert(referendumData.isOngoing)
	const ongoingRefSecondVote = referendumData.asOngoing
	votes.ayes += ayeVote / 10
	votes.nays += nayVote / 10
	votes.support += ayeVote
	await check(ongoingRefSecondVote.tally).toMatchObject(votes)
	const daveLockedFunds = await client.api.query.convictionVoting.classLocksFor(devAccounts.dave.address)
	const localDaveClassLocks = [[smallTipper[0], ayeVote + nayVote]]
	// Dave voted with `split`, which does not allow expression of conviction in votes.
	assert(daveLockedFunds.eq(localDaveClassLocks))
	// Check Dave's overall votes
	const votingByDave: PalletConvictionVotingVoteVoting = await client.api.query.convictionVoting.votingFor(
	devAccounts.dave.address,
	smallTipper[0],
	)
	assert(votingByDave.isCasting, "dave's votes are cast, not delegated")
	const daveCastVotes: PalletConvictionVotingVoteCasting = votingByDave.asCasting
	await check(daveCastVotes.votes[0][1])
	.redact({ removeKeys: unwantedRefIx })
	.toMatchSnapshot("dave's votes after casting his")
	assert(daveCastVotes.votes.length === 1)
	assert(daveCastVotes.votes[0][0].eq(referendumIndex))
	const daveVote = daveCastVotes.votes[0][1].asSplit
	assert(daveVote.aye.eq(ayeVote))
	assert(daveVote.nay.eq(nayVote))
	// After a vote the referendum's alarm is set to the block following the one the vote tx was
	// included in.
	ongoingRefSecondVote.alarm.unwrap()[0].eq(ongoingRefFirstVote.deciding.unwrap().since.add(new BN(1)))
	// Placing a split vote for a referendum should change nothing BUT:
	// 1. the tally, and
	// 2. its decision period, still counting down.
	referendumCmp(ongoingRefFirstVote, ongoingRefSecondVote, ['tally', 'alarm'])
	// Eve's vote
	const abstainVote = 2e10
	voteTx = client.api.tx.convictionVoting.vote(referendumIndex, {
	SplitAbstain: {
	aye: ayeVote,
	nay: nayVote,
	abstain: abstainVote,
	},
	})
	voteEvents = await sendTransaction(voteTx.signAsync(devAccounts.eve))
	await client.dev.newBlock()
	await checkEvents(voteEvents, 'convictionVoting')
	.redact({ removeKeys: unwantedFields })
	.toMatchSnapshot("events for eve's vote")
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	referendumData = referendumDataOpt.unwrap()
	await check(referendumData).redact({ removeKeys: unwantedFields }).toMatchSnapshot("referendum info after eve's vote")
	assert(referendumData.isOngoing)
	const ongoingRefThirdVote = referendumData.asOngoing
	votes.ayes += ayeVote / 10
	votes.nays += nayVote / 10
	votes.support += ayeVote + abstainVote
	await check(ongoingRefThirdVote.tally).toMatchObject(votes)
	const eveLockedFunds = await client.api.query.convictionVoting.classLocksFor(devAccounts.eve.address)
	const localEveClassLocks = [[smallTipper[0], ayeVote + nayVote + abstainVote]]
	// Eve voted with `splitAbstain`, which does not allow expression of conviction in votes.
	assert(eveLockedFunds.eq(localEveClassLocks))
	// Check Eve's overall votes
	const votingByEve: PalletConvictionVotingVoteVoting = await client.api.query.convictionVoting.votingFor(
	devAccounts.eve.address,
	smallTipper[0],
	)
	assert(votingByEve.isCasting, "eve's votes are cast, not delegated")
	const eveCastVotes: PalletConvictionVotingVoteCasting = votingByEve.asCasting
	await check(eveCastVotes.votes[0][1])
	.redact({ removeKeys: unwantedRefIx })
	.toMatchSnapshot("eve's votes after casting hers")
	assert(eveCastVotes.votes.length === 1)
	assert(eveCastVotes.votes[0][0].eq(referendumIndex))
	const eveVote = eveCastVotes.votes[0][1].asSplitAbstain
	assert(eveVote.aye.eq(ayeVote))
	assert(eveVote.nay.eq(nayVote))
	assert(eveVote.abstain.eq(abstainVote))
	// After a vote, the referendum's alarm is set to the block following the one the vote tx was
	// included in.
	ongoingRefThirdVote.alarm.unwrap()[0].eq(ongoingRefSecondVote.deciding.unwrap().since.add(new BN(1)))
	// Placing a split abstain vote for a referendum should change nothing BUT:
	// 1. the tally, and
	// 2. its decision period, still counting down.
	referendumCmp(ongoingRefSecondVote, ongoingRefThirdVote, ['tally', 'alarm'])
	// Attempt to cancel the referendum with a signed origin - this should fail.
	const cancelRefCall = client.api.tx.referenda.cancel(referendumIndex)
	await sendTransaction(cancelRefCall.signAsync(devAccounts.alice))
	await client.dev.newBlock()
	await checkSystemEvents(client, { section: 'system', method: 'ExtrinsicFailed' }).toMatchSnapshot(
	'cancelling referendum with signed origin',
	)
	// Cancel the referendum using the scheduler pallet to simulate a root origin
	scheduleInlineCallWithOrigin(client, cancelRefCall.method.toHex(), { system: 'Root' })
	await client.dev.newBlock()
	/**
	* Check cancelled ref's data
	*/
	// First, the emitted events
	// Retrieve the events for the latest block
	const events = await client.api.query.system.events()
	const referendaEvents = events.filter((record) => {
	const { event } = record
	return event.section === 'referenda'
	})
	assert(referendaEvents.length === 1, 'cancelling a referendum should emit 1 event')
	const cancellationEvent = referendaEvents[0]
	assert(client.api.events.referenda.Cancelled.is(cancellationEvent.event))
	const [index, tally] = cancellationEvent.event.data
	assert(index.eq(referendumIndex))
	assert(tally.eq(votes))
	// Now, check the referendum's data, post-cancellation
	referendumDataOpt = await client.api.query.referenda.referendumInfoFor(referendumIndex)
	// cancelling a referendum does not remove it from storage
	assert(referendumDataOpt.isSome, "referendum's data cannot be `None`")
	assert(referendumDataOpt.unwrap().isCancelled, 'referendum should be cancelled!')
	const cancelledRef: ITuple<[u32, Option<PalletReferendaDeposit>, Option<PalletReferendaDeposit>]> =
	referendumDataOpt.unwrap().asCancelled
	cancelledRef[0].eq(referendumIndex)
	// Check that the referendum's submission deposit was refunded to Alice
	cancelledRef[1].unwrap().eq({
	who: encodeAddress(devAccounts.alice.address, addressEncoding),
	amount: client.api.consts.referenda.submissionDeposit,
	})
	// Check that the referendum's submission deposit was refunded to Bob
	cancelledRef[2].unwrap().eq({
	who: encodeAddress(devAccounts.bob.address, addressEncoding),
	amount: smallTipper[1].decisionDeposit,
	})
	const testAccounts = {
	charlie: {
	classLocks: charlieClassLocks,
	localClassLocks: localCharlieClassLocks,
	votingBy: votingByCharlie,
	},
	dave: {
	classLocks: daveLockedFunds,
	localClassLocks: localDaveClassLocks,
	votingBy: votingByDave,
	},
	eve: {
	classLocks: eveLockedFunds,
	localClassLocks: localEveClassLocks,
	votingBy: votingByEve,
	},
	}
	// Check that cancelling the referendum has no effect on each voter's class locks
	for (const account of Object.keys(testAccounts)) {
	testAccounts[account].classLocks = await client.api.query.convictionVoting.classLocksFor(
	devAccounts[account].address,
	)
	assert(
	testAccounts[account].classLocks.eq(testAccounts[account].localClassLocks),
	`${account}'s class locks should be unaffected by referendum cancellation`,
	)
	}
	// Check that cancelling the referendum has no effect on accounts' votes, as seen via `votingFor`
	// storage item.
	for (const account of Object.keys(testAccounts)) {
	const postCancellationVoting: PalletConvictionVotingVoteVoting = await client.api.query.convictionVoting.votingFor(
	devAccounts[account].address as string,
	smallTipper[0],
	)
	assert(postCancellationVoting.isCasting, `pre-referendum cancellation, ${account}'s votes were cast, not delegated`)
	const postCancellationCastVotes: PalletConvictionVotingVoteCasting = postCancellationVoting.asCasting
	assert(
	postCancellationVoting.eq(testAccounts[account].votingBy),
	`${account}'s votes should be unaffected by referendum cancellation`,
	)
	await check(postCancellationCastVotes.votes[0][1])
	.redact({ removeKeys: unwantedRefIx })
	.toMatchSnapshot(`${account}'s votes after referendum's cancellation`)
	}
	/**
	* Vote withdrawal transactions, batched atomically.
	*/
	const removeCharlieVote = client.api.tx.convictionVoting.removeVote(smallTipper[0], referendumIndex).method
	const removeDaveVoteAsCharlie = client.api.tx.convictionVoting.removeOtherVote(
	devAccounts.dave.address,
	smallTipper[0],
	referendumIndex,
	).method
	const removeEveVoteAsCharlie = client.api.tx.convictionVoting.removeOtherVote(
	devAccounts.eve.address,
	smallTipper[0],
	referendumIndex,
	).method
	const batchAllTx = client.api.tx.utility.batchAll([
	removeCharlieVote,
	removeDaveVoteAsCharlie,
	removeEveVoteAsCharlie,
	])
	const batchEvents = await sendTransaction(batchAllTx.signAsync(devAccounts.charlie))
	await client.dev.newBlock()
	await checkEvents(batchEvents)
	.redact({ removeKeys: /who/ })
	.toMatchSnapshot('removal of votes in cancelled referendum')
	// Check that each voter's class locks remain unaffected by vote removal - these are subject to a
	// later update.
	//
	// Also check that voting for each account is appropriately empty.
	for (const account of Object.keys(testAccounts)) {
	testAccounts[account].classLocks = await client.api.query.convictionVoting.classLocksFor(
	devAccounts[account].address,
	)
	assert(
	testAccounts[account].classLocks.eq(testAccounts[account].localClassLocks),
	`${account}'s class locks should be unaffected by vote removal`,
	)
	await check(testAccounts[account].classLocks).toMatchSnapshot(
	`${account}'s class locks after their vote's rescission`,
	)
	testAccounts[account].votingBy = await client.api.query.convictionVoting.votingFor(
	devAccounts[account].address,
	smallTipper[0],
	)
	assert(testAccounts[account].votingBy.isCasting)
	const castVotes = testAccounts[account].votingBy.asCasting
	await check(castVotes).toMatchSnapshot(`${account}'s votes after rescission`)
	assert(castVotes.votes.isEmpty)
	}

The lifecycle test uses the SmallTipper governance track, as it is the track with the shortest preparation/decision/confirmation periods.
This enables the test to run through some of these stages and perform verifications, but even with this celerity it is not possible to test the various paths a referendum can take after its decision period elapses:

• if sufficiently supported, a transition to confirmation, and
• if not, failure and impending removal.

Furthermore, it also does not include a test to the execution of the referendum's proposed call (whether noted with a preimage, or submitted inline), which would have prevented, in referenda for the Collectives chain, issue polkadot-fellows/runtimes#614.

This issue is about improving governance E2E test cases to cover:

• referenda submitted from more kinds of OpenGov tracks,
• various possible referendum lifecycle outcomes
  • without needing to wait for full periods (storage injection)