[vitest-pool-workers] Workflows test support #10494

pombosilva · 2025-08-28T19:39:16Z

Fixes WOR-355 (related to WOR-854, WOR-855, WOR-856, WOR-857, WOR-858, WOR-859, WOR-860, WOR-861 and WOR-862).

What:

New testing features in vitest-pool-workers:

This provides two APIs for testing Workflows with vitest-pool-workers. These are the main entry points into changing Workflow instances behavior during tests:

introspectWorkflowInstance(env.WORKFLOW, "id123"): when the instance id is known beforehand, useful for Workflow unit tests
- returns WorkflowInstanceIntrospector that allows to call:
  - modify(fn): function to where modifications are applied (calls rpc functions to workflows-shared, where changes are made to DO storage)
  - waitFoRStepResult(step, result): waits until a step resolves into a result or rejects into an error
  - waitForStatus(status): waits until the provided workflow instance reaches a given state
  - dispose(): disposes the workflow instance, so state does not get persisted across tests using the same workflow instance (it is actually deleting the DO storage and then aborting it)
introspectWorkflow(env.WORKFLOW): when the instance id is not known beforehand, useful for integration tests
- proxies the Workflow to intercept .create() and .createBatch() calls
- returns WorkflowIntrospector that allows to call:
  - modifyAll(fn): same as modify(fn) but applies modifications to all workflow instances created after the introspectWorkflow call until the introspector cleanUp call
  - get(): returns a list of WorkflowInstanceIntrospector of all created workflow instances after the introspectWorkflow call
  - dispose(): disposes the introspector, meaning that no more Workflow calls will be intercepted, and also disposes all workflow instance introspector created

Both APIs support disposal of their returned objects by using the using keyword. This will automatically call cleanUp() at the end of the code block (at the end of the test).

How:

Miniflare:

~~Added an unsafeScriptName to DurableObjectsOptionsSchema of DO plugin, to allow defining the DO serviceName without the 'core:user:' prefix.~~

workflows-shared:

[new] Added unsafe functions to Workflow binding to be able to access Engine DOs from vitest-pool-workers
Added an RpcTarget class with the modifier functions that update the DO storage
Added logic into the Engine/Context to verify the DO storage to see if behavior needs to be changed accordingly

vitest-pool-workers:

~~For each Workflow binding present in a wrangler file, added a DO namespace binding to Runner Worker with unsafeScriptName (these are the bindings to the Engines DOs)~~
Changed the bundle.mjs file (with @edmundhung help) to make it possible importing from worklfows-shared (to avoid type duplication) by using explicit external list to customize what get bundled
Added workflows file to src/worker with the new APIs
Exposed the new APIs and necessary interfaces with some documentation

Fixtures:

Updated the tests with the new APIs in vitest-pool-workers-examples workflows

Potential problems:

Changes to files after running tests:

If I successfully run some tests with these new APIs, then make a change to the test file (can as little as just adding a space) and then save it (ctrl+s), the tests will automatically run again (if the vitest interactive process on the terminal does not get killed) and they will fail.
I suspect this is due to the instance cleanUp() function that is effectively aborting the instances DOs, and whenever the tests try to run again in the same vitest process lifetime, it does not construct the DOs again.

Workerd errors appear on terminal:

I've noticed that if I try to run a test that makes an error be thrown, workerd prints it into the console. Let's say I'm doing a test where I am making a step fail (through mockStepError), since the step failed and an error was thrown, workerd will print that out. This might be normal? The tests run accordingly, but can be confusing to see those logs appearing.

Tests
- Tests included: in fixtures
- Tests not necessary because:
Public documentation
- Cloudflare docs PR(s): [ Workers/ Workflows ] Add Workflows test support APIs cloudflare-docs#24832
- Documentation not necessary because:
Wrangler V3 Backport
- Wrangler PR:
- Not necessary because: Workflows were not GA in v3

changeset-bot · 2025-08-28T19:39:20Z

🦋 Changeset detected

Latest commit: 1adf6bf

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
miniflare	Patch
@cloudflare/vitest-pool-workers	Minor
@cloudflare/pages-shared	Patch
@cloudflare/vite-plugin	Patch
wrangler	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-08-28T19:42:09Z

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@10494

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@10494

miniflare

npm i https://pkg.pr.new/miniflare@10494

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@10494

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@10494

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@10494

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@10494

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@10494

wrangler

npm i https://pkg.pr.new/wrangler@10494

commit: 1adf6bf

edmundhung

Great work! This is looking good so far! I don't have a deep understanding of the workflow-shared package, so it would be good for the workflow team to take a closer look at those changes.

I will give it another pass later this week 👍🏼

.changeset/public-women-exist.md

fixtures/vitest-pool-workers-examples/workflows/src/index.ts

fixtures/vitest-pool-workers-examples/workflows/test/env.d.ts

fixtures/vitest-pool-workers-examples/workflows/test/integration.test.ts

packages/vitest-pool-workers/src/worker/workflows.ts

packages/workflows-shared/src/context.ts

penalosa

Two top level comments ans some initial review comments:

Is introspect the right name for this? I feel like inspect or mock would be simpler and more clearly represent what this does.
We're building a lot of custom APIs here. To what extent have we considered leaning into Vitest's mocking support instead of/in addition to some of these custom mocking APIs?

.changeset/public-women-exist.md

fixtures/vitest-pool-workers-examples/workflows/test/integration.test.ts

fixtures/vitest-pool-workers-examples/workflows/test/unit.test.ts

penalosa · 2025-09-04T11:55:42Z

packages/vitest-pool-workers/scripts/bundle.mjs

+	external: [
+		// Node.js built-ins (handled automatically by esbuild but listed for completeness)
+		"node:*",
+		"cloudflare:*",
+		"workerd:*",
+		// Virtual/runtime modules
+		"__VITEST_POOL_WORKERS_DEFINES",
+		"__VITEST_POOL_WORKERS_USER_OBJECT",
+		// All npm packages (previously handled by packages: "external")
+		"birpc",
+		"cjs-module-lexer",
+		"devalue",
+		"miniflare",
+		"semver",
+		"semver/*",
+		"wrangler",
+		"zod",
+		"undici",
+		"undici/*",
+		// Peer dependencies
+		"vitest",
+		"vitest/*",
+		"@vitest/runner",
+		"@vitest/snapshot",
+		"@vitest/snapshot/*",
+	],


Could you expand on the motivation for this change?

We couldn't import from workflows-shared due to bundling issues, and @edmundhung helped with this workaround. We just did not want to be copying types from workflows-shared into vitest-pool-workers, but if this change is not ideal I guess we could revert into copying what we need.

For context, @cloudflare/workflows-shared wasn't built or published. The vitest-pool-workers build script sets packages: "external", which marks all dependencies as external. So the only way I can see to bundle a dependency here is to remove that setting and provide an explicit external list.

penalosa · 2025-09-04T12:01:10Z

packages/vitest-pool-workers/src/pool/index.ts

+	// Add Workflows Engines DOs bindings to the Runner Worker
+	for (const value of Object.values(runnerWorker.workflows ?? {})) {
+		const engineName = `${WORKFLOW_ENGINE_BINDING}${value.name.toUpperCase()}`;
+		runnerWorker.durableObjects[engineName] = {
+			className: "Engine",
+			unsafeScriptName: `workflows:${value.name}`,
+			unsafeUniqueKey: `miniflare-workflows-${value.name}`,
+		};
+	}


I'm a bit worried about this strategy of connecting additional DOs to the runner worker in order to support accessing the workflow engine DO. Did we consider adding an unsafe method to a workflow binding to access the underlying engine instead? i.e. env.WORKFLOW.unsafeGetEngine()?

I think we considered that at first and it did not work - we can't serialize DOs. So, this was the strategy that worked and we all agreed that it was ok to use.

Just discussed with @pombosilva and we are looking into exposing some engine methods through the workflow bindings directly.

Yes! Done 👍 No more Engine DO bindings to the runner worker

pombosilva · 2025-09-05T10:22:25Z

Two top level comments ans some initial review comments:

Is introspect the right name for this? I feel like inspect or mock would be simpler and more clearly represent what this does.

It was the name we decided as a team!
inspect is not the best name because we can change the instance behavior
mock is not the best fit because we could just want to wait for step results and don't change the instance behavior at all

We're building a lot of custom APIs here. To what extent have we considered leaning into Vitest's mocking support instead of/in addition to some of these custom mocking APIs?

We are able to make the workflow instances explicitly change their behavior. Without manipulating the Engine DO (that users shouldn't even have to worry about that it existis or not), users can't make any mocks

packages/workflows-shared/src/context.ts

LuisDuarte1

huge kudos on this 🚀 - just a few comments below

LuisDuarte1 · 2025-09-08T20:39:14Z

fixtures/vitest-pool-workers-examples/workflows/test/integration.test.ts

+	// This example shows how to implicitly cleanUp Workflow instances
+
+	// CONFIG with `using` to ensure Workflow instances cleanup:
+	await using introspector = await introspectWorkflow(env.MODERATOR);


non-blocking: is await using needed? I guess it symbolizes Symbol.asyncDispose - but I wonder if there's a good way to doing it "synchronously" to have better syntax/DX? Maybe the runner DO can store the list of disposes and dispose it at the end of the test?

@edmundhung can we do this?

Decided on not worrying about this for now. Will be an improvement to be done later

I actually prefer await using for stuff like this because I've found them to be more reliable than afterEach hooks (which I'm guessing would be used in the sync version proposed?)

By reliable, I mean I've had tests hang/give weird errors when doing certain things in afterEach that went away when I switched to await using

The problem with afterEach is that it would break with isolated storage - without isolated storage works just fine. Isolated storage makes its assertions at the end of each test and therefore we need to enforce dispose before the test ends to avoid errors

TIL! Thank you for explaining

packages/vitest-pool-workers/src/worker/workflows.ts

packages/workflows-shared/src/modifier.ts

packages/workflows-shared/src/engine.ts

fixtures/vitest-pool-workers-examples/workflows/test/integration.test.ts

fixtures/vitest-pool-workers-examples/workflows/test/unit.test.ts

packages/vitest-pool-workers/src/worker/workflows.ts

packages/vitest-pool-workers/types/cloudflare-test.d.ts

packages/workflows-shared/src/binding.ts

packages/workflows-shared/src/engine.ts

Co-authored-by: Somhairle MacLeòid <[email protected]>

@ts-expect-error

* Add Workflow instances DOs (Engines) to vitest-pool-workers worker * Get waitUntil working * skip sleep * skip sleep works now * InstanceModifier is now inside worflows-shared with direct access to the engine * add dynamic engine binding * add forceEventTimeout test support * Strip out engine bindings from user facing env * Prevent waitUntil from crashing tests after 5s * Add mockEvent * fix: prevent forceEventTimeot from not working if a mockEvent gets called afterwards * Add mockStepResult * Allow list of sleep steps in disableSleeps() * Add mockStepError and cleanUp. Improve waitForStepResult and waitForStatus * Add forceStepTimeout * Make mockEvent not crash if instance has not been created yet * Update abort for testing * prevent the same step from being mocked multiple times * check edge cases for wairForStatus - prevents user from waiting indefinitely * Add workflows plugin name to loopback * Fix force step timeout * Make it possible to import from workflows-shared * Allow step errors and timeouts to be set and be performed in the correct order * Add introspectWorkflow API with proxies * Expose APIs with some documentation * Add workflow tests with introspection * Fix test * Remove Vite comment * Fixing documentation * fixtures READMEs updated * Add changeset * Prettier fix * Update bundle.mjs file * Put workflows-shared as a devDependency * Change waitUntil timeouts to 1 second * Fix createBatch interception from calling this.create * Add review comment suggestions * Apply suggestions from code review - changelog file Co-authored-by: Somhairle MacLeòid <[email protected]> * Add review suggestions (miniflare do plugin, dispose introspectors) * Support isolated storage and remove Engine DO bindings to runnerWorker * Update changeset * Add team review suggestions * Update fixtures example to match a simple workflow example * Remove @ts-expect-error and add some documentation * Change cleanup to dispose. Apply suggestions. --------- Co-authored-by: Sid Chatterjee <[email protected]> Co-authored-by: Somhairle MacLeòid <[email protected]>

workers-devprod added this to workers-sdk Aug 28, 2025

github-project-automation bot moved this to Untriaged in workers-sdk Aug 28, 2025

pombosilva changed the title ~~Osilva/vitest pool workers workflows support~~ [vitest-pool-workers] Workflows test support Aug 28, 2025

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch from eac0d1c to a3f94a3 Compare August 29, 2025 07:54

pombosilva marked this pull request as ready for review August 29, 2025 10:48

pombosilva requested review from a team as code owners August 29, 2025 10:48

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch 2 times, most recently from 3c4bf14 to a671b96 Compare September 2, 2025 09:56

edmundhung reviewed Sep 2, 2025

View reviewed changes

penalosa reviewed Sep 4, 2025

View reviewed changes

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch 4 times, most recently from c51a39e to bb01a01 Compare September 5, 2025 16:49

avenceslau reviewed Sep 8, 2025

View reviewed changes

packages/workflows-shared/src/context.ts Outdated Show resolved Hide resolved

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch from db80f02 to 45b9e7a Compare September 8, 2025 20:05

pombosilva requested review from avenceslau, edmundhung and penalosa September 8, 2025 20:41

LuisDuarte1 reviewed Sep 8, 2025

View reviewed changes

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch from 65c1975 to 401fb52 Compare September 9, 2025 09:37

edmundhung reviewed Sep 9, 2025

View reviewed changes

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch 3 times, most recently from 2e5c260 to 4ac6f44 Compare September 10, 2025 08:23

edmundhung removed the request for review from penalosa September 10, 2025 12:33

pombosilva and others added 21 commits September 11, 2025 10:08

Expose APIs with some documentation

6bc30cc

Add workflow tests with introspection

c4e4453

Fix test

b42a6c3

Remove Vite comment

c80393b

Fixing documentation

70e16ca

fixtures READMEs updated

e46aa31

Add changeset

6e1f9b8

Prettier fix

5d1670f

Update bundle.mjs file

d3d6507

Put workflows-shared as a devDependency

37c97a6

Change waitUntil timeouts to 1 second

f229505

Fix createBatch interception from calling this.create

ea400d8

Add review comment suggestions

b37be40

Apply suggestions from code review - changelog file

801edc5

Co-authored-by: Somhairle MacLeòid <[email protected]>

Add review suggestions (miniflare do plugin, dispose introspectors)

0f51b34

Support isolated storage and remove Engine DO bindings to runnerWorker

c96da01

Update changeset

040aca7

Add team review suggestions

71aeb7a

Update fixtures example to match a simple workflow example

606ef2a

Remove @ts-expect-error and add some documentation

5012fc4

Change cleanup to dispose. Apply suggestions.

1adf6bf

pombosilva force-pushed the osilva/vitest-pool-workers-workflows-support branch from 5b9e9a0 to 1adf6bf Compare September 11, 2025 09:09

penalosa merged commit e2b838f into cloudflare:main Sep 11, 2025
51 of 59 checks passed

github-project-automation bot moved this from Approved to Done in workers-sdk Sep 11, 2025

workers-devprod mentioned this pull request Sep 11, 2025

Version Packages #10590

Merged

This was referenced Sep 11, 2025

Update changeset #10619

Merged

Update nice-carrots-sink.md #10620

Merged

lrapoport-cf mentioned this pull request Oct 6, 2025

Add tests for workflows #7056

Closed

pyropy mentioned this pull request Oct 15, 2025

refactor: Extract workflows into a package filbeam/worker#357

Merged

[vitest-pool-workers] Workflows test support #10494

[vitest-pool-workers] Workflows test support #10494

Uh oh!

Conversation

pombosilva commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What:

New testing features in vitest-pool-workers:

How:

Miniflare:

workflows-shared:

vitest-pool-workers:

Fixtures:

Potential problems:

Changes to files after running tests:

Workerd errors appear on terminal:

Uh oh!

changeset-bot bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edmundhung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

penalosa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pombosilva commented Sep 5, 2025

Uh oh!

Uh oh!

LuisDuarte1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jahands Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pombosilva commented Aug 28, 2025 •

edited

Loading

changeset-bot bot commented Aug 28, 2025 •

edited

Loading

pkg-pr-new bot commented Aug 28, 2025 •

edited

Loading

jahands Sep 10, 2025 •

edited

Loading