feat(test-runner): continue enhanced retry options (follow-up to #8812) #9370
Conversation
…nhanced test retry control This adds three new retry options: - retryDelay: milliseconds to wait between retry attempts - retryCondition: string pattern or function to conditionally retry based on error - retryStrategy: 'immediate' (default), 'test-file', or 'deferred' retry timing
…bject Refactor retry-related options (retry, retryDelay, retryCondition, retryStrategy) into a single retry configuration object. This provides a cleaner API where retry can be either a number (for simple retry count) or an object with count, delay, condition, and strategy properties. This change improves type safety and makes the retry configuration more intuitive while maintaining backward compatibility with numeric retry values.
Group deferred tests by file and setup/cleanup file context before running retries. This ensures snapshot state is correctly initialized. Also update file result state after deferred retries complete.
✅ Deploy Preview for vitest-dev ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Hi @AriPerkkio, can you please review this PR? |
| - When a **function**, it receives the error and returns a boolean | ||
|
|
||
| > [!WARNING] | ||
| When defining `condition` as a function, it must be done in a test file directly, not in `vitest.config.ts` (configurations are serialized for worker threads). |
There was a problem hiding this comment.
| When defining `condition` as a function, it must be done in a test file directly, not in `vitest.config.ts` (configurations are serialized for worker threads). | |
| ::: warning | |
| When defining `condition` as a function, it must be done in a test file directly, not in `vitest.config.ts` (configurations are serialized for worker threads). | |
| ::: |
packages/runner/src/run.ts
Outdated
| const shouldRetry = shouldRetryTest(test, test.result.errors) | ||
|
|
||
| if (!shouldRetry) { | ||
| // Error doesn't match retry condition, stop retrying |
There was a problem hiding this comment.
Can we remove the AI generated comments that don't explain anything? I feel like they just pollute the code (I would remove most of the non jsdoc comments in this PR)
packages/runner/src/run.ts
Outdated
| /** | ||
| * Determines if a test should be retried based on its retryCondition configuration | ||
| */ | ||
| function shouldRetryTest(test: Test, errors: TestError[] | undefined): boolean { |
There was a problem hiding this comment.
This seems a little bit weird to me because it's applied after the retryCount > retry is checked, so maybe the name should be a bit different? (passesRetryCondition or something) On the !condition line I thought that we always just retry all tests until the pass
| const viteConfig = project._vite?.config | ||
| const optimizer = config.deps?.optimizer || {} | ||
|
|
||
| // Handle retry configuration serialization |
There was a problem hiding this comment.
I think this should actually be done in resolveConfig once, not for every serializeConfig call
There should also be a test for this (see failures tests in test/cli)
There was a problem hiding this comment.
Since I'm intentionally passing an invalid condition type (a function) in the config fixture, TypeScript is complaining. Is it ok to add // @ts-expect-error like this:
export default defineConfig({
test: {
retry: {
count: 3,
// @ts-expect-error
condition: () => true
}
}
})Or did I misunderstand you?
There was a problem hiding this comment.
I don't know what you are referring to, I am saying the code from this file should be moved to resolveConfig.ts
There was a problem hiding this comment.
Yes, I moved the code to resolveConfig.ts as requested. I'm just creating a test to verify this change as requested.
There was a problem hiding this comment.
Ah, sorry. Yes, it is ok to have a ts-expect-error in the test
4 participants
Description
Follow-up to #8812. Align retry types with serialization boundaries, pass
TestErrordirectly to retry condition, useSerializableRetryfor reporter/runtime-facing data, and sanitize function conditions before reporters. Keeps function conditions usable in test files (workers) while ensuring reporter/config data stays serializable.Resolves #8482
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
pnpm-lock.yamlunless you introduce a new test example.Tests
pnpm test:ci.Documentation
pnpm run docscommand.Changesets
feat:,fix:,perf:,docs:, orchore:.