Skip to content

Conversation

@satyakigh
Copy link
Collaborator

@satyakigh satyakigh commented Dec 9, 2025

Key Changes

Test Fixture Consolidation

  • Moved repeated mock schema creation from individual test cases to shared fixtures at the describe/module level
  • Introduced reusable schema constants (defaultSchemas, s3Schemas, emptySchemas, fourSchemas, etc.) to eliminate redundant combinedSchemas() calls
  • Extracted custom schema definitions (customTypeSchema, customTypeWithNestedIdSchema, etc.) to module-level constants

Mock Setup Simplification

  • Removed repetitive createMockResourceSchemas() and setupMockSchemas() helper functions in favor of pre-configured test fixtures
  • Consolidated beforeEach setup by initializing common mocks once at the describe level
  • Simplified createMockComponents() usage by passing pre-configured schema retrievers

Test Cleanup

  • Removed flaky timing-based tests in PyodideWorkerManager.test.ts that tested exponential backoff delays, jitter, and timeout behavior
  • Replaced with a single focused test that verifies correct config is passed to retryWithExponentialBackoff
  • Deleted wheel-download.test.ts which had external dependencies on Python/pip

Modified Test Files - Comparison

Test File Tests Before Tests After Time Before Time After Δ Time Δ %
ResourceEntityCompletionProvider.test.ts 13 13 616ms 23ms -593ms -96.3%
ResourcePropertyCompletionProvider.test.ts 62 62 4,353ms 1,487ms -2,866ms -65.8%
ResourceStateCompletionProvider.test.ts 17 17 4,169ms 23ms -4,146ms -99.4%
StackHandler.test.ts 42 42 1,966ms 178ms -1,788ms -90.9%
RelatedResourcesSnippetProvider.test.ts 10 10 2,955ms 41ms -2,914ms -98.6%
ResourceStateImporter.test.ts 36 36 14,972ms 65ms -14,907ms -99.6%
PyodideWorkerManager.test.ts 29 25 3,003ms 1,976ms -1,027ms -34.2%

Deleted Test Files

Test File Tests Time Reason
wheel-download.test.ts 2 16,647ms External Python/pip dependencies
standalone.test.ts 1 4ms Removed

Summary

Category Time Saved
Modified files -28,241ms
Deleted files -16,651ms
Total -44,892ms (~45s)

Overall Test Suite Summary

Metric Before After Change
Test Files 210 208 -2
Test Execution 123.25s 81.34s -41.91s (-34%)

Flaky test - https://github.com/aws-cloudformation/cloudformation-languageserver/actions/runs/20047430606/job/57495971389?pr=294

@satyakigh satyakigh marked this pull request as ready for review December 9, 2025 01:08
@satyakigh satyakigh requested a review from a team as a code owner December 9, 2025 01:08

test('should use exponential backoff delays', async () => {
// Create a worker manager with specific retry settings
const retryWorkerManager = new PyodideWorkerManager(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kddejong
kddejong previously approved these changes Dec 9, 2025
};
const s3BucketSchema = new ResourceSchema(Schemas.S3Bucket.contents);
const fourSchemas = combinedSchemas([
Schemas.S3Bucket,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is s3Bucket schema repeated

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what combineSchema is doing. It should be renamed to indicate it copies/renames/patches

const testSchemas = combinedSchemas([Schemas.S3Bucket, Schemas.EC2Instance, Schemas.LambdaFunction]);

beforeEach(() => {
mockComponents.schemaRetriever.getDefault.reset();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this reset is no longer needed

@satyakigh satyakigh merged commit eae1d57 into main Dec 9, 2025
16 checks passed
@satyakigh satyakigh deleted the flaky-tests branch December 9, 2025 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants