Skip to content

Conversation

@Hweinstock
Copy link
Contributor

@Hweinstock Hweinstock commented May 8, 2025

Problem

This test has been very flaky, and constantly failing CI (#7187).
This test is run for almost 40 different cases, and does the following:

  • create a text document
  • write it to the filesystem.
  • run the check on the text document.

One of the core issues is that when we write the text document to the filesystem, we create a new test workspace folder for each case. This involves creating a directory with a random id for each of the almost 40 different cases.

These excessive file system operations could be leading to the flakiness in CI.

Solution

  • reuse the same test workspace folder across all test cases. This should cut the file systems operations in half.

Future Work

  • If this doesn't reduce flakiness, we could avoid writing the text document to the fs since its not needed by the underlying test, but this involves mocking the text document which is undesirable imo.

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

@justinmk3
Copy link
Contributor

we could avoid writing the text document to the fs since its not needed by the underlying test, but this involves mocking the text document which is undesirable imo.

if vscode requires a text document to be on the filesystem, could the test system create a long-lived text document and re-use it for each test ?

@Hweinstock Hweinstock marked this pull request as ready for review May 8, 2025 15:35
@Hweinstock Hweinstock requested a review from a team as a code owner May 8, 2025 15:35
@Hweinstock Hweinstock merged commit 9b873dc into aws:master May 8, 2025
31 checks passed
@Hweinstock Hweinstock deleted the flakyDocTest branch May 8, 2025 15:37
@Hweinstock
Copy link
Contributor Author

Yeah that seems like a nice next step if this continues to be flaky.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants