Skip to content

chore(windows): Fix tests CS stop reliability LS-381#771

Open
rolysr wants to merge 1 commit intodevfrom
fix-unstable-win-cs-tests
Open

chore(windows): Fix tests CS stop reliability LS-381#771
rolysr wants to merge 1 commit intodevfrom
fix-unstable-win-cs-tests

Conversation

@rolysr
Copy link
Collaborator

@rolysr rolysr commented Feb 27, 2026

Recently, the test test_truncate_retry has been observed to fail on the Windows client CI due to chunkserver stop operations not taking effect immediately. This delay causes the test to become flaky. To mitigate the issue, a short sleep has been added after stopping the chunkserver to ensure that the stop command and its subsequent operations have fully taken effect before the test proceeds.

Additionally, the test test_ec_read_combinations was flaky because insufficient time was given for the file-generate utility to write all file chunk parts. This timing issue also caused intermittent failures on the Windows client CI.

Co-authored-by: Dave dave@leil.io
Signed-off-by: rolysr rolysr@leil.io

@rolysr rolysr requested a review from Copilot February 27, 2026 16:29
@rolysr rolysr self-assigned this Feb 27, 2026
@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request aims to enhance the reliability of system tests on Windows by mitigating timing-related flakiness. It introduces a small delay after chunkserver shutdown operations to ensure that the system state is consistent before proceeding with subsequent test steps, thereby stabilizing the CI pipeline without altering core application logic.

Highlights

  • Flaky Windows CI Tests Addressed: Tests on Windows client CI were observed to fail intermittently due to chunkserver stop commands not taking immediate effect, leading to flakiness.
  • Conditional Sleep Introduced: A short sleep of 0.1 seconds has been added after stopping chunkservers, specifically for Windows systems, to ensure that the stop command and its subsequent operations have fully taken effect before test execution continues.
Changelog
  • tests/test_suites/ShortSystemTests/test_ec_read_combinations.sh
    • Added a conditional sleep of 0.1 seconds after stopping chunkservers when running on a Windows system.
  • tests/test_suites/ShortSystemTests/test_truncate_retry.sh
    • Inserted conditional sleeps of 0.1 seconds after each instance of stopping a chunkserver when executing on a Windows system.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@rolysr
Copy link
Collaborator Author

rolysr commented Feb 27, 2026

@gemini-code-assist review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request effectively addresses Windows CI test flakiness by introducing a short, conditional sleep after chunkserver stop operations. This is a practical solution for improving test stability. While fixed delays can sometimes mask underlying timing issues or introduce unnecessary slowdowns, for test scripts, this approach is often acceptable. To enhance maintainability and clarity, consider defining the sleep duration as a named variable.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses test reliability issues on Windows by adding short sleep delays (0.1 seconds) after chunkserver stop commands in two specific test files. The changes target flaky tests that were failing on Windows CI due to the chunkserver stop command not taking effect immediately.

Changes:

  • Added Windows-specific sleep delays after saunafs_chunkserver_daemon stop commands in two test files
  • Used the established is_windows_system function to conditionally apply the delays only on Windows systems

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/test_suites/ShortSystemTests/test_truncate_retry.sh Added 0.1s sleep after two chunkserver stop commands (lines 16-18 and 26-28)
tests/test_suites/ShortSystemTests/test_ec_read_combinations.sh Added 0.1s sleep inside loop after each chunkserver stop command (lines 18-20)

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a pragmatic fix for flaky tests on Windows by adding a short sleep after stopping a chunkserver. This should help prevent race conditions where tests proceed before the stop operation is fully complete. My review includes a suggestion to refactor duplicated code in one of the test scripts to improve maintainability.

@rolysr rolysr changed the title chore(windows): Fix tests CS stop reliability chore(windows): Fix tests CS stop reliability LS-381 Feb 27, 2026
@rolysr rolysr marked this pull request as ready for review February 27, 2026 16:38
@rolysr rolysr force-pushed the fix-unstable-win-cs-tests branch 2 times, most recently from 7a3fab4 to 93fcfb5 Compare February 27, 2026 17:27
Recently, the test test_truncate_retry has been observed to fail on
the Windows client CI due to chunkserver stop operations not taking
effect immediately. This delay causes the test to become flaky. To
mitigate the issue, a short sleep has been added after stopping the
chunkserver to ensure that the stop command and its subsequent
operations have fully taken effect before the test proceeds.

Additionally, the test test_ec_read_combinations was flaky because
insufficient time was given for the file-generate utility to write
all file chunk parts. This timing issue also caused intermittent
failures on the Windows client CI.

Co-authored-by: Dave <dave@leil.io>

Signed-off-by: rolysr <rolysr@leil.io>
@rolysr rolysr force-pushed the fix-unstable-win-cs-tests branch from 93fcfb5 to 7ca3b75 Compare February 27, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants