Skip to content

Fix flaky CLI integration test with concurrent execution #331

@milliondreams

Description

@milliondreams

Problem

The CLI integration test in crates/mandrel-mcp-th/tests/cli_run.rs is flaky and fails when run as part of the full test suite but passes when run individually.

Details

  • Test: test_cli_run_command_with_working_mcp_server
  • Issue: The test fails with exit code 1 showing "Total Tests: 6, Passed: 5, Failed: 1"
  • Current Status: Temporarily ignored with #[ignore] attribute
  • Root Cause: Likely resource contention or state interference between concurrent tests

Expected Behavior

The test should:

  1. Pass consistently whether run individually or as part of the full test suite
  2. Handle concurrent execution properly
  3. Not interfere with other MCP server tests

Error Details

When the test fails, it shows:

  • Exit code: 1
  • Output: "Total Tests: 6, Passed: 5, Failed: 1"
  • One of the underlying filesystem operations in the MCP server is failing

Suggested Fixes

  1. Isolation: Ensure each test uses unique directories/ports to prevent conflicts
  2. Resource Cleanup: Improve cleanup of MCP server processes and temporary files
  3. Retry Logic: Add retry mechanism for flaky filesystem operations
  4. Test Ordering: Consider test dependencies and execution order
  5. Concurrent Limits: Limit concurrent MCP server instances

Priority

Medium - This affects CI/CD reliability but doesn't block development.

References

  • Test file: crates/mandrel-mcp-th/tests/cli_run.rs:58
  • Related to MCP filesystem server integration testing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions