Skip to content

Expose Flaky Test Data via API and Model Context Protocol (MCP) #890

@lucianoliberti

Description

@lucianoliberti

Is your improvement proposal related to a problem? Please describe.

There is a growing requirement to programmatically access insights regarding flaky tests within the CI/CD pipeline. Currently, while flaky test detection may be available within the UI, there is no standardized way to query this data via external interfaces.

The goal is to provide developers and automated agents (via MCP) with the ability to identify, track, and analyze unstable tests without manual dashboard monitoring. Providing this data via API would allow for:

  • Automated Reporting: Integrating flaky test statistics into internal health dashboards.
  • AI-Assisted Maintenance: Enabling LLMs to identify problematic test patterns and suggest fixes based on historical failure data.
  • Workflow Automation: Dynamically adjusting pipeline behavior or notifications based on the known "flakiness" of a specific test suite.

This improvement should focus on creating endpoints that return metadata for tests flagged as flaky, including failure frequency and recent execution history.

Describe the solution you'd like

TBD

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestsolution-tbdSolution information to be defined

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions