Skip to content

Feat: Multi-Release Historical Analysis and Regression Detection #33

@knabben

Description

@knabben

Description

The handbook requires flagging regressions as close to the source as possible (when offending code was merged). Currently, signalhound does not track test history or detect regressions.

Requirements

  • Track test history across multiple releases
  • Detect regressions (failures after code merge)
  • Identify PRs that introduced failures
  • Link failures to specific commits/PRs
  • Generate regression reports

Acceptance Criteria

  • Test history is tracked across releases
  • Regressions are detected (failures after merge)
  • PRs that introduced failures are identified
  • Failures are linked to commits/PRs
  • Regression reports are generated
  • Unit tests for regression detection
  • Integration tests with historical data

Implementation Notes

  • Query TestGrid for historical test data
  • Query GitHub API for PR/commit information
  • Match test failures to PR merge dates
  • Generate regression timeline
  • Support --detect-regressions command

Metadata

Metadata

Assignees

No one assigned

    Labels

    lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions