Skip to content

Retain failing inputs for regression runs #55

Description

@twof

Context

The corpus refactor (Signal/Evaluator/Ledger → the corpus is now purely the scheduler's retained set, materialized at run-end) removed the path that used to persist failing inputs.

Previously the shrinking plugin emitted a .submitToCorpus(.failure) action: it minimized a failing input and wrote it into the corpus as a .failure-typed entry. Because the corpus is persisted and reloaded as seeds, a failing input would automatically replay on the next run (cheap regression).

That path is now gone:

  • submitToCorpus action + SubmitToCorpusAction removed.
  • CorpusEntry.entryType / CorpusEntry.failure, CorpusEntryType, and FailureInfo removed.
  • The corpus is now exclusively the scheduler's retained set (no plugin-submitted entries, no mid-run writers).

The shrinking plugin still minimizes the failing input, biases mutation toward it (selectForMutation), and records the issue — it just no longer persists it.

What's lost

Minimized failing inputs are no longer saved to disk, so they don't auto-replay as seeds on the next run. Failing inputs are still surfaced at runtime via result.failures (raw, un-minimized); they're just not persisted for regression.

What we want

A first-class mechanism to retain failing inputs for regression, decoupled from the coverage corpus:

  • Separate failures store: persist result.failures (and/or shrunk forms) to their own on-disk location, and load them as seeds on the next run. Keeps failure artifacts out of the scheduler's retained-set corpus (the point of the refactor).
  • Raw vs minimized: decide whether regression replays the raw failing input or the shrunk one (shrunk = faster repro, raw = closer to the discovered case).
  • Schedule bytes: failures found under schedule fuzzing need their schedule bytes persisted (element-0 convention) so the interleaving reproduces.

Explicitly not wanted: routing failures back through the scheduler's pool (conflates the mutation pool with failure-artifact storage; a minimized reproducer is a poor mutation parent).

Acceptance

  • A failing input found in run N is deterministically replayed at the start of run N+1 without re-discovering it.
  • Failure retention does not reintroduce coverage/entry-type bookkeeping into the corpus.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions