Skip to content

Support a "known issue" test result type #226

@echeran

Description

@echeran

We should add a "known issue" result type. It would correspond to when a test input for a platform/library matches criteria for an existing bug filed in the upstream project of the platform.

The benefit of having the result type is to better categorize / triage the results of our executor implementations, as we continue to update and improve them. Currently, we have test failures, and those failures could be the result of: the library (unknown bug), the library (known bug), or a deficiency in our executor. Sometimes, a known issue causes a systematic set of many test failures.

For the purposes of our project, it would help instill confidence in our self-assessment of correct executor implementation work to not include known bugs in the set of test failures.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions