Skip to content

Create complete tests for syntax #843

@catamorphism

Description

@catamorphism

This is from https://github.com/unicode-org/message-format-wg/wiki/Things-That-Need-Doing . I didn't see an existing issue for it.

Full test coverage is probably impossible given the size of the grammar. But there are a couple possibilities:

  1. Manually generate tests from the grammar, with a limited search depth (i.e. expand every production up to a certain depth and write a test that fits it)
  2. Use an exhaustive test generator tool like abnfgen, Eusthasius or Gramtest to generate tests.
  3. Use a fuzzer like abnffuzzer and extract interesting tests to include in the test suite.

As far as option 2, with a few modifications, I was able to run the grammar through abnfgen and Eusthasius. The problem is that both of them generate tests that resemble line noise, suggesting that fuzzing might be preferable.

Option 1 has the advantage that it's easy to write readable, minimal test cases. Of course the disadvantage is that it's tedious.

Any suggestions are welcome.

Metadata

Metadata

Assignees

Labels

blocker-candidateThe submitter thinks this might be a block for the next releasetest-suiteIssue pertains to tests

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions