Skip to content

Improve Reftests #146

@hexaeder

Description

@hexaeder

In #145, I just needed to update a bunch of reftest images again. We somehow need to change how we handle them. Problems:

  1. All assets are stored in the git repo. This is not ideal since the git repo will grow over time.
  2. Currently, the reftests are numbered based on their ordering in the containing file; therefore, if you want to introduce another reference test in between, all the following tests change their name.
  3. There is no "interactive" way of going through the tests and accepting updates.

On the other hand, I like the simplicity of the system, and I like that I only need to mark plot commands in the docs with @reftest to test them. Some time ago, I created a experimental reftest package based on those ideas in GraphMakie, https://github.com/hexaeder/PlotReferenceTests.jl which solves 2 by changing the syntax to something like

@reftest "reference name" plot(...) 

Problem 3 is also easy to solve by implementing some tooling.
However, I don't really know how to deal with 1, which blocks further improvements. Ideas?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions