Skip to content

Commit 05ac34e

Browse files
committed
Implement new method for writing plot tests
This commit introduces the a new way of writing plot tests where a sample plot is recorded and compared against the current rendering of the plot. These types of tests have the ability to automatically detect changes in the way the plots are rendered. This commit only adds tests for changes in #5 and #59, since these were recently merged and had no tests, as well as #55 since this pull request used 3D plots, and I wanted to validate the method for 3D plots as well. Instead of comparing bitmap or vector images, plots are rendered using `plot/dc` or `plot3d/dc` to a special `record-dc%` object and the draw steps are written to a file. When running the tests, the new set of draw steps are generated and compared to the saved ones. The `record-dc%` used for drawing is updated to report consistent values for text dimensions, regardless of the font -- this allows the data files to be used across different platforms which would generate slightly different plot images due to different fonts being available. In addition to this, a sample PNG image is saved for each plot -- this allows the user to manually run the tests and having something to compare the result. The verification steps are automated in `check-draw-steps` and `check-draw-steps3d` such that, when a verification fails, a new set of draw steps are written to file, as well as a new sample image. The GitHub Actions `ci.yml` has been updated to upload these files as an artifact, so we have the data to analyze failed runs on GitHub Actions. The test will work fine without this feature, so they can be run on DrDr or the users computer.
1 parent 74c382f commit 05ac34e

18 files changed

+264
-20
lines changed

.github/workflows/ci.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,13 @@ jobs:
4444

4545
- run: sudo raco pkg install --batch --auto plot-test
4646
- run: sudo xvfb-run raco test --no-run-if-absent --package plot-test
47+
48+
# If any of the plot-test tests failed, they will generate new draw step
49+
# files and sample images. Upload these as an Github Actions Artifact,
50+
# so they can be inspected by the developer.
51+
52+
- uses: actions/upload-artifact@v2
53+
if: failure()
54+
with:
55+
name: 'Updated Data Files'
56+
path: plot-test/plot/tests/auto/data/new-*

plot-test/plot/tests/PRs/55.rkt

Lines changed: 0 additions & 20 deletions
This file was deleted.

plot-test/plot/tests/auto/data/pr5-data.rktd

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.
37.8 KB
Loading
378 KB
Loading

plot-test/plot/tests/auto/data/pr55-parametric-polar3d-data.rktd

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.
292 KB
Loading

plot-test/plot/tests/auto/data/pr55-parametric-suface3d-data.rktd

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.
42.9 KB
Loading

plot-test/plot/tests/auto/data/pr55-polygons3d-data.rktd

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)