Skip to content

Conversation

@ChrisDryden
Copy link
Collaborator

@ChrisDryden ChrisDryden commented Dec 14, 2025

This is something I was hoping we could add to every integration test we have. We have all of these integration tests for regression, but we don't test GNU using those integration tests to see if those integration tests are even correct. Once this is in I was hoping to add this to as many tests as possible to surface when the integration tests are not matching the behaviour of the GNU implementation

@github-actions
Copy link

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@ChrisDryden
Copy link
Collaborator Author

Lol I think this actually caught an issue in the first one I tried. The echo on these platforms is actually using the bash built in echo

@ChrisDryden ChrisDryden force-pushed the matches_gnu_test_method branch from 07ed3cd to fa467ae Compare December 15, 2025 00:28
@codspeed-hq
Copy link

codspeed-hq bot commented Dec 15, 2025

CodSpeed Performance Report

Merging #9660 will not alter performance

Comparing ChrisDryden:matches_gnu_test_method (2e2ce2d) with main (2d718f2)

Summary

✅ 127 untouched
⏩ 6 skipped1

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@github-actions
Copy link

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@ChrisDryden
Copy link
Collaborator Author

Looks like it actually was a bug on my end, should be good for starting to review

@sylvestre
Copy link
Contributor

i don't think we want to do that.
All tests should match gnu

@ChrisDryden
Copy link
Collaborator Author

I've come across a bunch of our own integration tests that test for things that don't match the GNU integration tests, its tricky to add to the ones that also change the state of the environment so it can't be added to all of them directly.

Maybe I could use this and add it to all of the tests in a utility to show how some tests diverge?

@sylvestre
Copy link
Contributor

have a look to the fuzzers. example:
https://github.com/uutils/coreutils/blob/main/fuzz/fuzz_targets/fuzz_echo.rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants