Skip to content

[docs] Add Included Suites to Test Suite Guide #128937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+41 −0
Open

[docs] Add Included Suites to Test Suite Guide #128937

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c36f638
[docs] Add Included Suites to Test Suite Guide
lenary Feb 26, 2025
20ad5df
Merge remote-tracking branch 'origin/main' into pr/test-suite-guide-i…
lenary Apr 4, 2025
ba30024
Nuance is good
lenary Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions llvm/docs/TestSuiteGuide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,27 @@ performance. You can find more information at
of the LLVM project is hosted at [http://lnt.llvm.org](http://lnt.llvm.org).


Included Test Suites
--------------------

The LLVM Test Suite repository includes a number of test suites that we have
copied into the repository from external sources, and hand-written CMake files
to hook into the LLVM Test Suite system. For example, these include GCC's C
and Fortran Torture test suites.
Copy link
Contributor

@tarunprabhu tarunprabhu Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Slight change in wording to be a bit more accurate: we include all the gfortran tests, not just the torture tests.

Suggested change
to hook into the LLVM Test Suite system. For example, these include GCC's C
and Fortran Torture test suites.
to hook into the LLVM Test Suite system. These include GCC's C Torture tests and gfortran's test suite.


In some cases, these external tests have a different license to other code in
the llvm-test-suite repository, which must be documented in `LICENSE.TXT` in the
root of the test suite repository, under the heading "Software from third
parties included in the LLVM Project".

The only changes we can accept to the tests in these suites are where we bring
in all the changes from the external sources. Users should not make individual
downstream changes to these tests.
Copy link
Contributor

@asb asb Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "users" and "downstream" might be a bit ambiguous in this context. Unless I'm getting the wrong end of the stick, I think you're referring to LLVM contributors making changes and attempting to submit them to the llvm-test-suite repo (downstream from the original tests/benchmarks, but "upstream" from many people's point of view!). I assume we're not trying to pass judgement on whatever atrocities people apply to an llvm-test-suite checkout on their own machine for their downstream only purposes(!).

I have a more general comment about the policy I'll leave on the Discourse thread.

Copy link
Member Author

@lenary lenary Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that this is ambiguous. I think the consensus in Discourse is that this needs significant rewriting anyway, but I will make sure I am clearer about what I'm referring to.


We do accept changes to the CMake integration files for these tests, and where a
test is wrong/broken, we should have the ability to exclude the test from our
integration with the suite.

External Suites
---------------

Expand Down
Loading