add BLAS test #268
Merged
add BLAS test #268
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 14 commits
June 18, 2025 15:46
Collaborator
|
Sorry for the late checking @smoors . I tried running these tests today but I am getting empty job script files when I try to run. Now I am trying to figure out why. |
casparvl
reviewed
Jul 2, 2025
Collaborator
Author
|
@satishskamath can you try again to see if your issue is fixed with the latest update? |
Collaborator
|
@smoors I'm not sure how you got this to work for EESSI - it shouldn't. I'm getting: And this is totally expected if you 'just' compile something on top of EESSI. The reason is that we don't set Everything in EESSI is With that, in the staging dir, I do: Tada, it now finds Now, I don't know how to properly implement this in a generalized test (it's clear that this is our first compiled test - we're hitting all kinds of stuff). Should we start defining programming environments, that then load this? Should we make some clever code that loads this module if EESSI is being used? If so, how do we determine the correct version: your FlexiBLAS is at GCC level, you need knowledge on the toolchain hierarchy to know which Honestly, I'm not sure how to resolve this, we need to think about it... I guess one option would be to just pass the right arguments for |
Collaborator
|
On the upside, on our local module stack, this works out of the box: That first one ( |
Collaborator
Sorry for the confusion, it does run. The earlier error happened because I ran from a ReFrame from the 2024 stack and we have a hierarchical setup here. |
Collaborator
Author
yes, this is done to run the single-threaded test on a node with no other jobs running on it. a similar thing was done for the OSU test. |
added 12 commits
August 16, 2025 11:44
casparvl
reviewed
Oct 21, 2025
eessi/testsuite/eessi_mixin.py
Outdated
| measure_memory_usage = variable(bool, value=False) | ||
| exact_memory = variable(bool, value=False) | ||
| user_executable_opts = variable(str, value='') | ||
| thread_binding = variable(str, value='None') # takes priority over compact_thread_binding |
Collaborator
There was a problem hiding this comment.
I might be missing something, but why do we need both this and compact_thread_binding?
Collaborator
Author
There was a problem hiding this comment.
i was under the impression that you can't override the thread_binding variable in a test class, but that's actually not true. you can override it, but not in a base class.
fixed in 8a948e4
casparvl
reviewed
Oct 21, 2025
eessi/testsuite/eessi_mixin.py
Outdated
| hooks.add_buildenv_module(self) | ||
|
|
||
| thread_binding = self.thread_binding.lower() | ||
| if thread_binding == 'true' or (thread_binding == 'none' and self.compact_thread_binding): |
Collaborator
There was a problem hiding this comment.
If thread_binding is a variable, what if it's not True or False, but someone set it to compact? Or made a type Ture?. I think we should catch all cases we know (true / compact => do compact binding, false => do nothing, anything else => do nothing and warn the user that an invalid value was set for thread_binding).
Collaborator
Author
There was a problem hiding this comment.
i opted for a hard error in case an invalid value was set
fixed in 8a948e4
casparvl
reviewed
Oct 21, 2025
casparvl
reviewed
Oct 21, 2025
casparvl
reviewed
Oct 21, 2025
Collaborator
casparvl
left a comment
casparvl
left a comment
There was a problem hiding this comment.
I ran with reframe -c eessi/testsuite/tests/libs/blas/ --run -t '1_8_node|1_core', everything worked perfectly fine.
I left some minor comments / questions.
casparvl
reviewed
Oct 21, 2025
Collaborator
Author
|
@casparvl thanks for the great comments, should all be addressed now. |
casparvl
approved these changes
Oct 22, 2025
Collaborator
casparvl
left a comment
casparvl
left a comment
There was a problem hiding this comment.
Rerunning one final time, but looks good to me. I can already confirm the Metalwalls warning was resolved.
Collaborator
|
As a final note: everything worked. I do still see some weird differences in the performance of this test for the 2023a modules for some BLAS routines. E.g. Note the poor performance of I'd assume the issue is not the test here. @smoors what do you think? Or does this ring a bell and do you think it could be somewhere in the test setup? Also: do you see the same behavior on your system? (note: I've used the EESSI software stack for these tests). If you also agree @smoors : I think it's worth checking out, but it's not worth blocking this PR over. Maybe we just copy the above to an issue and take it from there... |
Collaborator
Author
|
i see this too on 2023a using our local clusters. 2024a looks fine though. checking the output files, the problem arises for larger matrices: the it looks indeed not a problem with the test but with this version. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this adds the BLAS test included in the BLIS sources, using a modified/simplified Makefile so it works with FlexiBLAS.
supported BLAS libs: OpenBLAS, BLIS, AOCL-BLAS, imkl
ready for testing and review.
notes:
executables are compiled in every run-only test job, no separate compile-only test jobs:
the downside is that the executables have to be built for every test case, while with a compile-only test job this has to be done only once for each architecture. i'm fine with adding a compile-only job if you prefer.
no programming environments (environs) are used:
for BLIS and AOCL-BLAS, no loop-specific
BLIS_**_NTenvironment variables are set, only the genericOMP_NUM_THREADS.update 2025-08-16:
split_module(). if needed we can always add exceptions, like is already done forintel-compilers.8_corescale to accomodate nodes with high cpu counts.select_matching_modules, which selects from a list of modules the ones that match a reference module (i.e. their toolchains are compatible). this function requires the availability of EasyBuild (the test will be skipped if it's not available).