Skip to content

Change backend threading behavior #2417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

Change backend threading behavior #2417

wants to merge 7 commits into from

Conversation

@afilogo
Copy link
Contributor

@afilogo afilogo commented May 22, 2025 •
edited
Loading

This PR is based on the discussion #2159, having into account your feedback. I apologize for the late reply and hope did not make any mistake.

@github-actions
Copy link
Contributor

github-actions bot commented May 22, 2025

Review checklist

This checklist is meant to assist creators of PRs (to let them know what reviewers will typically look for) and reviewers (to guide them in a structured review process). Items do not need to be checked explicitly for a PR to be eligible for merging.

Purpose and scope

  • The PR has a single goal that is clear from the PR title and/or description.
  • All code changes represent a single set of modifications that logically belong together.
  • No more than 500 lines of code are changed or there is no obvious way to split the PR into multiple PRs.

Code quality

  • The code can be understood easily.
  • Newly introduced names for variables etc. are self-descriptive and consistent with existing naming conventions.
  • There are no redundancies that can be removed by simple modularization/refactoring.
  • There are no leftover debug statements or commented code sections.
  • The code adheres to our conventions and style guide, and to the Julia guidelines.

Documentation

  • New functions and types are documented with a docstring or top-level comment.
  • Relevant publications are referenced in docstrings (see example for formatting).
  • Inline comments are used to document longer or unusual code sections.
  • Comments describe intent ("why?") and not just functionality ("what?").
  • If the PR introduces a significant change or new feature, it is documented in NEWS.md with its PR number.

Testing

  • The PR passes all tests.
  • New or modified lines of code are covered by tests.
  • New or modified tests run in less then 10 seconds.

Performance

  • There are no type instabilities or memory allocations in performance-critical parts.
  • If the PR intent is to improve performance, before/after time measurements are posted in the PR.

Verification

  • The correctness of the code was verified using appropriate tests.
  • If new equations/methods are added, a convergence test has been run and the results
    are posted in the PR.

Created with ❤️ by the Trixi.jl community.

@DanielDoehring DanielDoehring requested a review from vchuravy May 25, 2025 12:32
@DanielDoehring
Copy link
Member

DanielDoehring commented May 25, 2025

Thanks for your contribution! I am not so sure if we really want to offer other threading strategies (namely dynamic) in the main repository. Same holds also for the serial option - IMO, this is not really necessary, as one could just run the program with a single thread.

vchuravy
vchuravy reviewed May 25, 2025
View reviewed changes
vchuravy
vchuravy reviewed May 25, 2025
View reviewed changes
@afilogo afilogo closed this May 26, 2025
@afilogo afilogo reopened this May 29, 2025
@afilogo
Copy link
Contributor Author

afilogo commented May 29, 2025

Thanks for the feedback @DanielDoehring and @vchuravy. Appreciate you taking the time to review it. I thought it might increase user control, but I also understand your perspective.

vchuravy
vchuravy reviewed May 29, 2025
View reviewed changes
@afilogo @vchuravy
Update src/solvers/dg.jl
3ad849b 
Co-authored-by: Valentin Churavy <[email protected]>
@ranocha ranocha added breaking parallelization Related to MPI, threading, tasks etc. labels Jun 2, 2025
@ranocha ranocha requested a review from vchuravy June 2, 2025 14:54
vchuravy
vchuravy previously approved these changes Jun 2, 2025
View reviewed changes
Copy link
Member

@vchuravy vchuravy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks that seems like a good improvement.

@codecov
Copy link

codecov bot commented Jun 2, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 40.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 96.79%. Comparing base (55d44d9) to head (657311f).
Report is 41 commits behind head on main.

Files with missing lines Patch % Lines
src/auxiliary/math.jl 0.00% 6 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2417      +/-   ##
==========================================
- Coverage   96.80%   96.79%   -0.01%     
==========================================
  Files         495      495              
  Lines       40917    40919       +2     
==========================================
- Hits        39608    39607       -1     
- Misses       1309     1312       +3
Flag Coverage Δ
unittests 96.79% <40.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ranocha
Copy link
Member

ranocha commented Jun 2, 2025

CI is failing, so it appears that something is off. Could you please take a look and see what's going on?

@afilogo
Copy link
Contributor Author

afilogo commented Jun 2, 2025

CI is failing, so it appears that something is off. Could you please take a look and see what's going on?

I believe the issue is the in static macro "@static if _PREFERENCE_THREADING === :polyester && LoopVectorization.check_args(u_ode)" code line. It should be fine now.

@ranocha ranocha requested a review from vchuravy June 2, 2025 20:17
@afilogo
Copy link
Contributor Author

afilogo commented Jun 2, 2025

I've also missed a previous _PREFERENCE_POLYESTER line. Sorry about that.

@ranocha ranocha mentioned this pull request Jun 3, 2025
Closed
9 tasks
ranocha
ranocha requested changes Jun 5, 2025
View reviewed changes
Copy link
Member

@ranocha ranocha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add an entry to the NEWS.md file since this is a breaking change?

src/auxiliary/auxiliary.jl
Comment on lines +228 to +237
elseif _PREFERENCE_THREADING === :dynamic
quote
let
if $Threads.nthreads() == 1
$(expr)
else
$Threads.@threads :dynamic $(expr)
end
end
end
Copy link
Member

@ranocha ranocha Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will introduce bugs for code like

Trixi.jl/src/solvers/dgsem_tree/dg_2d.jl

Lines 945 to 950 in f35d749

@threaded for mortar in eachmortar(dg, cache)
# Choose thread-specific pre-allocated container
fstar_primary_upper = fstar_primary_upper_threaded[Threads.threadid()]
fstar_primary_lower = fstar_primary_lower_threaded[Threads.threadid()]
fstar_secondary_upper = fstar_secondary_upper_threaded[Threads.threadid()]
fstar_secondary_lower = fstar_secondary_lower_threaded[Threads.threadid()]

won't it? @vchuravy

Copy link
Member

@vchuravy vchuravy Jul 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, yes it will. @benegee and I were just talking about #2212 and the fact that creating or adapting the cache is a weird operation due to these backend specific containers. We are baking quite a bit of structure that is dependent on how we are performing the computation into the cache object.

So eventually we might need something like create_cache(..., backend), but even that is fraught since with #2212 I am able to change the storage type and thus the backend.

Maybe, we need struct StaticThreadIDCache end and struct GPUCache end...

Copy link
Member

@ranocha ranocha Jul 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think we will need to remove the "dynamic" option for now from this PR.

Copy link
Member

@ranocha ranocha Jul 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elseif _PREFERENCE_THREADING === :dynamic
quote
let
if $Threads.nthreads() == 1
$(expr)
else
$Threads.@threads :dynamic $(expr)
end
end
end

ranocha
ranocha reviewed Jul 3, 2025
View reviewed changes
@vchuravy vchuravy self-assigned this Jul 18, 2025
@vchuravy vchuravy mentioned this pull request Jul 18, 2025
Merged
@vchuravy
Copy link
Member

vchuravy commented Jul 18, 2025

@afilogo thank you very much for your contribution. I am going to bring this PR to the finish line in #2476

@vchuravy vchuravy closed this Jul 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ranocha ranocha ranocha requested changes

@vchuravy vchuravy Awaiting requested review from vchuravy

Assignees

@vchuravy vchuravy

Labels

breaking parallelization Related to MPI, threading, tasks etc.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@afilogo @DanielDoehring @ranocha @vchuravy