Perform topological sort to order components by cmacmackin · Pull Request #428 · boutproject/hermes-3

cmacmackin · 2025-11-27T18:24:54Z

Using the access control information introduced in #421, this PR makes it possible for Hermes-3 to work out the order of components at run-time. This will make things far simpler and more robust for users. It will also fail faster if there is an unsatisfiable or circular dependency.

Closes #384.

cmacmackin · 2025-11-27T18:30:22Z

This is nearly done. Remaining tasks:

Merge Control access to state variables in transform method #421 and rebase.
There is one integration test giving me strange h5py errors on my computer. It doesn't look to be related to any changes I've made. We'll see what happens in CI. I seem to recall we saw something similar elsewhere at one point.
Update documentation.

codecov · 2025-11-27T18:32:44Z

Codecov Report

❌ Patch coverage is 90.54054% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.51%. Comparing base (053c9bb) to head (8fc17e0).

Files with missing lines	Patch %	Lines
include/component.hxx	60.00%	6 Missing ⚠️
src/component_scheduler.cxx	96.55%	0 Missing and 4 partials ⚠️
src/sheath_boundary_simple.cxx	0.00%	2 Missing ⚠️
src/amjuel_reaction.cxx	0.00%	0 Missing and 1 partial ⚠️
src/sheath_boundary_insulating.cxx	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #428      +/-   ##
==========================================
+ Coverage   29.52%   30.51%   +0.98%     
==========================================
  Files          94       94              
  Lines        8768     8903     +135     
  Branches     1228     1257      +29     
==========================================
+ Hits         2589     2717     +128     
- Misses       5914     5916       +2     
- Partials      265      270       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

Note that components still aren't setting up any permissions, so there would be runtime errors when executing the transform methods. Furthermore, there are some missing methods I realise I need for GuardedOptions. The unit tests are also failing to compile, although I don't understand what the problem is for them, yet.

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

mikekryjak · 2026-01-07T13:48:51Z

I missed this in #421, but I see that you added a long-needed feature - a better way to determine species type which is based on the charge:

hermes-3/src/component_scheduler.cxx

Lines 339 to 352 in 8fc17e0

    
           // FIXME: Would there be any spcies without AA? Is there any other 
        
           // reliable way to identify what is a species? 
        
           else if (component_options[name_trimmed].isSet("AA")) { 
        
             if (component_options[name_trimmed].isSet("charge")) { 
        
               const BoutReal charge = component_options[name_trimmed]["charge"]; 
        
               if (charge > 1e-5) { 
        
                 positive_ions.push_back(name_trimmed); 
        
               } else if (charge < -1e-5) { 
        
                 negative_ions.push_back(name_trimmed); 
        
               } else { 
        
                 neutrals.push_back(name_trimmed); 
        
               } 
        
             } else { 
        
               neutrals.push_back(name_trimmed);

There is already a function that does the same in hermes_utils and is used in a few places in the code:

hermes-3/include/hermes_utils.hxx

Lines 61 to 70 in 053c9bb

    
           /// Identify species name string as electron, ion or neutral 
        
           inline SpeciesType identifySpeciesType(const std::string& species) { 
        
             if (species == "e") { 
        
               return SpeciesType::electron; 
        
             } else if ((species == "i") or 
        
                        species.find(std::string("+")) != std::string::npos) { 
        
               return SpeciesType::ion; 
        
             } 
        
             // Not electron or ion -> neutral 
        
             return SpeciesType::neutral;

Ideally there should be only one tool to do this, and your new one seems more robust. Is there any reason not to replace the one in hermes_utils with the new one?

cmacmackin · 2026-01-07T13:54:37Z

Ideally there should be only one tool to do this, and your new one seems more robust. Is there any reason not to replace the one in hermes_utils with the new one?

Probably not. I just wasn't sure if people would be happy about me changing that bit of code. Note that this will require changing the function signature of identifySpeciesType so that it takes the Options instead of just a string.

mikekryjak · 2026-01-07T13:55:01Z

I went through the sorting algorithm and I see that you have left a good amount of comments on the individual bits. However, I found it difficult to get my head around what's going on just because of the amount of steps involved. It would be very useful to have a paragraph describing how the algorithm works step-by-step, either in the docs or in the comments (or both)

cmacmackin · 2026-01-07T13:57:29Z

There are some places (e.g., when writing tests) where it was convenient to just be able to have a list of species names and use the old heuristics to categorise them. Probably not a good enough reason to keep that though.

ZedThree

LGTM, thanks @cmacmackin!

There's some trivial bits that I'm happy to fix myself

ZedThree · 2026-01-07T11:41:13Z

src/component_scheduler.cxx

+const std::set<std::string> ComponentScheduler::predeclared_variables = {
+    "time",          "linear",      "units:inv_meters_cubed", "units:eV", "units:Tesla",
+    "units:seconds", "units:meters"};


Given this is only used in this file, we can move the declaration out of the header and just have it here.

ZedThree · 2026-01-07T11:45:16Z

src/component_scheduler.cxx

+
+/// Get all the parent sections of a variable "path". Sections are
+/// separated by colons in the path.
+std::set<std::string> getParents(const std::string& name) {


Is this just strsplit(name, ':')?

No, because rather than splitting on ':' it is returning the hierarchy of parent sections. So, e.g., getParents("species:d:collision_frequencies:d_d_coll") would return {"species", "species:d", "species:d:collision_frequencies"}.

Ah, I see! I'll add that example to the docstring, that's very clear

ZedThree · 2026-01-07T11:56:04Z

src/component_scheduler.cxx

+    // FIXME: this isn't an empty set
+    //result.insert({name, {name}});
+    result.insert({name, {}});


I'm a bit confused by the comment -- is it outdated?

Yes, should have deleted that and the commented line. I must have left that during some debugging.

ZedThree · 2026-01-07T16:12:59Z

src/component_scheduler.cxx

+/// In pratice, `item` here represents the index of a particular
+/// component. The indices of the dependencies of `item` are stored in
+/// the corresponding element of `dependencies`.
+void topological_sort(const std::vector<std::set<size_t>>& dependencies, size_t item,


All these local functions can go in an anonymous namespace

What is the advantage of putting them in an anonymous namespace?

It's a guard against One Definition Rule (ODR) violations -- basically ensures that functions have names unique to the translation unit, and so if there happens to be another function with the same name somewhere else, they won't clash at link time.

cmacmackin · 2026-01-07T18:12:59Z

docs/sphinx/developer.rst

+
+All component constructors must pass a `Permissions` object to the
+constructor on the `Component::Component` base class. This specifies
+which variables will be read/written by the `Component::transform`
+method and will be used to construct a `GuardedOptions` object to be
+passed into `Component::transform_impl`. The `Permissions` object
+(`Component::state_variable_access`) can be further updated in the
+body of the constructor of your component, based on what
+configurations were specified in ``alloptions``. You should give read
+and write permissions to the minimum number of variables necessary, to
+avoid circular dependencies arising among components.


This paragraph appears to be a duplicate which arose due to a mistake during a rebase (almost certainly my fault!). It should be deleted.

cmacmackin changed the title ~~WIP: Perform topologicalk sort to order components~~ WIP: Perform topological sort to order components Nov 27, 2025

cmacmackin marked this pull request as draft November 27, 2025 18:45

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from a75b2ce to 8d049e2 Compare December 2, 2025 15:57

cmacmackin marked this pull request as ready for review December 5, 2025 15:07

cmacmackin changed the title ~~WIP: Perform topological sort to order components~~ Perform topological sort to order components Dec 5, 2025

This was referenced Dec 8, 2025

Control access to state variables in transform method #421

Merged

Remove use of const_cast with Options #443

Open

cmacmackin mentioned this pull request Dec 18, 2025

Allow components to call for creation of sub components #445

Open

cmacmackin added a commit that referenced this pull request Jan 2, 2026

Responded to some comments by @ZedThree

e650c1c

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from c31490e to e650c1c Compare January 2, 2026 14:28

cmacmackin added a commit that referenced this pull request Jan 5, 2026

Responded to some comments by @ZedThree

fff0581

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from e650c1c to 9dbeed5 Compare January 5, 2026 17:42

cmacmackin added a commit that referenced this pull request Jan 6, 2026

Responded to some comments by @ZedThree

b1dfb25

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

cmacmackin force-pushed the cmacmackin/topological_sort_components branch from 9dbeed5 to 04a8977 Compare January 6, 2026 16:00

cmacmackin added 12 commits January 7, 2026 10:46

Added unit test for topological sorting of components

9e5775d

Add ability to order components automatically

a02d845

Ensure ordering of components accounts for pre-set state variables

c903e91

Added support for sorting components based on section permissions

d37d2ad

Mention new sorting and remove references to component ordering

a6c9454

Check results are the same regardless of component order

a16afd6

Check only one component can make final write

7d1851d

Fixed some incorrect access controls causing errors

7559d27

Add better comments explaining component sorting process

904d987

Responded to some comments by @ZedThree

975a912

The comments were in PR #445, but some of them actually relate to work done in #428. Only the latter are dealt with in this PR.

Add missed variable permission

8fc17e0

ZedThree force-pushed the cmacmackin/topological_sort_components branch from 04a8977 to 8fc17e0 Compare January 7, 2026 10:46

ZedThree approved these changes Jan 7, 2026

View reviewed changes

cmacmackin commented Jan 7, 2026

View reviewed changes

This was referenced Jan 14, 2026

Suppress permission warnings #468

Merged

Components not covered by integrated tests #475

Open

Conversation

cmacmackin commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmacmackin commented Nov 27, 2025 • edited by ZedThree Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mikekryjak commented Jan 7, 2026

Uh oh!

cmacmackin commented Jan 7, 2026

Uh oh!

mikekryjak commented Jan 7, 2026

Uh oh!

cmacmackin commented Jan 7, 2026

Uh oh!

ZedThree left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cmacmackin commented Nov 27, 2025 •

edited

Loading

cmacmackin commented Nov 27, 2025 •

edited by ZedThree

Loading

codecov bot commented Nov 27, 2025 •

edited

Loading