Skip to content

[CI Broken] Test failures in Rules and Simulation Results tests due to upstream API changes #186

@ChrisRackauckas-Claude

Description

@ChrisRackauckas-Claude

Summary

CI is failing on the main branch. The health check PR (#185) reveals test failures in both the Rules and Simulation Results test suites.

Failed CI Jobs

All Julia versions (1.x, LTS, and pre-release) are failing with the same test errors:

  • Julia 1 - ubuntu-latest - x64: Failed (33m 31s)
  • Julia lts - ubuntu-latest - x64: Failed (21m 27s)
  • Julia pre - ubuntu-latest - x64: Failed (33m 1s)

Link to failed run: https://github.com/SciML/SBMLToolkit.jl/actions/runs/20580524518

Error Details

1. Rules test error (1 error)

Location: test/rules.jl:65

ODESystem(m) called at test/rules.jl:65
MethodError: no method matching ODESystem(::SBML.Model; name::Symbol)

The test is calling @named sys = ODESystem(m) where m is an SBML.Model, but this constructor doesn't exist or has changed signature.

2. Simulation Results test failures (2 failures)

Location: test/simresults.jl:22

Both failures are for test cases that expect success but now error:

Case 00170 (Model using parameters and rules only):

ErrorException("An equation (S2*k2 - k1*S1(t) ~ Differential(t)(S1(t))) contains a differential with respect to a species. This is currently not supported. If this is a functionality you require, please raise an issue on the Catalyst GitHub page and we can consider the best way to implement it.")

Case 00325 (One reactions and two rate rules with four species in a 2D compartment):

ErrorException("An equation (0.5compartment*k1 ~ Differential(t)(S3(t))) contains a differential with respect to a species. This is currently not supported. If this is a functionality you require, please raise an issue on the Catalyst GitHub page and we can consider the best way to implement it.")

Root Cause Analysis

  1. Rules test failure: The ODESystem constructor for SBML.Model appears to no longer accept the model directly. The API may have changed in a recent ModelingToolkit or SBMLToolkit version.

  2. Simulation test failures: These are caused by upstream changes in Catalyst.jl that now explicitly reject equations containing differentials with respect to species variables. This is a deliberate Catalyst.jl design decision that breaks SBMLToolkit's handling of SBML rate rules that modify species.

Suggested Fix Approach

  1. For the Rules test error: Investigate if the ODESystem(::SBML.Model) method was removed or renamed. May need to update the test or add a new method.

  2. For the Simulation results failures:

    • Consider opening an issue on Catalyst.jl to discuss support for rate rules that modify species (as the error message suggests)
    • Alternatively, update how SBMLToolkit generates equations to avoid using Differential(t)(species) directly
    • May need to mark these test cases as "expected to fail" if Catalyst.jl won't support this pattern

Test Summary

Test Summary:        | Pass  Fail  Error  Total      Time
SBMLToolkit.jl       |  112     2      1    115  20m38.7s
  Quality Assurance  |   10                  10   1m30.9s
  Systems            |   42                  42   1m17.8s
  Reactions          |   28                  28     22.8s
  Rules              |    9            1     10     30.2s
  Events             |    3                   3      4.0s
  Utils              |   11                  11      2.2s
  Simulation results |    7     2             9   2m09.1s
  Wuschel            |    2                   2  14m41.5s

cc @ChrisRackauckas

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions