Skip to content

refactor: significantly improve performance of *Problem generation #3751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 26 commits into from
Jun 25, 2025
Merged

refactor: significantly improve performance of *Problem generation #3751

merged 26 commits into from
Jun 25, 2025

Conversation

@AayushSabharwal
Copy link
Member

Viva la Tracy.jl

@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch 8 times, most recently from d532cab to ce25349 Compare June 24, 2025 13:25
@AayushSabharwal
Copy link
Member Author

Well benchmarks taking this long is weird. It ran in a reasonable amount of time on my system. I'll investigate.

@AayushSabharwal
Copy link
Member Author

Okay I must have run it on a lower N. The problem is that linear_expansion takes ages, so alias_elimination! takes ages. There's not much that comes to mind that will fix this immediately, but I'll keep it in mind. Until then, I'll just make the benchmark less painful.

These are the results from this 3 hour run:

Benchmark Results (Julia vlts)

Time benchmarks
master ce25349... master / ce25349...
ODEProblem 0.253 ± 0.019 s 0.205 ± 0.013 s 1.23 ± 0.12
init 0.0619 ± 0.013 ms 0.0601 ± 0.0017 ms 1.03 ± 0.22
large_parameter_init/ODEProblem 0.142 h 0.052 h 2.73
large_parameter_init/init 0.346 ± 0.012 ms 0.336 ± 0.01 ms 1.03 ± 0.048
mtkcompile 0.0426 ± 0.0031 s 0.0438 ± 0.0043 s 0.972 ± 0.12
time_to_load 5.9 ± 0.043 s 5.73 ± 0.088 s 1.03 ± 0.018
Memory benchmarks
master ce25349... master / ce25349...
ODEProblem 1.64 M allocs: 0.0556 GB 1.45 M allocs: 0.0431 GB 1.29
init 0.908 k allocs: 0.0479 MB 0.908 k allocs: 0.0479 MB 1
large_parameter_init/ODEProblem 2.61 G allocs: 85.7 GB 1.13 G allocs: 36.8 GB 2.33
large_parameter_init/init 5.86 k allocs: 0.245 MB 5.86 k allocs: 0.245 MB 1
mtkcompile 0.256 M allocs: 9.39 MB 0.256 M allocs: 9.4 MB 0.999
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

So yeah, this PR does help quite a bit. Note that even with the changes to make the benchmark less insane, this PR will still take a while to run because master will take a while to run.

@AayushSabharwal
Copy link
Member Author

Turns out that the trivial tearing optimization wasn't actually implemented correctly and wasn't even doing anything - so that 2.73x performance improvement is about to get a whole lot better.

@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch 3 times, most recently from bcadee8 to 194ebc7 Compare June 25, 2025 07:48
@AayushSabharwal
refactor: do not build and use paramsubs in `generate_initializesys…
0948934 
…tem`

Simplification doesn't care for metadata, so this is unnecessary and
expensive.
@AayushSabharwal
feat: add fast path for constructing MTKParameters in `process_SciM…
58a47ab 
…LProblem`
@AayushSabharwal
refactor: avoid unnecessary computation in evaluate_varmap!
c8618c0
@AayushSabharwal
refactor: avoid unnecesary computation in build_operating_point!
9982ed9
@AayushSabharwal
refactor: batch computation of temporary values in `maybe_build_initi…
dd8b432 
…alization_problem`
@AayushSabharwal
feat: add an always-present mutable cache key to the system
29eee6d
@AayushSabharwal
fix: fix unnecessary warnings when no stream connections in `expand_c…
b86cc52 
…onnections`
@AayushSabharwal
feat: cache intermediate results for observed_equations_used_by
d3a7e71
@AayushSabharwal
fix: remove unnecessary scalarization in InitializationProblem
3a0abbb
@AayushSabharwal
refactor: add fast path in build_operating_point!
27702b6
@AayushSabharwal
fix: do not rely on metadata in process_parameter_equations
7db8dec
@AayushSabharwal
refactor: remove source of allocations in InitializationProblem
5bfb2a9
@AayushSabharwal
fix: properly handle values given to parameter dependencies in `late_…
fffc983 
…binding_update_u0_p`
@AayushSabharwal
feat: invalidate cache in @set!
7ec630d
@AayushSabharwal
refactor: update tests to account for new initsys generation
d4079a8
@AayushSabharwal
fix: handle metadata merging in extend
7709760
@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch from 194ebc7 to 1fbff9b Compare June 25, 2025 11:36
@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch from 1fbff9b to 2efd558 Compare June 25, 2025 12:10
@AayushSabharwal
Copy link
Member Author

This is finally good to go. I've verified that the MTKStdlib failures are due to different guesses being required because the way the initialization is formulated changed. The failing initializations are still fully determined, and with the additional guesses they pass tests.

Benchmark Results (Julia vlts)

###Time benchmarks

master 2efd558... master / 2efd558...
ODEProblem 0.246 ± 0.0021 s 0.205 ± 0.0023 s 1.2 ± 0.017
init 0.0602 ± 0.002 ms 0.0597 ± 0.0018 ms 1.01 ± 0.045
large_parameter_init/ODEProblem 33.1 s 0.952 ± 0.0063 s 34.8
large_parameter_init/init 0.11 ± 0.0029 ms 0.109 ± 0.0029 ms 1.01 ± 0.038
mtkcompile 0.0434 ± 0.0015 s 0.043 ± 0.0014 s 1.01 ± 0.048
time_to_load 5.7 ± 0.082 s 5.61 ± 0.13 s 1.02 ± 0.027

Memory benchmarks

master 2efd558... master / 2efd558...
ODEProblem 1.65 M allocs: 0.0557 GB 1.45 M allocs: 0.0426 GB 1.31
init 0.908 k allocs: 0.0479 MB 0.908 k allocs: 0.0479 MB 1
large_parameter_init/ODEProblem 0.175 G allocs: 5.88 GB 5.52 M allocs: 0.205 GB 28.6
large_parameter_init/init 1.75 k allocs: 0.0874 MB 1.75 k allocs: 0.0874 MB 1
mtkcompile 0.256 M allocs: 9.39 MB 0.258 M allocs: 9.62 MB 0.976
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

Which is amazing! The big numbers are absurd and get more absurd if the benchmark problem increases in size.

@ChrisRackauckas ChrisRackauckas merged commit e2ef9ca into SciML:master Jun 25, 2025
40 of 50 checks passed
@AayushSabharwal AayushSabharwal deleted the as/initsys-optimization branch June 25, 2025 17:20
@AayushSabharwal AayushSabharwal mentioned this pull request Jun 26, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ChrisRackauckas ChrisRackauckas ChrisRackauckas approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AayushSabharwal @ChrisRackauckas