Skip to content

Conversation

@AayushSabharwal
Copy link
Member

Viva la Tracy.jl

@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch 8 times, most recently from d532cab to ce25349 Compare June 24, 2025 13:25
@AayushSabharwal
Copy link
Member Author

Well benchmarks taking this long is weird. It ran in a reasonable amount of time on my system. I'll investigate.

@AayushSabharwal
Copy link
Member Author

Okay I must have run it on a lower N. The problem is that linear_expansion takes ages, so alias_elimination! takes ages. There's not much that comes to mind that will fix this immediately, but I'll keep it in mind. Until then, I'll just make the benchmark less painful.

These are the results from this 3 hour run:

Benchmark Results (Julia vlts)

Time benchmarks
master ce25349... master / ce25349...
ODEProblem 0.253 ± 0.019 s 0.205 ± 0.013 s 1.23 ± 0.12
init 0.0619 ± 0.013 ms 0.0601 ± 0.0017 ms 1.03 ± 0.22
large_parameter_init/ODEProblem 0.142 h 0.052 h 2.73
large_parameter_init/init 0.346 ± 0.012 ms 0.336 ± 0.01 ms 1.03 ± 0.048
mtkcompile 0.0426 ± 0.0031 s 0.0438 ± 0.0043 s 0.972 ± 0.12
time_to_load 5.9 ± 0.043 s 5.73 ± 0.088 s 1.03 ± 0.018
Memory benchmarks
master ce25349... master / ce25349...
ODEProblem 1.64 M allocs: 0.0556 GB 1.45 M allocs: 0.0431 GB 1.29
init 0.908 k allocs: 0.0479 MB 0.908 k allocs: 0.0479 MB 1
large_parameter_init/ODEProblem 2.61 G allocs: 85.7 GB 1.13 G allocs: 36.8 GB 2.33
large_parameter_init/init 5.86 k allocs: 0.245 MB 5.86 k allocs: 0.245 MB 1
mtkcompile 0.256 M allocs: 9.39 MB 0.256 M allocs: 9.4 MB 0.999
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

So yeah, this PR does help quite a bit. Note that even with the changes to make the benchmark less insane, this PR will still take a while to run because master will take a while to run.

@AayushSabharwal
Copy link
Member Author

Turns out that the trivial tearing optimization wasn't actually implemented correctly and wasn't even doing anything - so that 2.73x performance improvement is about to get a whole lot better.

@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch 3 times, most recently from bcadee8 to 194ebc7 Compare June 25, 2025 07:48
@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch from 194ebc7 to 1fbff9b Compare June 25, 2025 11:36
@AayushSabharwal AayushSabharwal force-pushed the as/initsys-optimization branch from 1fbff9b to 2efd558 Compare June 25, 2025 12:10
@AayushSabharwal
Copy link
Member Author

This is finally good to go. I've verified that the MTKStdlib failures are due to different guesses being required because the way the initialization is formulated changed. The failing initializations are still fully determined, and with the additional guesses they pass tests.

Benchmark Results (Julia vlts)

###Time benchmarks

master 2efd558... master / 2efd558...
ODEProblem 0.246 ± 0.0021 s 0.205 ± 0.0023 s 1.2 ± 0.017
init 0.0602 ± 0.002 ms 0.0597 ± 0.0018 ms 1.01 ± 0.045
large_parameter_init/ODEProblem 33.1 s 0.952 ± 0.0063 s 34.8
large_parameter_init/init 0.11 ± 0.0029 ms 0.109 ± 0.0029 ms 1.01 ± 0.038
mtkcompile 0.0434 ± 0.0015 s 0.043 ± 0.0014 s 1.01 ± 0.048
time_to_load 5.7 ± 0.082 s 5.61 ± 0.13 s 1.02 ± 0.027

Memory benchmarks

master 2efd558... master / 2efd558...
ODEProblem 1.65 M allocs: 0.0557 GB 1.45 M allocs: 0.0426 GB 1.31
init 0.908 k allocs: 0.0479 MB 0.908 k allocs: 0.0479 MB 1
large_parameter_init/ODEProblem 0.175 G allocs: 5.88 GB 5.52 M allocs: 0.205 GB 28.6
large_parameter_init/init 1.75 k allocs: 0.0874 MB 1.75 k allocs: 0.0874 MB 1
mtkcompile 0.256 M allocs: 9.39 MB 0.258 M allocs: 9.62 MB 0.976
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

Which is amazing! The big numbers are absurd and get more absurd if the benchmark problem increases in size.

@ChrisRackauckas ChrisRackauckas merged commit e2ef9ca into SciML:master Jun 25, 2025
40 of 50 checks passed
@AayushSabharwal AayushSabharwal deleted the as/initsys-optimization branch June 25, 2025 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants