Test JSOBenchmarks with SolverBenchmarks: try non-forked branches by MaxenceGollier · Pull Request #309 · JuliaSmoothOptimizers/RegularizedOptimization.jl

MaxenceGollier · 2026-03-07T20:05:22Z

DO NOT MERGE

github-actions · 2026-03-09T13:28:49Z

Gist: https://gist.github.com/dpo/7a44ce5a8bd668d39e452de951d037bd

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-09T13:35:53Z

Gist: https://gist.github.com/dpo/bfe8576c5f8e93940a57007513aebb0f

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

MaxenceGollier · 2026-03-09T14:18:58Z

@dpo, this is where I am now, please see the gist.

Apart from performance profiles, what else do you want to see in the gist ?
Is it really necessary to upload .jld2 artifacts if we already show all the results in the gist ?

github-actions · 2026-03-09T20:32:31Z

Gist: https://gist.github.com/dpo/2b0037224b7b5ef213703ec7071aa14b

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-09T21:31:30Z

Gist: https://gist.github.com/dpo/fb3dab228a9ce0dc01fe4b364a037064

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-09T21:37:32Z

Gist: https://gist.github.com/dpo/84b41bbc32e8cb8718791e742c626deb

Full results stored as artifacts

Overview

[## This commit vs reference: R2_precise

.svg](https://gist.github.com/dpo/84b41bbc32e8cb8718791e742c626deb/raw/## This commit vs reference: R2_precise

.svg?sanitize=true)

[## This commit vs reference: R2_imprecise

.svg](https://gist.github.com/dpo/84b41bbc32e8cb8718791e742c626deb/raw/## This commit vs reference: R2_imprecise

.svg?sanitize=true)

github-actions · 2026-03-11T23:18:31Z

Gist: https://gist.github.com/dpo/8840379da84333fbdccb167ff5618a34

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-12T14:03:28Z

Gist: https://gist.github.com/dpo/f3597a3f87791de52d0c75b844d70dfd

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-12T15:06:36Z

Gist: https://gist.github.com/dpo/4266f018a77295baed6fb63c9b4e5baa

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-12T15:19:49Z

Gist: https://gist.github.com/dpo/682937b59b931d94bfb82f5860341159

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-12T15:35:14Z

Gist: https://gist.github.com/dpo/d7bcb00bb0a1d5496149479252f11cd7

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-12T23:00:38Z

Gist: https://gist.github.com/dpo/16da40d3078fb6f212260c12cc96e2ea

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-13T13:52:01Z

Gist: https://gist.github.com/dpo/945f48c4159f6f8e7094351fb918f211

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-13T14:30:53Z

Gist: https://gist.github.com/dpo/9c2701b731f97c34e95bfb1d00f79a97

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-13T19:43:53Z

Gist: https://gist.github.com/dpo/999e6af22d5321f9257549f6ccb605c8

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-13T20:17:16Z

Gist: https://gist.github.com/dpo/6c9ec1cb1fc3d4f754b047a0ab0f3c56

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-16T14:59:18Z

SolverBenchmark Profiles

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-16T15:15:10Z

SolverBenchmark Profiles

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

github-actions · 2026-03-16T17:18:53Z

Gist: https://gist.github.com/dpo/b7a23a011aa9ca88229c021da425b11a

Full results stored as artifacts

Overview

this_commit_vs_reference_R2_time.svg

this_commit_vs_reference_R2_memory.svg

this_commit_vs_reference_R2_gctime.svg

this_commit_vs_reference_R2_allocations.svg

profiles_this_commit_vs_reference_R2.svg

Judgement

Benchmark Report for RegularizedOptimization

Job Properties

Time of benchmarks:
- Target: 16 Mar 2026 - 17:06
- Baseline: 16 Mar 2026 - 17:07
Package commits:
- Target: ae7d95c
- Baseline: c5606a2
Julia commits:
- Target: 5fe89b8
- Baseline: 5fe89b8
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Brackets display tolerances for the benchmark estimates. Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["R2", "bpdn_B0"]`	3.87 (5%) ❌	2.89 (1%) ❌
`["R2", "bpdn_l0"]`	0.67 (5%) ✅	0.99 (1%)
`["R2", "bpdn_l1"]`	3.17 (5%) ❌	1.10 (1%) ❌

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["R2"]

Julia versioninfo

Target

Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 24.6.0 Darwin Kernel Version 24.6.0: Mon Jan 19 22:02:01 PST 2026; root:xnu-11417.140.69.708.3~1/RELEASE_ARM64_VMAPPLE arm64 arm
  CPU: Apple M1 (Virtual): 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz        104 s          0 s         77 s         68 s          0 s  
       #2  2400 MHz        109 s          0 s         64 s         75 s          0 s  
       #3  2400 MHz        110 s          0 s         60 s         78 s          0 s  
  Memory: 7.0 GB (171.578125 MB free)
  Uptime: 248.0 sec
  Load Avg:  11.32568359375  24.6806640625  13.5771484375
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 3 virtual cores)

Baseline

Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 24.6.0 Darwin Kernel Version 24.6.0: Mon Jan 19 22:02:01 PST 2026; root:xnu-11417.140.69.708.3~1/RELEASE_ARM64_VMAPPLE arm64 arm
  CPU: Apple M1 (Virtual): 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz        129 s          0 s         85 s         79 s          0 s  
       #2  2400 MHz        136 s          0 s         71 s         86 s          0 s  
       #3  2400 MHz        136 s          0 s         67 s         90 s          0 s  
  Memory: 7.0 GB (224.25 MB free)
  Uptime: 292.0 sec
  Load Avg:  9.35498046875  22.31103515625  13.275390625
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 3 virtual cores)

this_commit

Benchmark Report for RegularizedOptimization

Job Properties

Time of benchmark: 16 Mar 2026 - 17:06
Package commit: ae7d95c
Julia commit: 5fe89b8
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["R2", "bpdn_B0"]`	11.163 ms (5%)	1.81 MiB (1%)	4319
`["R2", "bpdn_l0"]`	489.090 ms (5%)	3.27 MiB (1%)	175079
`["R2", "bpdn_l1"]`	6.395 ms (5%)	369.67 KiB (1%)	2604

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["R2"]

Julia versioninfo

Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 24.6.0 Darwin Kernel Version 24.6.0: Mon Jan 19 22:02:01 PST 2026; root:xnu-11417.140.69.708.3~1/RELEASE_ARM64_VMAPPLE arm64 arm
  CPU: Apple M1 (Virtual): 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz        104 s          0 s         77 s         68 s          0 s  
       #2  2400 MHz        109 s          0 s         64 s         75 s          0 s  
       #3  2400 MHz        110 s          0 s         60 s         78 s          0 s  
  Memory: 7.0 GB (171.578125 MB free)
  Uptime: 248.0 sec
  Load Avg:  11.32568359375  24.6806640625  13.5771484375
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 3 virtual cores)

Reference

Benchmark Report for RegularizedOptimization

Job Properties

Time of benchmark: 16 Mar 2026 - 17:07
Package commit: c5606a2
Julia commit: 5fe89b8
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	memory	allocations
`["R2", "bpdn_B0"]`	2.884 ms (5%)	642.27 KiB (1%)	946
`["R2", "bpdn_l0"]`	733.902 ms (5%)	3.29 MiB (1%)	176301
`["R2", "bpdn_l1"]`	2.020 ms (5%)	336.62 KiB (1%)	666

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["R2"]

Julia versioninfo

Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 24.6.0 Darwin Kernel Version 24.6.0: Mon Jan 19 22:02:01 PST 2026; root:xnu-11417.140.69.708.3~1/RELEASE_ARM64_VMAPPLE arm64 arm
  CPU: Apple M1 (Virtual): 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz        129 s          0 s         85 s         79 s          0 s  
       #2  2400 MHz        136 s          0 s         71 s         86 s          0 s  
       #3  2400 MHz        136 s          0 s         67 s         90 s          0 s  
  Memory: 7.0 GB (224.25 MB free)
  Uptime: 292.0 sec
  Load Avg:  9.35498046875  22.31103515625  13.275390625
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 3 virtual cores)

SolverBenchmark Profiles

this_commit_vs_reference_R2_precise.svg

this_commit_vs_reference_R2_imprecise.svg

MaxenceGollier mentioned this pull request Mar 7, 2026

Test JSOBenchmarks with SolverBenchmarks #308

Closed

MaxenceGollier added the run benchmarks Used to trigger benchmarks in a PR label Mar 7, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier mentioned this pull request Mar 9, 2026

Support SolverBenchmarks JuliaSmoothOptimizers/JSOBenchmarks.jl#7

Open

MaxenceGollier force-pushed the jso-bmarks-pr branch 3 times, most recently from f6f6463 to f73c6dc Compare March 9, 2026 20:27

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier closed this Mar 9, 2026

MaxenceGollier reopened this Mar 9, 2026

MaxenceGollier force-pushed the jso-bmarks-pr branch 4 times, most recently from e160981 to 591d306 Compare March 11, 2026 22:22

MaxenceGollier force-pushed the jso-bmarks-pr branch 2 times, most recently from dff2fe9 to c0848ed Compare March 12, 2026 13:59

MaxenceGollier force-pushed the jso-bmarks-pr branch from 28d8b09 to 6c55e25 Compare March 12, 2026 14:54

MaxenceGollier force-pushed the jso-bmarks-pr branch 2 times, most recently from c85df6c to 5f332b9 Compare March 12, 2026 23:42

MaxenceGollier force-pushed the jso-bmarks-pr branch 3 times, most recently from af71c0c to abfd6cd Compare March 13, 2026 19:38

MaxenceGollier force-pushed the jso-bmarks-pr branch from abfd6cd to f36d60f Compare March 13, 2026 20:12

MaxenceGollier and others added 3 commits March 16, 2026 11:07

Change η1 parameter to a fixed value of 0.1

0206ba8

worsen R2 to see difference

7c9e430

further worsen R2

776c6c2

MaxenceGollier force-pushed the jso-bmarks-pr branch from f36d60f to 776c6c2 Compare March 16, 2026 15:07

Conversation

MaxenceGollier commented Mar 7, 2026

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

MaxenceGollier commented Mar 9, 2026

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

github-actions bot commented Mar 9, 2026

Uh oh!

github-actions bot commented Mar 11, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 16, 2026

Uh oh!

github-actions bot commented Mar 16, 2026

Uh oh!

github-actions bot commented Mar 16, 2026

Benchmark Report for RegularizedOptimization

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Benchmark Report for RegularizedOptimization

Job Properties

Results

Benchmark Group List

Julia versioninfo

Benchmark Report for RegularizedOptimization

Job Properties

Results

Benchmark Group List

Julia versioninfo

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant