Unit testing code cleanup

[mikeingold]

Changes

• Minor code cleanup
  • Use let blocks to constrain scope of test geometries
  • Improved comments to communicate intent

Context

The original intent of this PR was to implement some @test_throws coverage for geometry type cases where Enzyme.jacobian AD is known to error (requiring fallback to FiniteDifference) so that we could monitor for improved future support.

That turned out to be more complicated than expected. The Meshes.jl (v0.54.2) parametric functions for Cylinder and CylinderSurface actually seem to Enzyme.jacobian without issue in Julia v1.10.10 but then error when tested in Julia v1.11.6. The likely cause seems to be Julia compiling the same code down to different LLVM with instructions that Enzyme can't differentiate.

[github-actions]

Benchmark Results (Julia v1.10)

Time benchmarks

	main	73470a4...	main / 73470a4...
Differentials/Differential	0.205 ± 0.001 μs	0.206 ± 0.001 μs	0.995 ± 0.0068
Differentials/Jacobian	0.169 ± 0.0001 μs	0.169 ± 0.001 μs	1 ± 0.0059
Integrals/Segment/Scalar GaussKronrod	0.641 ± 0.019 μs	0.621 ± 0.019 μs	1.03 ± 0.044
Integrals/Segment/Scalar GaussLegendre	1.66 ± 0.01 μs	1.67 ± 0.011 μs	0.994 ± 0.0089
Integrals/Segment/Scalar HAdaptiveCubature	0.872 ± 0.07 μs	0.841 ± 0.031 μs	1.04 ± 0.092
Integrals/Segment/Vector GaussKronrod	2.72 ± 0.25 μs	2.75 ± 0.28 μs	0.989 ± 0.14
Integrals/Segment/Vector GaussLegendre	17.5 ± 0.48 μs	17.6 ± 0.55 μs	0.991 ± 0.041
Integrals/Segment/Vector HAdaptiveCubature	3.64 ± 0.32 μs	3.72 ± 0.36 μs	0.978 ± 0.13
Integrals/Sphere/Scalar GaussKronrod	0.0688 ± 0.00057 ms	0.07 ± 0.00067 ms	0.984 ± 0.012
Integrals/Sphere/Scalar GaussLegendre	1.87 ± 0.011 ms	1.96 ± 0.012 ms	0.953 ± 0.0083
Integrals/Sphere/Scalar HAdaptiveCubature	0.0478 ± 0.00011 ms	0.0478 ± 9.1e-05 ms	0.998 ± 0.003
Integrals/Sphere/Vector GaussKronrod	0.104 ± 0.0015 ms	0.103 ± 0.0016 ms	1 ± 0.021
Integrals/Sphere/Vector GaussLegendre	3.35 ± 0.072 ms	3.31 ± 0.096 ms	1.01 ± 0.037
Integrals/Sphere/Vector HAdaptiveCubature	0.0992 ± 0.0016 ms	0.099 ± 0.0015 ms	1 ± 0.022
Rules/GaussLegendre	21.9 ± 0.6 μs	21.9 ± 0.62 μs	1 ± 0.04
Specializations/Scalar GaussLegendre/BezierCurve	0.253 ± 0.0069 ms	0.253 ± 0.007 ms	1 ± 0.039
Specializations/Scalar GaussLegendre/Line	7.02 ± 0.11 μs	6.95 ± 0.08 μs	1.01 ± 0.02
Specializations/Scalar GaussLegendre/Plane	0.743 ± 0.0039 ms	0.744 ± 0.0028 ms	0.998 ± 0.0065
Specializations/Scalar GaussLegendre/Ray	5.92 ± 0.06 μs	5.9 ± 0.05 μs	1 ± 0.013
Specializations/Scalar GaussLegendre/Rope	0.0513 ± 0.00027 ms	0.0515 ± 0.00023 ms	0.996 ± 0.0069
Specializations/Scalar GaussLegendre/Tetrahedron	0.161 ± 0.0012 s	0.155 ± 0.0015 s	1.04 ± 0.013
Specializations/Scalar GaussLegendre/Triangle	0.602 ± 0.0073 ms	0.691 ± 0.014 ms	0.871 ± 0.021
time_to_load	1.44 ± 0.055 s	1.39 ± 0.013 s	1.03 ± 0.041

Memory benchmarks

	main	73470a4...	main / 73470a4...
Differentials/Differential	3 allocs: 0.0938 kB	3 allocs: 0.0938 kB	1
Differentials/Jacobian	0 allocs: 0 B	0 allocs: 0 B
Integrals/Segment/Scalar GaussKronrod	2 allocs: 0.0469 kB	2 allocs: 0.0469 kB	1
Integrals/Segment/Scalar GaussLegendre	0 allocs: 0 B	0 allocs: 0 B
Integrals/Segment/Scalar HAdaptiveCubature	7 allocs: 0.492 kB	7 allocs: 0.492 kB	1
Integrals/Segment/Vector GaussKronrod	0.063 k allocs: 4.88 kB	0.063 k allocs: 4.88 kB	1
Integrals/Segment/Vector GaussLegendre	0.606 k allocs: 0.04 MB	0.606 k allocs: 0.04 MB	1
Integrals/Segment/Vector HAdaptiveCubature	0.085 k allocs: 6.75 kB	0.085 k allocs: 6.75 kB	1
Integrals/Sphere/Scalar GaussKronrod	0.198 k allocs: 5.19 kB	0.198 k allocs: 5.19 kB	1
Integrals/Sphere/Scalar GaussLegendre	0 allocs: 0 B	0 allocs: 0 B
Integrals/Sphere/Scalar HAdaptiveCubature	7 allocs: 0.648 kB	7 allocs: 0.648 kB	1
Integrals/Sphere/Vector GaussKronrod	1.11 k allocs: 0.0776 MB	1.11 k allocs: 0.0776 MB	1
Integrals/Sphere/Vector GaussLegendre	0.06 M allocs: 3.97 MB	0.06 M allocs: 3.97 MB	1
Integrals/Sphere/Vector HAdaptiveCubature	1.42 k allocs: 0.108 MB	1.42 k allocs: 0.108 MB	1
Rules/GaussLegendre	7 allocs: 0.0325 MB	7 allocs: 0.0325 MB	1
Specializations/Scalar GaussLegendre/BezierCurve	23 allocs: 0.75 kB	23 allocs: 0.75 kB	1
Specializations/Scalar GaussLegendre/Line	21 allocs: 0.938 kB	21 allocs: 0.938 kB	1
Specializations/Scalar GaussLegendre/Plane	22 allocs: 1.03 kB	22 allocs: 1.03 kB	1
Specializations/Scalar GaussLegendre/Ray	21 allocs: 0.938 kB	21 allocs: 0.938 kB	1
Specializations/Scalar GaussLegendre/Rope	0.068 k allocs: 1.62 kB	0.068 k allocs: 1.62 kB	1
Specializations/Scalar GaussLegendre/Tetrahedron	3 M allocs: 0.179 GB	3 M allocs: 0.179 GB	1
Specializations/Scalar GaussLegendre/Triangle	22 allocs: 1.03 kB	22 allocs: 1.03 kB	1
time_to_load	0.153 k allocs: 14.5 kB	0.153 k allocs: 14.5 kB	1

[mikeingold]

Weird. GitHub CI indicates test failures (no error) on

MeshIntegrals.jl/test/utils.jl

Line 44 in c04420f

@test_throws "jl_get_builtin_fptr" jacobian(cyl, (0.5, 0.5, 0.5), AutoEnzyme())

and

MeshIntegrals.jl/test/utils.jl

Line 49 in c04420f

@test_throws "jl_get_builtin_fptr" jacobian(cylsurf, (0.5, 0.5), AutoEnzyme())

but these pass (error) on my local computer.

[ Info: Julia test item process launching
  Activating new project at `C:\Users\*\AppData\Local\Temp\jl_6xwBkW`
   Resolving package versions...
    Updating `C:\Users\*\AppData\Local\Temp\jl_6xwBkW\Project.toml`
  [dadec2fd] + MeshIntegrals v0.16.3 `c:\Users\*\Dev\MeshIntegrals.jl`
    Updating `C:\Users\*\AppData\Local\Temp\jl_6xwBkW\Manifest.toml`
...
  [dadec2fd] + MeshIntegrals v0.16.3 `c:\Users\*\Dev\MeshIntegrals.jl`
  [eacbb407] + Meshes v0.54.2
...
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling MeshIntegralsEnzymeExt...
   4333.4 ms  ✓ MeshIntegrals → MeshIntegralsEnzymeExt
  1 dependency successfully precompiled in 6 seconds. 181 already precompiled.
Test Summary:                                                                        | Pass  Total   Time
c:\Users\*\Dev\MeshIntegrals.jl\test\utils.jl:Differentiation (EnzymeExt loaded) |   16     16  42.9s

[JoshuaLampert]

Interesting. Do you also use Julia v1.10 and the same versions of Enzyme.jl and Enzyme_jll.jl as in CI?

[mikeingold]

Interesting. Do you also use Julia v1.10 and the same versions of Enzyme.jl and Enzyme_jll.jl as in CI?

That test run was in 1.11.5 locally via VS Code's Julia extension. I just used juliaup to switch my default down to 1.10.10 and now I get the same result as CI. I'm debugging now to see if I can figure out which Julia/package caused the difference.

[mikeingold]

Tests from my local machine:

Software	Config A	Config B
Julia	1.10.10	1.11.6
Enzyme.jl	0.13.62	0.13.62
Meshes.jl	0.54.2	0.54.2

Config A: No errors thrown on jacobian(::Cylinder, ts..., AutoEnzyme()) or jacobian(::CylinderSurface, ts..., AutoEnzyme()), which causes @test_throws failures.

Config B: Errors ("jl_get_builtin_fptr") thrown on jacobian(::Cylinder, ts..., AutoEnzyme()) and jacobian(::CylinderSurface, ts..., AutoEnzyme()), so tests pass.

I'm not sure how to check the Enzyme jll versions to see if those are different. This does seem like some kind of regression, but I've got no idea from where.

[wsmoses]

this is a julia version issue, Julia emits different code in the two cases

[mikeingold]

this is a julia version issue, Julia emits different code in the two cases

So I guess technically not a bug or regression in that the newer version of Julia is compiling the code differently and seems to be using something that Enzyme doesn't know how to handl?

[wsmoses]

yeah pretty much

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (8bd66b4) to head (73470a4).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #193   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           19        19           
  Lines          190       198    +8     
=========================================
+ Hits           190       198    +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.