Initial support for CUDA + factorizations #336

kshyatt · 2026-01-02T12:48:28Z

OK, now only truncation is not working, which is expected because it is also a WIP in MatrixAlgebraKit!

github-actions · 2026-01-02T12:48:57Z

Your PR no longer requires formatting changes. Thank you for your contribution!

ext/TensorKitCUDAExt/cutensormap.jl

lkdvos · 2026-01-02T23:39:12Z

test/cuda/factorizations.jl

                @test w * p ≈ t
                @test isisometric(w)
-                @test isposdef(p)
+                @test isposdef(project_hermitian!(p))


Do we need the projection here, and if so, why? Shouldn't the result already be hermitian?

Currently in the extension we're checking the input is approximately hermitian then wrapping it in a Hermitian it if so for isposdef. I'll try removing the projection here (or maybe we need to project out the extraneous non posdef terms?).

But positive definite requires hermitian anyways right? Testing if the result is positive definite should also mean testing it is hermitian, which is why I'm confused we have to project it to be hermitian, rather than checking for that property.

I would also think that our left_polar implementations output a p factor which is exactly hermitian. If this is not the case, this is something to add to the TODO list of MAK.

I think there is also now an actual problem with the test for AdjointTensorMap (related to copying?) which I'll dig into further today

test/cuda/factorizations.jl

lkdvos

Overall definitely looks good to me!
I'm slightly worried about the code duplication for the tests, similar to how we have that for MatrixAlgebraKit, but I also don't think there are better alternatives so I'm ok with merging like this, except for some small comments.

test/cuda/factorizations.jl

codecov · 2026-01-05T06:50:23Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
ext/TensorKitCUDAExt/cutensormap.jl	`75.67% <100.00%> (+13.17%)`	⬆️
src/factorizations/adjoint.jl	`73.17% <100.00%> (+1.37%)`	⬆️
src/tensors/linalg.jl	`82.33% <100.00%> (ø)`

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Jutho · 2026-01-05T14:00:39Z

ext/TensorKitCUDAExt/cutensormap.jl

 function LinearAlgebra.isposdef(t::CuTensorMap)
    domain(t) == codomain(t) ||
        throw(SpaceMismatch("`isposdef` requires domain and codomain to be the same"))
    InnerProductStyle(spacetype(t)) === EuclideanInnerProduct() || return false
    for (c, b) in blocks(t)
        # do our own hermitian check
        isherm = MatrixAlgebraKit.ishermitian(b; atol = eps(real(eltype(b))), rtol = eps(real(eltype(b))))
        isherm || return false
-        isposdef(Hermitian(b)) || return false
+        isposdef(project_hermitian!(b)) || return false
    end
    return true
 end


Still confused by this. The CPU version doesn't have this approximate hermiticity check and then hermitian projection. It just fails directly if it is not exactly hermitian. Why does the "generic" implementation in factorizations.jl fail for CUDA? Does it not support isposdef!?

isposdef! works BUT I think CUSOLVER is very brittle about what counts as being posdef (see https://github.com/JuliaGPU/CUDA.jl/blob/0c00b83fe9df007efcb3e738fa5f1973603f79c4/lib/cusolver/dense.jl#L905)

So for example if you have an eigenvalue -1e-17 that may return "not positive definite"

I think that we should really try and follow the CPU conventions for this, so indeed maybe this should just really check for exact hermitian, and go from there.
If this gives issues for the GPU projections, we might want to alter these implementations to put a project_hermitian at the end of these, rather than try and alter the checks?

OK, so don't project_hermitian here, and make the check above exact?

The exact ishermitian check is already part of isposdef; I think if the MAK implementation of PolarViaSVD is fixed (see Zulip), this standard isposdef check should work without need for a CUDA specialisation.

ext/TensorKitCUDAExt/cutensormap.jl

src/tensors/linalg.jl

lkdvos · 2026-01-05T22:15:28Z

I think this would now be passing as soon as we tag a new version of MatrixAlgebraKit, due to the changes in QuantumKitHub/MatrixAlgebraKit.jl#143.
I would be happy with either tagging a patch release there and bumping the compat entry here, or alternatively marking the isposdef tests as broken here and fixing this later.

Co-authored-by: Lukas Devos <[email protected]>

Co-authored-by: Jutho <[email protected]>

kshyatt · 2026-01-08T18:36:26Z

OK, CUDA on 1.10 is working. CUDA on 1.12 will be fixed by JuliaGPU/GPUArrays.jl#676.

lkdvos

I would be happy to force merge this since these errors are unrelated, and just deal with the failing buildkite tests in the meantime.

In any case we'll have to tag a release of TensorKit to make sure the latest releases version has working svd and eigenvalue matrixalgebrakit implementations for diagonal

kshyatt · 2026-01-08T19:12:27Z

IMO let's push it and unblock the truncation stuff etc

kshyatt requested review from Jutho and lkdvos January 2, 2026 12:48

kshyatt force-pushed the ksh/cuda_factorizations branch from 633ceb2 to c75dc4f Compare January 2, 2026 12:50

lkdvos reviewed Jan 2, 2026

View reviewed changes

ext/TensorKitCUDAExt/cutensormap.jl Outdated Show resolved Hide resolved

lkdvos reviewed Jan 2, 2026

View reviewed changes

test/cuda/factorizations.jl Outdated Show resolved Hide resolved

lkdvos reviewed Jan 2, 2026

View reviewed changes

Jutho reviewed Jan 4, 2026

View reviewed changes

test/cuda/factorizations.jl Outdated Show resolved Hide resolved

Jutho reviewed Jan 5, 2026

View reviewed changes

ext/TensorKitCUDAExt/cutensormap.jl Outdated Show resolved Hide resolved

kshyatt force-pushed the ksh/cuda_factorizations branch from 92cd3c2 to 623edf5 Compare January 5, 2026 14:33

lkdvos reviewed Jan 5, 2026

View reviewed changes

src/tensors/linalg.jl Outdated Show resolved Hide resolved

lkdvos force-pushed the ksh/cuda_factorizations branch from 6a2d615 to 56fd179 Compare January 5, 2026 22:16

kshyatt force-pushed the ksh/cuda_factorizations branch from 56fd179 to fc83d58 Compare January 6, 2026 06:15

kshyatt mentioned this pull request Jan 7, 2026

Try running GPU + truncation to see how well it works (or doesn't) #348

Draft

kshyatt and others added 6 commits January 7, 2026 16:25

Initial support for CUDA + factorizations

585cb3c

Working adjoint

b477d54

Update test/cuda/factorizations.jl

57b676c

Co-authored-by: Lukas Devos <[email protected]>

Update test/cuda/factorizations.jl

9e3f238

Co-authored-by: Jutho <[email protected]>

Fix posdef and copy

7582ca1

Comments

1702854

kshyatt force-pushed the ksh/cuda_factorizations branch from fc83d58 to 1702854 Compare January 7, 2026 15:25

lkdvos approved these changes Jan 8, 2026

View reviewed changes

kshyatt merged commit fac25d3 into main Jan 8, 2026
40 of 41 checks passed

kshyatt deleted the ksh/cuda_factorizations branch January 8, 2026 19:50

Initial support for CUDA + factorizations #336

Initial support for CUDA + factorizations #336

Conversation

kshyatt commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lkdvos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lkdvos commented Jan 5, 2026

Uh oh!

kshyatt commented Jan 8, 2026

Uh oh!

lkdvos left a comment

Choose a reason for hiding this comment

Uh oh!

kshyatt commented Jan 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kshyatt commented Jan 2, 2026 •

edited

Loading

github-actions bot commented Jan 2, 2026 •

edited

Loading

codecov bot commented Jan 5, 2026 •

edited

Loading