Add GPU tests for CUDA, AMDGPU, and OpenCL backends (#205)

Krastanov-agent · Claude · claude · web-flow · commit 6f91d1569a88 · 2025-08-16T20:57:01.000-04:00
* Add CLAUDE.md documentation file Adds comprehensive documentation for Claude Code users including: - Project structure overview - Development commands for testing and documentation - Key dependencies and version requirements - Contributing guidelines 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add code formatting guidelines to CLAUDE.md Includes: - Commands to remove trailing whitespaces from .jl files - Cross-platform sed commands (Linux and macOS) - General formatting guidelines for Julia code - Updated contributing section to reference formatting 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add GPU tests for CUDA, AMDGPU, and OpenCL backends Implements comprehensive GPU testing infrastructure: - GPU test structure based on QuantumClifford.jl patterns - Support for CUDA, AMDGPU (ROCm), and OpenCL backends - Basic operations testing: operators, kets, bras - Matrix operations: multiplication, adjoint, trace, norm - Test filtering by environment variables (CUDA_TEST, AMDGPU_TEST, OpenCL_TEST) - Buildkite CI configuration for automated GPU testing - Uses Adapt.jl for seamless CPU-GPU array adaptation Tests verified working locally with OpenCL backend. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Update CLAUDE.md with master version and add pull requirement - Uses CLAUDE.md content from master branch to resolve conflicts - Adds requirement to pull latest changes before starting new features - Emphasizes keeping repository up to date in contributing guidelines 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve test setup and add branch pull requirement to CLAUDE.md - Add Pkg to test/Project.toml for proper dependency management - Remove manual TestItemRunner addition (already in test environment) - Update CLAUDE.md with branch pull requirement - Recommend using Pkg.test() over direct runtests.jl calls - Add examples for GPU-specific test commands - All 61 OpenCL tests passing with new setup 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Remove incorrect alternative test command from CLAUDE.md The direct runtests.jl approach was incorrect and shouldn't be recommended. Only Pkg.test() provides the proper test environment. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add newline formatting guidelines and fix missing newlines - Add section in CLAUDE.md for ensuring files end with newlines - Provide sed command to add missing newlines: `sed -i '$a\' *.jl` - Apply the command to fix existing files missing newlines - Update formatting guidelines to emphasize proper file endings 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * manual changes * Revert formatting-only changes and add PR focus guideline Remove newline additions from 28 test files that were not part of the GPU tests feature. Add guideline to CLAUDE.md about keeping PRs focused and avoiding mixing feature work with formatting changes to unrelated files. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * minor clarifications * comment out the AMDGPU tests --------- Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Stefan Krastanov <stefan@krastanov.org> Co-authored-by: Stefan Krastanov <github.acc@krastanov.org>
diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
@@ -0,0 +1,100 @@
+env:
+  CODECOV_TOKEN: adb3f22a-231a-4f7b-8ed4-7c6c56453cbe
+  JULIA_NUM_THREADS: auto
+  PYTHON: ""
+  PYCALL_DEBUG_BUILD: yes
+
+steps:
+  - label: "Package Tests"
+    plugins:
+      - JuliaCI/julia#v1:
+          version: "1"
+      - QuantumSavory/julia-xvfb#v1:
+      - JuliaCI/julia-test#v1: ~
+      - JuliaCI/julia-coverage#v1:
+          codecov: true
+    command:
+      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+      - julia --project='~' -e '
+        using Pkg;
+        pkg"dev .";'
+
+  - label: "JET Tests"
+    plugins:
+      - JuliaCI/julia#v1:
+          version: "1"
+      - QuantumSavory/julia-xvfb#v1:
+      - JuliaCI/julia-test#v1: ~
+      - JuliaCI/julia-coverage#v1:
+          codecov: true
+    env:
+      JET_TEST: true
+    command:
+      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+      - julia --project='~' -e '
+        using Pkg;
+        pkg"dev .";'
+
+  - label: "CUDA Tests"
+    agents:
+      queue: cuda
+    plugins:
+      - JuliaCI/julia#v1:
+          version: "1"
+      - JuliaCI/julia-test#v1: ~
+      - JuliaCI/julia-coverage#v1:
+          codecov: true
+    env:
+      CUDA_TEST: true
+    command:
+      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+      - julia --project='~' -e '
+        using Pkg;
+        pkg"dev .";'
+
+#  - label: "AMDGPU Tests"
+#    agents:
+#      queue: rocm
+#    plugins:
+#      - JuliaCI/julia#v1:
+#          version: "1"
+#      - JuliaCI/julia-test#v1: ~
+#      - JuliaCI/julia-coverage#v1:
+#          codecov: true
+#    env:
+#      AMDGPU_TEST: true
+#    command:
+#      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+#      - julia --project='~' -e '
+#        using Pkg;
+#        pkg"dev .";'
+
+  - label: "OpenCL Tests"
+    plugins:
+      - JuliaCI/julia#v1:
+          version: "1"
+      - QuantumSavory/julia-xvfb#v1:
+      - JuliaCI/julia-test#v1: ~
+      - JuliaCI/julia-coverage#v1:
+          codecov: true
+    env:
+      OpenCL_TEST: true
+    command:
+      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+      - julia --project='~' -e '
+        using Pkg;
+        pkg"dev .";'
+
+  - label: "Downstream QuantumOptics"
+    plugins:
+      - JuliaCI/julia#v1:
+          version: "1"
+      - QuantumSavory/julia-xvfb#v1:
+    command:
+      - echo "Julia depot path $${JULIA_DEPOT_PATH}"
+      - julia --project=$(mktemp -d) -e '
+        using Pkg;
+        pkg"dev .";
+        Pkg.add("QuantumOptics");
+        Pkg.build("QuantumOptics");
+        Pkg.test("QuantumOptics");'
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -21,6 +21,11 @@ QuantumOpticsBase.jl provides the base functionality for QuantumOptics.jl. It im
 ```bash
 # Run all tests
 julia --project=. -e "using Pkg; Pkg.test()"
+
+# Run with only specific GPU backend tests
+CUDA_TEST=true julia --project=. -e "using Pkg; Pkg.test()"
+AMDGPU_TEST=true julia --project=. -e "using Pkg; Pkg.test()"
+OpenCL_TEST=true julia --project=. -e "using Pkg; Pkg.test()"
 ```
 
 ### Building Documentation
@@ -84,13 +89,21 @@ Special test configurations:
 ## Code Formatting
 
 ### Removing Trailing Whitespaces
-Before committing, ensure there are no trailing whitespaces in Julia files:
+Before committing, ensure there are no trailing whitespaces in Julia files. Do not format files that are not part of the specific feature under development.
 
 ```bash
 # Remove trailing whitespaces from all .jl files (requires gnu tools)
 find . -type f -name '*.jl' -exec sed --in-place 's/[[:space:]]\+$//' {} \+
 ```
 
+### Ensuring Files End with Newlines
+Ensure all Julia files end with a newline to avoid misbehaving CLI tools. Do not format files that are not part of the specific feature under development.
+
+```bash
+# Add newline to end of all .jl files that don't have one
+find . -type f -name '*.jl' -exec sed -i '$a\' {} \+
+```
+
 ### General Formatting Guidelines
 - Use 4 spaces for indentation (no tabs)
 - Remove trailing whitespaces from all lines
@@ -101,8 +114,13 @@ find . -type f -name '*.jl' -exec sed --in-place 's/[[:space:]]\+$//' {} \+
 ## Contributing
 
 This package follows standard Julia development practices:
+- **Always pull latest changes first**: Before creating any new feature or starting work, ensure you have the latest version by running `git pull origin master` (or `git pull origin main`)
+- **Pull before continuing work**: Other maintainers might have modified the branch you are working on. Always call `git pull` before continuing work on an existing branch
+- **Push changes to remote**: Always push your local changes to the remote branch to keep the PR up to date: `git push origin <branch-name>`
+- **Run all tests before submitting**: Before creating or updating a PR, always run the full test suite to ensure nothing is broken: `julia --project=. -e "using Pkg; Pkg.test()"`
 - Fork and create feature branches
 - Write tests for new functionality
 - Ensure documentation builds successfully
 - Follow code formatting guidelines above
-- All tests must pass before merging
+- All tests must pass before merging
+- **Keep PRs focused**: A PR should implement one self-contained change. Avoid mixing feature work with formatting changes to unrelated files, even for improvements like adding missing newlines. Format unrelated files in separate commits or PRs.
diff --git a/test/Project.toml b/test/Project.toml
@@ -7,6 +7,7 @@ FillArrays = "1a297f60-69ca-5386-bcde-b61e274b549b"
 JET = "c3a54625-cd67-489e-a8e7-0a5a0ff4e31b"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 OrdinaryDiffEq = "1dea7af3-3e70-54e6-95c3-0bf5283fa5ed"
+Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
 QuantumInterface = "5717a53b-5d69-4fa3-b976-0bf2f97ca1e5"
 QuantumOptics = "6e0679c1-51ea-5a7c-ac74-d61b76210b0c"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
diff --git a/test/gpu/implementation/definitions.jl b/test/gpu/implementation/definitions.jl
@@ -0,0 +1,13 @@
+# Test parameters
+const test_sizes = [2, 4, 8]
+const max_rows = 16
+const round_count = 2
+const block_sizes = fill(64, round_count)
+const batch_sizes = fill(1, round_count)
+
+# Helper function for distance comparison
+D(op1::AbstractOperator, op2::AbstractOperator) = abs(tracedistance_nh(dense(op1), dense(op2)))
+D(x1::StateVector, x2::StateVector) = norm(x2-x1)
+
+# Test tolerances for GPU computations
+const GPU_TOL = 1e-10
diff --git a/test/gpu/implementation/imports.jl b/test/gpu/implementation/imports.jl
@@ -0,0 +1,8 @@
+# GPU backend imports
+import Adapt
+using QuantumOpticsBase
+using LinearAlgebra, Random, Test
+using SparseArrays
+
+# For memory management in GPU tests
+using GPUArrays: AllocCache, @cached, unsafe_free!
diff --git a/test/gpu/implementation/test_basic_operations.jl b/test/gpu/implementation/test_basic_operations.jl
@@ -0,0 +1,68 @@
+function test_basic_operations(AT, synchronize)
+    """Test basic quantum operations on GPU arrays."""
+    cache = AllocCache()
+    
+    for n in test_sizes
+        for r in 1:round_count
+            @cached cache begin
+                
+                @testset "Basic Operations - Size $n" begin
+                    # Create test bases
+                    b1 = GenericBasis(n)
+                    b2 = GenericBasis(n)
+                    
+                    # Test operator creation and adaptation
+                    cpu_op, gpu_op = create_test_operator(b1, b2, AT)
+                    
+                    @test typeof(gpu_op.data) <: AT
+                    @test verify_gpu_result(cpu_op, gpu_op)
+                    
+                    # Test ket creation and adaptation  
+                    cpu_ket, gpu_ket = create_test_ket(b1, AT)
+                    
+                    @test typeof(gpu_ket.data) <: AT
+                    @test verify_gpu_result(cpu_ket, gpu_ket)
+                    
+                    # Test bra creation and adaptation
+                    cpu_bra, gpu_bra = create_test_bra(b1, AT)
+                    
+                    @test typeof(gpu_bra.data) <: AT
+                    @test verify_gpu_result(cpu_bra, gpu_bra)
+                    
+                    # Test operator multiplication (both on GPU)
+                    cpu_op2, gpu_op2 = create_test_operator(b2, b1, AT)
+                    
+                    cpu_result = cpu_op * cpu_op2
+                    gpu_result = gpu_op * gpu_op2
+                    synchronize()
+                    
+                    @test verify_gpu_result(cpu_result, gpu_result)
+                    
+                    # Test adjoint/dagger operation
+                    cpu_dag = dagger(cpu_op)
+                    gpu_dag = dagger(gpu_op)
+                    synchronize()
+                    
+                    @test verify_gpu_result(cpu_dag, gpu_dag)
+                    
+                    # Test trace operation
+                    if n == length(b2)  # Square matrices only
+                        cpu_trace = tr(cpu_op)
+                        gpu_trace = tr(gpu_op)
+                        synchronize()
+                        
+                        @test isapprox(cpu_trace, gpu_trace, atol=GPU_TOL)
+                    end
+                    
+                    # Test norm operations
+                    cpu_norm = norm(cpu_ket)
+                    gpu_norm = norm(gpu_ket)
+                    synchronize()
+                    
+                    @test isapprox(cpu_norm, gpu_norm, atol=GPU_TOL)
+                end
+                
+            end
+        end
+    end
+end
diff --git a/test/gpu/implementation/test_platform.jl b/test/gpu/implementation/test_platform.jl
@@ -0,0 +1,10 @@
+include("imports.jl")
+include("definitions.jl")
+include("utilities.jl")
+include("test_basic_operations.jl")
+
+@inline function test_platform(AT, synchronize)
+    @testset "Basic Operations" begin
+        test_basic_operations(AT, synchronize)
+    end
+end
diff --git a/test/gpu/implementation/utilities.jl b/test/gpu/implementation/utilities.jl
@@ -0,0 +1,36 @@
+# Utility functions for GPU testing
+
+function create_test_operator(basis_l, basis_r, AT)
+    """Create a test operator and adapt it to the specified array type."""
+    data = rand(ComplexF64, length(basis_l), length(basis_r))
+    cpu_op = DenseOperator(basis_l, basis_r, data)
+    gpu_op = Adapt.adapt(AT, cpu_op)
+    return cpu_op, gpu_op
+end
+
+function create_test_ket(basis, AT)
+    """Create a test ket and adapt it to the specified array type."""
+    data = rand(ComplexF64, length(basis))
+    normalize!(data)
+    cpu_ket = Ket(basis, data)
+    gpu_data = Adapt.adapt(AT, data)
+    gpu_ket = Ket(basis, gpu_data)
+    return cpu_ket, gpu_ket
+end
+
+function create_test_bra(basis, AT)
+    """Create a test bra and adapt it to the specified array type."""
+    data = rand(ComplexF64, length(basis))
+    normalize!(data)
+    cpu_bra = Bra(basis, data)
+    gpu_data = Adapt.adapt(AT, data)
+    gpu_bra = Bra(basis, gpu_data)
+    return cpu_bra, gpu_bra
+end
+
+function verify_gpu_result(cpu_result, gpu_result, tolerance=GPU_TOL)
+    """Verify that GPU computation matches CPU result within tolerance."""
+    cpu_data = cpu_result isa AbstractOperator ? cpu_result.data : cpu_result.data
+    gpu_data_cpu = Array(gpu_result isa AbstractOperator ? gpu_result.data : gpu_result.data)
+    return isapprox(cpu_data, gpu_data_cpu, atol=tolerance)
+end
diff --git a/test/gpu/test_platform_AMDGPU.jl b/test/gpu/test_platform_AMDGPU.jl
@@ -0,0 +1,21 @@
+@testitem "AMDGPU" tags = [:amdgpu] begin
+
+    include("implementation/test_platform.jl")
+
+    using AMDGPU: ROCArray, AMDGPU
+    const AT = ROCArray
+
+    const can_run = AMDGPU.functional()
+
+    @testset "Device availability" begin
+        @test can_run
+    end
+
+    if can_run
+        synchronize() = AMDGPU.synchronize()
+        test_platform(AT, synchronize)
+    else
+        @info "Skipping AMDGPU tests - AMDGPU not functional"
+    end
+
+end
diff --git a/test/gpu/test_platform_CUDA.jl b/test/gpu/test_platform_CUDA.jl
@@ -0,0 +1,21 @@
+@testitem "CUDA" tags = [:cuda] begin
+
+    include("implementation/test_platform.jl")
+
+    using CUDA: CuArray, CUDA
+    const AT = CuArray
+
+    const can_run = CUDA.functional()
+
+    @testset "Device availability" begin
+        @test can_run
+    end
+
+    if can_run
+        synchronize() = CUDA.synchronize()
+        test_platform(AT, synchronize)
+    else
+        @info "Skipping CUDA tests - CUDA not functional"
+    end
+
+end
diff --git a/test/gpu/test_platform_OpenCL.jl b/test/gpu/test_platform_OpenCL.jl
@@ -0,0 +1,28 @@
+@testitem "OpenCL" tags = [:opencl] begin
+
+    include("implementation/test_platform.jl")
+
+    import pocl_jll
+    using OpenCL: CLArray, cl
+    const AT = CLArray
+
+    const can_run = try
+        length(cl.platforms()) > 0 && any(
+            length(cl.devices(platform)) > 0 for platform in cl.platforms()
+        )
+    catch
+        false
+    end
+
+    @testset "Device availability" begin
+        @test can_run
+    end
+
+    if can_run
+        synchronize() = cl.finish(cl.queue())
+        test_platform(AT, synchronize)
+    else
+        @info "Skipping OpenCL tests - no devices available"
+    end
+
+end
diff --git a/test/runtests.jl b/test/runtests.jl
