Interpolation GPU Offload

[l90lpa]

Background:

I have been working on and have completed a first pass at enabling the GPU offload of Atlas's matrix-based interpolation routines.

This work was initially submitted as 2 PR's:

• Add wrappers for cu/hipsparse to hic ecmwf/atlas#237 (branch: feature/hicsparse): adds wrappers for cuSparse/hipSparse into hic
• Offload matrix based interpolation ecmwf/atlas#239 (branch: feature/hicsparse-sparse-linalg-backend):
  • adds a multiply-add interface to sparse linalg
  • adds a sparse matrix class which supports GPU memory management
  • adds a hicsparse backend to sparse linalg
  • updates interpolation to support use of linalg sparse hicsparse backend

However, after discussion with Willem it was decided to break-up the second PR into 4 smaller PRs based on the above breakdown (see ecmwf#239 (comment)).

Current State:

Below I've presented the current state broken down by branch

0. branch: feature/hicsparse (merged into Atlas - no further actions requried):

   • adds wrappers for cuSparse/hipSparse into hic
   • currently based on tag 0.40.0
   • PR submitted, Add wrappers for cu/hipsparse to hic ecmwf/atlas#237, and has been reviewed. It is just waiting on Willem to test on an AMD platform. This PR has no dependencies.

1. branch: feature/sparse-linalg-multiply-add (merged into Atlas - no further actions requried):

   • adds a multiply-add interface to sparse linalg
   • currently based on tag 0.40.0
   • PR submitted, Extend sparse linear algebra interface with multiply-add ecmwf/atlas#240, and has not yet been reviewed. This PR has no dependencies, however, there is a slight overlap in scope with feature/add-gpu-offload-capable-sparse-matrix which also makes modification to files within atlas/linalg/sparse, although the only merge conflict might be in atlas/test/linalg/CMakeLists.txt where 2 tests have been added in the same place.
   • Agreed with Willem to leave introducing $\alpha$ and $\beta$ into interface as a follow-on (see Extend sparse linear algebra interface with multiply-add ecmwf/atlas#240 (comment))

2. branch: feature/add-gpu-offload-capable-sparse-matrix (superseded by feature/sparse_storage_with_view):

   • PR opened, Add GPU offloadable SparseMatrixStorage and read-only SparseMatrixView. ecmwf/atlas#247, and under active development by Willem.

3. branch: feature/add-hicsparse-backend-to-linalg-sparse (merged into Atlas - no further actions requried):

   • adds a hicsparse backend to sparse linalg
   • currently based on feature/sparse_storage_with_view, and depends on no other branches.
   • PR submitted, Add a hicsparse backend to linalg sparse ecmwf/atlas#246 and depends on Add GPU offloadable SparseMatrixStorage and read-only SparseMatrixView. ecmwf/atlas#247. Once dependencies are merged the branch of this PR should be rebased onto develop (any fixes made dues to changes in dependencies) and a PR submitted. If one wants to test this branch it is straight-forward to simply merge in its dependencies.
   • Note: this PR previously depended on Add GPU offload capable sparse matrix ecmwf/atlas#241, but Add GPU offload capable sparse matrix ecmwf/atlas#241 has been superseded by Add GPU offloadable SparseMatrixStorage and read-only SparseMatrixView. ecmwf/atlas#247

4. branch: feature/update-interpolation-to-support-sparse-linalg-hicsparse-backend (waiting on dependencies to merge):

   • updates interpolation to support use of linalg sparse hicsparse backend
   • currently based on feature/sparse_storage_with_view and depends on feature/add-hicsparse-backend-to-linalg-sparse.
   • No PR submitted yet. Once all of this branches dependencies are merged this branch should be rebased onto develop (and any fixes made dues to changes in dependencies). If one wants to test this branch it is straight-forward to simply merge in its dependencies.
   • Note: on rebasing this branch onto feature/sparse_storage_with_view I made some simplifications along side necessary changes.