Linear Algebra in the Cxx Library

Please feel free to dump initial ideas, links to useful resources, etc on either this wiki or in the repository.

If you don't have write access to the GitHub repository, please email Bryce Adelstein Lelbach ([email protected]) and he'll add you.

Questions we should work on answering:

• Q: What are we trying to accomplish in this space? What are the intended use cases?
  • Graphics.
  • Scientific Computing and numerical simulation.
  • Neural network training and inference.
  • Data analytics on multidimensional data.
  • Sparse Linear Algebra?
• Q: What are the design requirements and principles?
  • Builds on top of the mdspan feature and interoperates with the 2D Graphics proposal.
  • Clear path for implementing operations with a high performance BLAS library.
  • Supports both compile-time and run-time meta-data (extents, striding, padding, etc).
  • Supports mixing compile-time and run-time meta-data (a la mdspan).
  • Separate data storage from algorithms (e.g. the string_view model).
  • Supports mixed precision and mixed type operations.
  • Supports non-fundamental numeric types (e.g. integer, fixed-point, units, chrono::duration etc.)
  • Supports arbitrary dimensions, i.e. is not limited to 2D or 3D arrays. Dimensionality support should be at least as good as Fortran 77.
• Q: What capabilities must be included in a minimum viable product (e.g. version 1)?
  • Abstractions for element iteration (e.g. multi-dimensional for loop).
  • Efficient (in both human and computer time) expression of array slicing and subarray extraction.
  • Efficient (in both human and computer time) expression of dimension/index/order permutations (aka transposition).
• Q: What are the risks?
  • Performs poorly (which could be anything from 10% to 10x overhead, depending on the user) and thus many C++ users will continue to rely on non-standard linear algebra libraries/frameworks
  • Fails to match Numpy, Matlab Fortran 90 in terms of ease of use for common operations
  • Is prohibitively difficult to learn for users with basic knowledge of linear algebra
  • Introduces run-time overhead over using graphics oriented geometry libraries (e.g. GLM)

Existing C++ Linear Algebra Projects

If you know of a project which does things like the above mentioned features list it here with a synopsis of how it addresses aspects of what we want to address.

Kokkos-Kernels

https://github.com/kokkos/kokkos-kernels

Provides BLAS, Sparse and Graph kernels on node using Kokkos.

• C++ based interface using Kokkos::View (i.e. the class mdspan is mostly based off).

  • template<class AViewType,
         class XViewType,
         class YViewType>
    void gemv (const char trans[],
      typename AViewType::const_value_type& alpha, const AViewType& A,
      const XViewType& x, typename YViewType::const_value_type& beta, const YViewType& y);

  • Can accept any Scalar type combination (i.e. AViewType, XViewType, YViewType can have different Scalar types)
  • Can accept any layouts (e.g. XViewType can be a non-contigues subarray for example a row in a column major matrix)