Skip to content

Feat: Adding Linear Algebra Dot operation support #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
Jul 22, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
bc333df
interfacing with qblas
SwayamInSync Jul 9, 2025
babaa96
adding test cases
SwayamInSync Jul 9, 2025
c3aaa05
test-1: ci
SwayamInSync Jul 9, 2025
ca7dd6d
fixing ci
SwayamInSync Jul 9, 2025
04314e3
fixing ci
SwayamInSync Jul 9, 2025
037021a
fixing ci
SwayamInSync Jul 9, 2025
d6fc9c6
fixing linux CI
SwayamInSync Jul 10, 2025
f99f565
fixing linux CI
SwayamInSync Jul 10, 2025
fb3579c
fixing linux CI
SwayamInSync Jul 10, 2025
03e9acd
fixing linux CI
SwayamInSync Jul 10, 2025
1ed7bab
fixing linux CI
SwayamInSync Jul 10, 2025
63a355e
fixing linux CI
SwayamInSync Jul 10, 2025
88a98d1
fixing linux CI
SwayamInSync Jul 10, 2025
764fc72
updating qblas:
SwayamInSync Jul 10, 2025
1669e5f
fixing macos CI
SwayamInSync Jul 10, 2025
b35bac3
bumping macos deployment target
SwayamInSync Jul 10, 2025
042b25a
bumping macos deployment target
SwayamInSync Jul 10, 2025
f78dd90
dynamic macos deployment target
SwayamInSync Jul 10, 2025
cd88de0
explicit init of res array in dot-mat-mat
SwayamInSync Jul 10, 2025
abf0224
fixing windows CI
SwayamInSync Jul 10, 2025
c5198d1
disabling qblas for windows; MSVC incompatibility
SwayamInSync Jul 10, 2025
c0d93f8
updating CI triggering paths
SwayamInSync Jul 11, 2025
838adee
updating CI triggering paths
SwayamInSync Jul 11, 2025
433aa90
reverting branch to main
SwayamInSync Jul 11, 2025
5836505
bumping qblas
SwayamInSync Jul 11, 2025
33b48fe
umath refactor
SwayamInSync Jul 16, 2025
a35abce
updaing ci
SwayamInSync Jul 16, 2025
8516544
switching to apt
SwayamInSync Jul 16, 2025
335f425
submodule fix
SwayamInSync Jul 16, 2025
85e7840
submodule fix
SwayamInSync Jul 16, 2025
e467f4b
submodule fix
SwayamInSync Jul 16, 2025
e201b90
initial matmul ufunc setup
SwayamInSync Jul 17, 2025
09918a3
mid-way test
SwayamInSync Jul 17, 2025
70ca644
shifting to matmul ufunc
SwayamInSync Jul 17, 2025
f89c2e6
will figure out later
SwayamInSync Jul 17, 2025
894a84d
matmul registered with naive
SwayamInSync Jul 19, 2025
6800a90
adding initial qblas support to matmul ufunc, something is breaking, nan
SwayamInSync Jul 19, 2025
742ce64
matmul ufunc completed, naive plugged, qblas experimental
SwayamInSync Jul 19, 2025
d993bc9
adding release tracker to keep record for tasks, v1.0.0
SwayamInSync Jul 19, 2025
c518a29
it should be failing but passes on x86-64
SwayamInSync Jul 19, 2025
bbce2ac
ahh stupid me :), fallback to naive for MSVC
SwayamInSync Jul 19, 2025
5e5fa65
switching to internal function use only
SwayamInSync Jul 19, 2025
cec5ace
this should fix them all
SwayamInSync Jul 19, 2025
1fe6c81
wrapping up
SwayamInSync Jul 19, 2025
8f16b99
updated branch to main
SwayamInSync Jul 19, 2025
238ef89
Merge pull request #2 from SwayamInSync/matmul-ufunc
SwayamInSync Jul 19, 2025
ed47e33
added test coverage in release_tracker.md
SwayamInSync Jul 20, 2025
573eb76
more edge tests
SwayamInSync Jul 20, 2025
c795ef3
adding windows instructions and some small refactor as per reviews
SwayamInSync Jul 22, 2025
07a16c0
rename utils
SwayamInSync Jul 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 93 additions & 5 deletions quaddtype/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ np.array([1,2,3], dtype=QuadPrecDType("longdouble"))

## Installation from source

The code needs the quad precision pieces of the sleef library, which
is not available on most systems by default, so we have to generate
that first. The below assumes one has the required pieces to build
sleef (cmake and libmpfr-dev), and that one is in the package
directory locally.
The code needs the quad precision pieces of the sleef library, which is not available on most systems by default, so we have to generate that first. Choose the appropriate section below based on your operating system.

### Linux/Unix/macOS

The below assumes one has the required pieces to build sleef (cmake and libmpfr-dev), and that one is in the package directory locally.

```bash
git clone --branch 3.8 https://github.com/shibatch/sleef.git
Expand Down Expand Up @@ -68,3 +68,91 @@ python -m pip install . -v --no-build-isolation -Cbuilddir=build -C'compile-args
cd ..
python -m pytest
```

### Windows

#### Prerequisites

- **Visual Studio 2017 or later** (with MSVC compiler)
- **CMake** (≥3.15)
- **Python 3.10+**
- **Git**

#### Step-by-Step Installation

1. **Setup Development Environment**

Open a **Developer Command Prompt for VS** or **Developer PowerShell for VS** to ensure MSVC is properly configured.

2. **Clone and Build SLEEF**

```powershell
# Clone SLEEF library
git clone --branch 3.8 https://github.com/shibatch/sleef.git
cd sleef

# Configure with CMake for Windows
cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DSLEEF_BUILD_QUAD:BOOL=ON -DSLEEF_BUILD_SHARED_LIBS:BOOL=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON

# Build and install SLEEF
cmake --build build --config Release
cmake --install build --prefix "C:/sleef" --config Release

cd ..
```

3. **Setup Python Environment**

```powershell
# Create and activate virtual environment
python -m venv numpy_quad_env
.\numpy_quad_env\Scripts\Activate.ps1

# Install build dependencies
pip install -U pip
pip install meson-python numpy pytest ninja meson
```

4. **Set Environment Variables**

```powershell
# Set up paths and compiler flags
$env:INCLUDE = "C:/sleef/include;$env:INCLUDE"
$env:LIB = "C:/sleef/lib;$env:LIB"
$env:PATH = "C:/sleef/bin;$env:PATH"

# Note: QBLAS is disabled on Windows due to MSVC compatibility issues
$env:CFLAGS = "/IC:/sleef/include /DDISABLE_QUADBLAS"
$env:CXXFLAGS = "/IC:/sleef/include /DDISABLE_QUADBLAS"
$env:LDFLAGS = "C:/sleef/lib/sleef.lib C:/sleef/lib/sleefquad.lib"
```

5. **Build and Install numpy-quaddtype**

```powershell
# Ensure submodules are initialized
git submodule update --init --recursive

# Build and install the package
python -m pip install . -v --no-build-isolation -Cbuilddir=build -C'compile-args=-v'
```

6. **Test Installation**

```powershell
# Run tests
pytest -s tests/
```

1. **QBLAS Disabled**: QuadBLAS optimization is automatically disabled on Windows builds due to MSVC compatibility issues. This is handled by the `-DDISABLE_QUADBLAS` compiler flag.

2. **Visual Studio Version**: The instructions assume Visual Studio 2022. For other versions, adjust the generator string:
- VS 2019: `"Visual Studio 16 2019"`
- VS 2017: `"Visual Studio 15 2017"`

3. **Architecture**: The instructions are for x64. For x86 builds, change `-A x64` to `-A Win32`.

4. **Alternative SLEEF Location**: If you prefer to install SLEEF elsewhere, update all path references accordingly.

#### Windows Troubleshooting
- **Link errors**: Verify that `sleef.lib` and `sleefquad.lib` exist in `C:/sleef/lib/`
2 changes: 0 additions & 2 deletions quaddtype/numpy_quaddtype/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@
get_quadblas_version
)

import multiprocessing

__all__ = [
'QuadPrecision', 'QuadPrecDType', 'SleefQuadPrecision', 'LongDoubleQuadPrecision',
'SleefQuadPrecDType', 'LongDoubleQuadPrecDType', 'is_longdouble_128',
Expand Down
13 changes: 9 additions & 4 deletions quaddtype/numpy_quaddtype/src/umath/matmul.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ quad_matmul_resolve_descriptors(PyObject *self, PyArray_DTypeMeta *const dtypes[
if (descr_in1->backend != BACKEND_SLEEF || descr_in2->backend != BACKEND_SLEEF) {
PyErr_SetString(PyExc_NotImplementedError,
"QBLAS-accelerated matmul only supports SLEEF backend. "
"Other backends are not supported with QBLAS.");
"Please raise the issue at SwayamInSync/QBLAS for longdouble support");
return (NPY_CASTING)-1;
}

Expand All @@ -61,7 +61,8 @@ quad_matmul_resolve_descriptors(PyObject *self, PyArray_DTypeMeta *const dtypes[
QuadPrecDTypeObject *descr_out = (QuadPrecDTypeObject *)given_descrs[2];
if (descr_out->backend != target_backend) {
PyErr_SetString(PyExc_NotImplementedError,
"QBLAS-accelerated matmul only supports SLEEF backend for output.");
"QBLAS-accelerated matmul only supports SLEEF backend. "
"Please raise the issue at SwayamInSync/QBLAS for longdouble support");
return (NPY_CASTING)-1;
}
else {
Expand Down Expand Up @@ -118,7 +119,9 @@ quad_matmul_strided_loop_aligned(PyArrayMethod_Context *context, char *const dat

QuadPrecDTypeObject *descr = (QuadPrecDTypeObject *)context->descriptors[0];
if (descr->backend != BACKEND_SLEEF) {
PyErr_SetString(PyExc_RuntimeError, "Internal error: non-SLEEF backend in QBLAS matmul");
PyErr_SetString(PyExc_NotImplementedError,
"QBLAS-accelerated matmul only supports SLEEF backend. "
"Please raise the issue at SwayamInSync/QBLAS for longdouble support");
return -1;
}

Expand Down Expand Up @@ -206,7 +209,9 @@ quad_matmul_strided_loop_unaligned(PyArrayMethod_Context *context, char *const d

QuadPrecDTypeObject *descr = (QuadPrecDTypeObject *)context->descriptors[0];
if (descr->backend != BACKEND_SLEEF) {
PyErr_SetString(PyExc_RuntimeError, "Internal error: non-SLEEF backend in QBLAS matmul");
PyErr_SetString(PyExc_NotImplementedError,
"QBLAS-accelerated matmul only supports SLEEF backend. "
"Please raise the issue at SwayamInSync/QBLAS for longdouble support");
return -1;
}

Expand Down
108 changes: 2 additions & 106 deletions quaddtype/tests/test_dot.py
Original file line number Diff line number Diff line change
@@ -1,109 +1,9 @@
import pytest
import numpy as np
from test_utils import create_quad_array, assert_quad_equal, assert_quad_array_equal, arrays_equal_with_nan
from numpy_quaddtype import QuadPrecision, QuadPrecDType


# ================================================================================
# UTILITIES
# ================================================================================

def assert_quad_equal(a, b, rtol=1e-15, atol=1e-15):
"""Assert two quad precision values are equal within tolerance"""
# Ensure both operands are QuadPrecision objects for the comparison
if not isinstance(a, QuadPrecision):
a = QuadPrecision(str(a), backend='sleef')
if not isinstance(b, QuadPrecision):
b = QuadPrecision(str(b), backend='sleef')

# Use quad-precision arithmetic to calculate the difference
diff = abs(a - b)
tolerance = QuadPrecision(str(atol), backend='sleef') + QuadPrecision(str(rtol), backend='sleef') * max(abs(a), abs(b))

# Assert using quad-precision objects
assert diff <= tolerance, f"Values not equal: {a} != {b} (diff: {diff}, tol: {tolerance})"


def assert_quad_array_equal(a, b, rtol=1e-25, atol=1e-25):
"""Assert two quad precision arrays are equal within tolerance"""
assert a.shape == b.shape, f"Shapes don't match: {a.shape} vs {b.shape}"

flat_a = a.flatten()
flat_b = b.flatten()

for i, (val_a, val_b) in enumerate(zip(flat_a, flat_b)):
try:
assert_quad_equal(val_a, val_b, rtol, atol)
except AssertionError as e:
raise AssertionError(f"Arrays differ at index {i}: {e}")


def create_quad_array(values, shape=None):
"""Create a QuadPrecision array from values using Sleef backend"""
dtype = QuadPrecDType(backend='sleef')

if isinstance(values, (list, tuple)):
if shape is None:
# 1D array
quad_values = [QuadPrecision(str(float(v)), backend='sleef') for v in values]
return np.array(quad_values, dtype=dtype)
else:
# Reshape to specified shape
if len(shape) == 1:
quad_values = [QuadPrecision(str(float(v)), backend='sleef') for v in values]
return np.array(quad_values, dtype=dtype)
elif len(shape) == 2:
m, n = shape
assert len(values) == m * n, f"Values length {len(values)} doesn't match shape {shape}"
quad_matrix = []
for i in range(m):
row = [QuadPrecision(str(float(values[i * n + j])), backend='sleef') for j in range(n)]
quad_matrix.append(row)
return np.array(quad_matrix, dtype=dtype)

raise ValueError("Unsupported values or shape")


def is_special_value(val):
"""Check if a value is NaN or infinite"""
try:
float_val = float(val)
return np.isnan(float_val) or np.isinf(float_val)
except:
return False


def arrays_equal_with_nan(a, b, rtol=1e-15, atol=1e-15):
"""Compare arrays that may contain NaN values"""
if a.shape != b.shape:
return False

flat_a = a.flatten()
flat_b = b.flatten()

for i, (val_a, val_b) in enumerate(zip(flat_a, flat_b)):
# Handle NaN cases
if is_special_value(val_a) and is_special_value(val_b):
float_a = float(val_a)
float_b = float(val_b)
# Both NaN
if np.isnan(float_a) and np.isnan(float_b):
continue
# Both infinite with same sign
elif np.isinf(float_a) and np.isinf(float_b) and np.sign(float_a) == np.sign(float_b):
continue
else:
return False
elif is_special_value(val_a) or is_special_value(val_b):
return False
else:
try:
assert_quad_equal(val_a, val_b, rtol, atol)
except AssertionError:
return False

return True


# ================================================================================
# VECTOR-VECTOR DOT PRODUCT TESTS
# ================================================================================
Expand Down Expand Up @@ -789,8 +689,4 @@ def test_dimension_mismatch_matrices(self):
B = create_quad_array([1, 2, 3, 4, 5, 6], shape=(3, 2)) # Wrong size

with pytest.raises(ValueError, match=r"matmul: Input operand 1 has a mismatch in its core dimension 0"):
np.matmul(A, B)


if __name__ == "__main__":
pytest.main([__file__, "-v"])
np.matmul(A, B)
98 changes: 98 additions & 0 deletions quaddtype/tests/test_utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
from numpy_quaddtype import QuadPrecision, QuadPrecDType
import numpy as np

def assert_quad_equal(a, b, rtol=1e-15, atol=1e-15):
"""Assert two quad precision values are equal within tolerance"""
# Ensure both operands are QuadPrecision objects for the comparison
if not isinstance(a, QuadPrecision):
a = QuadPrecision(str(a), backend='sleef')
if not isinstance(b, QuadPrecision):
b = QuadPrecision(str(b), backend='sleef')

# Use quad-precision arithmetic to calculate the difference
diff = abs(a - b)
tolerance = QuadPrecision(str(atol), backend='sleef') + QuadPrecision(str(rtol), backend='sleef') * max(abs(a), abs(b))

# Assert using quad-precision objects
assert diff <= tolerance, f"Values not equal: {a} != {b} (diff: {diff}, tol: {tolerance})"


def assert_quad_array_equal(a, b, rtol=1e-25, atol=1e-25):
"""Assert two quad precision arrays are equal within tolerance"""
assert a.shape == b.shape, f"Shapes don't match: {a.shape} vs {b.shape}"

flat_a = a.flatten()
flat_b = b.flatten()

for i, (val_a, val_b) in enumerate(zip(flat_a, flat_b)):
try:
assert_quad_equal(val_a, val_b, rtol, atol)
except AssertionError as e:
raise AssertionError(f"Arrays differ at index {i}: {e}")


def create_quad_array(values, shape=None):
"""Create a QuadPrecision array from values using Sleef backend"""
dtype = QuadPrecDType(backend='sleef')

if isinstance(values, (list, tuple)):
if shape is None:
# 1D array
quad_values = [QuadPrecision(str(float(v)), backend='sleef') for v in values]
return np.array(quad_values, dtype=dtype)
else:
# Reshape to specified shape
if len(shape) == 1:
quad_values = [QuadPrecision(str(float(v)), backend='sleef') for v in values]
return np.array(quad_values, dtype=dtype)
elif len(shape) == 2:
m, n = shape
assert len(values) == m * n, f"Values length {len(values)} doesn't match shape {shape}"
quad_matrix = []
for i in range(m):
row = [QuadPrecision(str(float(values[i * n + j])), backend='sleef') for j in range(n)]
quad_matrix.append(row)
return np.array(quad_matrix, dtype=dtype)

raise ValueError("Unsupported values or shape")


def is_special_value(val):
"""Check if a value is NaN or infinite"""
try:
float_val = float(val)
return np.isnan(float_val) or np.isinf(float_val)
except:
return False


def arrays_equal_with_nan(a, b, rtol=1e-15, atol=1e-15):
"""Compare arrays that may contain NaN values"""
if a.shape != b.shape:
return False

flat_a = a.flatten()
flat_b = b.flatten()

for i, (val_a, val_b) in enumerate(zip(flat_a, flat_b)):
# Handle NaN cases
if is_special_value(val_a) and is_special_value(val_b):
float_a = float(val_a)
float_b = float(val_b)
# Both NaN
if np.isnan(float_a) and np.isnan(float_b):
continue
# Both infinite with same sign
elif np.isinf(float_a) and np.isinf(float_b) and np.sign(float_a) == np.sign(float_b):
continue
else:
return False
elif is_special_value(val_a) or is_special_value(val_b):
return False
else:
try:
assert_quad_equal(val_a, val_b, rtol, atol)
except AssertionError:
return False

return True
Loading