ARROW-17933: [C++] SparseCOOTensor raises error when created with zero elements by rok · Pull Request #14378 · apache/arrow

rok · 2022-10-11T21:52:44Z

This is to resolve ARROW-17933.

github-actions · 2022-10-11T21:53:08Z

https://issues.apache.org/jira/browse/ARROW-17933

pitrou

Thanks for the fix.

Could you add tests on the C++ side?
Could you also add similar tests (if not already existing) for other types of sparse tensors?

pitrou · 2022-10-12T11:46:42Z

python/pyarrow/tests/test_sparse_tensor.py

Does this actually create a tensor with zero elements?

Yeah, coo_matrix automatically ignores zeros, I'll add an assertion checking for number of elements to make this explicit.

Well actually it's:

scipy_matrix = coo_matrix(([0], ([0], [0])), shape=(2, 4)) assert scipy_matrix.nnz == 1 scipy_matrix = coo_matrix([[0, 0], [0, 0]]) assert scipy_matrix.nnz == 0

Right, but the shape is all non-zero.

Well those are actually different constructors (shape seems to be ignored in this case):

1. coo_matrix((data, (i, j)), [shape=(M, N)]) to construct from three arrays: data[:] the entries of the matrix, in any order i[:] the row indices of the matrix entries j[:] the column indices of the matrix entries scipy_matrix = coo_matrix(([0], ([0], [0]))) assert scipy_matrix.nnz == 1 2. coo_matrix(D) with a dense matrix D scipy_matrix = coo_matrix([[0, 0], [0, 0]]) assert scipy_matrix.nnz == 0

https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.coo_matrix.html

But this doesn't address the issue, does it?

The reported issue was due to scipy.coo_matrix returning a sparse matrix with a dimension of zero size for an all zeros dense tensor. This is not an issue if the sparse matrix is created from components. Propose change tests from dense creation path in Python with scipy.coo_matrix, scipy.csr_matrix and sparse.COO. It also tests C++ SparseTensor creation from a dense tensor with one zero-sized dimension for SparseCOOTensor, SparseCSRMatrix, SparseCSRMatrix and SparseCSFTensor.

I might be missing the point though :D

@pitrou ping

I believe @rok has the correct test here but I could be wrong. The original issue is about a sparse matrix with a valid non-zero shape but no elements:

>>> scipy.sparse.coo_matrix(numpy.zeros((2,4)), dtype=numpy.float32) <2x4 sparse matrix of type '<class 'numpy.float32'>' with 0 stored elements in COOrdinate format> >>> numpy.zeros((2,4)) array([[0., 0., 0., 0.], [0., 0., 0., 0.]]) >>> numpy.zeros((2,4)).shape (2, 4)

rok · 2022-10-13T02:27:29Z

@pitrou I added c++ tests for a case where we create a sparse tensor from a dense one filled with zeros.

pitrou · 2022-10-13T13:45:14Z

cpp/src/arrow/sparse_tensor_test.cc

+
+TYPED_TEST_SUITE_P(TestSparseTensorFromDense);
+
+TYPED_TEST_P(TestSparseTensorFromDense, TestNonZeroLength) {


Ok, but is it the situation that the PR is meant to be fixing?
AFAIU, the problem was not when there were no non-zero values, but when the logical tensor was empty (one of the dimensions in the shape being zero).

Indeed. I changed the c++ test so that one dimension is now 0.

rok · 2023-01-07T20:47:33Z

@pitrou is there something left to address here?

amol- · 2023-03-30T17:09:13Z

Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍

rok · 2023-03-30T17:51:09Z

This is waiting for review.

bkmartinjr · 2023-03-30T18:17:57Z

We are actively using the sparse tensor classes and would greatly appreciate the fix. Thanks!

amol- · 2023-04-13T15:35:53Z

@rok who are you waiting review from? Only @AlenkaF ? @pitrou is probably unable to review this one on a short term, but maybe @westonpace can take a look

rok · 2023-04-13T16:04:40Z

I was waiting for @pitrou. I suppose @AlenkaF, @westonpace or @jorisvandenbossche would be good candidates.

AlenkaF · 2023-04-13T16:30:52Z

python/pyarrow/tests/test_sparse_tensor.py

                          sparse_tensor.to_tensor().to_numpy())
+
+    sparse_array = sparse.COO.from_numpy([[0, 0], [0, 0]])
+    sparse_tensor = pa.SparseCOOTensor.from_pydata_sparse(sparse_array,


Is there a reason why from_scipy() and to_scipy() is not used here?

westonpace · 2023-04-20T14:21:27Z

cpp/src/arrow/sparse_tensor.cc


  RETURN_NOT_OK(internal::CheckSparseIndexMaximumValue(type, shape));

+  // Indexes with no values are considered valid


I don't understand what is happening here (I don't understand much about the tensors so not surprising). Why is it ok if one of the shape elements is zero? I would expect an empty sparse matrix to still have a shape:

>>> scipy.sparse.coo_matrix(numpy.zeros((2,4)), dtype=numpy.float32).shape (2, 4)

westonpace · 2023-04-20T14:22:21Z

python/pyarrow/tests/test_sparse_tensor.py

I believe @rok has the correct test here but I could be wrong. The original issue is about a sparse matrix with a valid non-zero shape but no elements:

>>> scipy.sparse.coo_matrix(numpy.zeros((2,4)), dtype=numpy.float32) <2x4 sparse matrix of type '<class 'numpy.float32'>' with 0 stored elements in COOrdinate format> >>> numpy.zeros((2,4)) array([[0., 0., 0., 0.], [0., 0., 0., 0.]]) >>> numpy.zeros((2,4)).shape (2, 4)

westonpace · 2023-04-20T14:23:39Z

python/pyarrow/tests/test_sparse_tensor.py

+    scipy_matrix = coo_matrix([[0, 0], [0, 0]])
+    sparse_tensor = pa.SparseCOOTensor.from_scipy(scipy_matrix,
+                                                  dim_names=dim_names)
+    out_scipy_matrix = sparse_tensor.to_scipy()


Can we verify the shape of sparse_tensor here? It should be 2 x 2 correct?

github-actions · 2025-11-18T11:23:15Z

Thank you for your contribution. Unfortunately, this pull request has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this PR will be closed in 14 days. Feel free to re-open this if it has been closed in error. If you do not have repository permissions to reopen the PR, please tag a maintainer.

github-actions bot added Component: C++ Component: Python labels Oct 11, 2022

rok force-pushed the ARROW-17933 branch from 4ea7eeb to 21da6f4 Compare October 11, 2022 22:17

rok marked this pull request as ready for review October 11, 2022 23:19

pitrou changed the title ~~ARROW-17933: SparseCOOTensor raises error when created with zero elements~~ ARROW-17933: [C++] SparseCOOTensor raises error when created with zero elements Oct 12, 2022

pitrou requested changes Oct 12, 2022

View reviewed changes

First commit

aaa4987

rok force-pushed the ARROW-17933 branch from 21da6f4 to cadffe5 Compare October 13, 2022 02:19

Review feedback

6201e8b

rok force-pushed the ARROW-17933 branch from cadffe5 to 6201e8b Compare October 13, 2022 02:31

pitrou reviewed Oct 13, 2022

View reviewed changes

Review feedback

de2440f

rok requested a review from pitrou February 10, 2023 20:56

amol- closed this Mar 30, 2023

rok reopened this Mar 30, 2023

rok requested a review from AlenkaF as a code owner March 30, 2023 17:49

AlenkaF reviewed Apr 13, 2023

View reviewed changes

github-actions bot added the awaiting committer review Awaiting committer review label Apr 13, 2023

westonpace reviewed Apr 20, 2023

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Apr 20, 2023

github-actions bot added the Status: stale-warning Issues and PRs flagged as stale which are due to be closed if no indication otherwise label Nov 18, 2025

github-actions bot closed this Dec 9, 2025

rok reopened this Dec 9, 2025

rok requested a review from raulcd as a code owner December 9, 2025 12:07

github-actions bot removed the Status: stale-warning Issues and PRs flagged as stale which are due to be closed if no indication otherwise label Dec 10, 2025


		TYPED_TEST_SUITE_P(TestSparseTensorFromDense);

		TYPED_TEST_P(TestSparseTensorFromDense, TestNonZeroLength) {


		RETURN_NOT_OK(internal::CheckSparseIndexMaximumValue(type, shape));

		// Indexes with no values are considered valid

Conversation

rok commented Oct 11, 2022

Uh oh!

github-actions bot commented Oct 11, 2022

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Oct 13, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Jan 7, 2023

Uh oh!

amol- commented Mar 30, 2023

Uh oh!

rok commented Mar 30, 2023

Uh oh!

bkmartinjr commented Mar 30, 2023

Uh oh!

amol- commented Apr 13, 2023

Uh oh!

rok commented Apr 13, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments