Skip to content

Commit 75cee0a

Browse files
authored
Merge pull request #267 from esmf-org/esmpy_support_large_arrays
ESMPy: Fix type cast to support large Arrays In a few places, the size of an Array was being cast to a c_int. This caused an overflow if the size exceeded 32 bits. An example of the issue is reported here, and I have reproduced this: https://github.com/orgs/esmf-org/discussions/263 From some reading, I think the correct cast here should be to ssize_t, which resolves to a long int (64 bits) on a 64-bit architecture. I am not 100% positive of this, but this seems consistent with what I have found; e.g., see: Argument list in the PyMemoryView_FromMemory function: https://docs.python.org/3/c-api/memoryview.html https://docs.python.org/3/c-api/intro.html#c.Py_ssize_t https://stackoverflow.com/questions/20987390/cython-why-when-is-it-preferable-to-use-py-ssize-t-for-indexing
2 parents e8332f8 + fe50b8f commit 75cee0a

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

src/addon/esmpy/src/esmpy/util/esmpyarray.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ def ndarray_from_esmf(data, dtype, shape):
2929
if sys.version_info[0] >= 3:
3030
buffer = ct.pythonapi.PyMemoryView_FromMemory
3131
buffer.restype = ct.py_object
32-
buffer = buffer(data, ct.c_int(size), 0x200)
32+
buffer = buffer(data, ct.c_ssize_t(size), 0x200)
3333
else:
3434
buffer = np.core.multiarray.int_asbuffer(
3535
ct.addressof(data.contents), size)
@@ -65,7 +65,7 @@ def __new__(cls, data, mask, dtype, shape):
6565
if sys.version_info[0] >= 3:
6666
buffer = ct.pythonapi.PyMemoryView_FromMemory
6767
buffer.restype = ct.py_object
68-
buffer = buffer(data, ct.c_int(size), 0x200)
68+
buffer = buffer(data, ct.c_ssize_t(size), 0x200)
6969
else:
7070
buffer = np.core.multiarray.int_asbuffer(
7171
ct.addressof(data.contents), size)
@@ -108,7 +108,7 @@ def __new__(cls, data, dtype, shape):
108108
if sys.version_info[0] >= 3:
109109
buffer = ct.pythonapi.PyMemoryView_FromMemory
110110
buffer.restype = ct.py_object
111-
buffer = buffer(data, ct.c_int(size), 0x200)
111+
buffer = buffer(data, ct.c_ssize_t(size), 0x200)
112112
else:
113113
buffer = np.core.multiarray.int_asbuffer(
114114
ct.addressof(data.contents), size)

0 commit comments

Comments
 (0)