Segmentation fault when editing a list in-place in a C extesnion

[dd-dreams]

Bug report

Bug description:

Short untested example:

lst = ... # a numpy darray
res = func(lst.tolist()) # function in the C extension. edits in-place lst.tolist() -> returns a List. res depends on `lst`
print(res[0]) # Segmentation fault when accessing res in some way

func edits lst.tolist() in-place, and res depends on lst.tolist(), possibly res is lst.tolist().
We get a segfault when printing. It results of a GC collection error, probably because lst.tolist() is invalidly being freed. Assigning lst.tolist() to a variable fixes the segfault.

It might be Python intent to act like this.

CPython versions tested on:

3.9

Operating systems tested on:

Linux

[ZeroIntensity]

Please provide the implementation of func. I think the problem is probably that you're releasing a reference too many times.

[dd-dreams]

Reproduce:

segfault.c

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject* seg(PyObject *self, PyObject *args) {
    PyObject *x = NULL;

    if(!PyArg_ParseTuple(args, "O", x))
        return NULL;
    PyList_SetItem(x, 0, PyList_New(2));

    return x;
}

// Other PyMethodDef etc...

segfaultpy.py

import numpy as np
import segfault

np.random.seed(1234)

if __name__ == "__main__":
    lst = np.random.rand(2, 2)
    result = segfault.seg(lst.tolist())
    print(result)

[sergey-miryanov]

It is because PyList_New(2) contains NULL elements and this is not allowed.

[ZeroIntensity]

Yup, if you want an empty list, just call PyList_New(0), otherwise you have to manually set the items.

[dd-dreams]

Though this still happens if I manually set the items, or pass an empty list.

[ZeroIntensity]

Please send the code again.

[dd-dreams]

It's the same exact code only this time we do PyList_New(0) instead of 2.

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject* seg(PyObject *self, PyObject *args) {
    PyObject *x = NULL;

    if(!PyArg_ParseTuple(args, "O", x))
        return NULL;
    PyList_SetItem(x, 0, PyList_New(0));

    return x;
}

// Other PyMethodDef etc...

[dd-dreams]

It also doesn't work when creating a List and filling it with ints.

[ZeroIntensity]

Oh, the issue is that you're passing x to PyArg_ParseTuple, not &x. Consider using METH_O for single arg functions.

[dd-dreams]

With &x it still isn't working. Just a reminder, assigning a = lst.tolist() and then passing to the function fixes the crash.

[sergey-miryanov]

You should incref x before return. I think it is a better to ask such questions at https://discuss.python.org/c/help/7