You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
gh-40594: Fix segmentation fault in libgap function call
Fixes#37026.
As explained in https://trofi.github.io/posts/312-the-sagemath-
saga.html, the relevant code are the following.
```c
static PyObject *__pyx_pf_4sage_4libs_3gap_7element_19GapElement_Functio
n_2__call__(struct
__pyx_obj_4sage_4libs_3gap_7element_GapElement_Function *__pyx_v_self,
PyObject *__pyx_v_args) {
PyObject *__pyx_t_6 = NULL;
sig_GAP_Enter();
__pyx_t_6 = __Pyx_GetItemInt_List(__pyx_v_a, 2, long, 1,
__Pyx_PyInt_From_long, 1, 0, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0,
2528, __pyx_L14_error)
__pyx_v_result = GAP_CallFunc3Args(__pyx_v_self->__pyx_base.value,
((struct __pyx_obj_4sage_4libs_3gap_7element_GapElement
*)__pyx_t_5)->value, ((struct
__pyx_obj_4sage_4libs_3gap_7element_GapElement *)__pyx_t_4)->value,
((struct __pyx_obj_4sage_4libs_3gap_7element_GapElement
*)__pyx_t_6)->value);
__Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
GAP_Leave();
__Pyx_XDECREF(__pyx_t_6);
}
```
where
```c
#define __Pyx_XDECREF(r) do { if((r) == NULL); else {__Pyx_DECREF(r);
}} while(0)
```
so that simplifies to
```c
t_6 = 0;
if (!setjmp()) {
t_6 = 1;
GAP_CallFunc3Args(); // longjmp inside here
t_6 = 0;
}
if (t_6 != 0) access(t_6);
```
The "easy" fix is to declare `t_6` volatile, but we cannot do that since
it's a Cython-generated temporary variable.
A better way is to avoid mutating `t_6` between `setjmp()` and
`longjmp()`.
We look at the original code to see why `t_6` is being mutated in the
first place. It's because we're accessing
`a[i]`, where `a` is a Python list, so Cython needs to store the result
into a Python-reference-counted temporary variable.
By changing `a` from a Python list to a C array, the new code becomes
```c
sig_GAP_Enter();
__pyx_t_12 = sig_on(); if (unlikely(__pyx_t_12 == ((int)0)))
__PYX_ERR(0, 2514, __pyx_L9_error)
switch (__pyx_v_n) {
case 0:
__pyx_v_result =
GAP_CallFunc0Args(__pyx_v_self->__pyx_base.value);
break;
case 1:
__pyx_v_result = GAP_CallFunc1Args(__pyx_v_self->__pyx_base.value,
(__pyx_v_a[0]));
break;
case 2:
__pyx_v_result = GAP_CallFunc2Args(__pyx_v_self->__pyx_base.value,
(__pyx_v_a[0]), (__pyx_v_a[1]));
break;
case 3:
__pyx_v_result = GAP_CallFunc3Args(__pyx_v_self->__pyx_base.value,
(__pyx_v_a[0]), (__pyx_v_a[1]), (__pyx_v_a[2]));
break;
default:
__pyx_v_result = GAP_CallFuncList(__pyx_v_self->__pyx_base.value,
__pyx_v_arg_list);
break;
}
sig_off();
}
GAP_Leave();
goto __pyx_L10;
}
```
Moral of the story: **do not touch any Python object within
`sig_on()`...`sig_off()`!**
The [cysignals documentation](https://cysignals.readthedocs.io/en/latest
/interrupt.html) also warned about this:
> The code inside `sig_on()` should be pure C or Cython code. If you
call any Python code or manipulate any Python object (even something
trivial like `x = []`), an interrupt can mess up Python’s internal
state. When in doubt, try to use sig_check() instead.
------
This does not violate the GAP API either.
```
Code which uses the GAP API exposed by this header file should sandwich
any such calls between uses of the GAP_Enter() and GAP_Leave() macro as
follows:
int ok = GAP_Enter();
if (ok) {
... // any number of calls to GAP APIs
}
GAP_Leave();
This is in particular crucial if your code keeps references to any GAP
functions in local variables: Calling GAP_Enter() ensures that GAP is
aware of such references, and will not garbage collect the referenced
objects. Failing to use these macros properly can lead to crashes, or
worse, silent memory corruption. You have been warned!
```
### 📝 Checklist
<!-- Put an `x` in all the boxes that apply. -->
- [ ] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.
### ⌛ Dependencies
<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - #12345: short description why this is a dependency -->
<!-- - #34567: ... -->
URL: #40594
Reported by: user202729
Reviewer(s):
0 commit comments