Managing GIL and objects in between different threads

[prashant-saxena]

Hello,

First of all, nanobind is awesome.... :-) and I am just loving it.

I have created a small framework based on taskflow and wrapping it using nanobind.
When framework finishes the graph evaluation, it can send a callback that you can register. I am trying to register a python function but the application is getting crashed.

.def("set_graph_evaluation_finish_callback", &PyScene::set_graph_evaluation_finish_callback)

void PyScene::set_graph_evaluation_finish_callback(
    nb::callable cb) {
    nb::gil_scoped_acquire gil;
    callback = std::move(cb);
    scene.setGraphEvaluationFinishCallback([this]() {
        nb::gil_scoped_acquire gil;
        if (this->callback)
            this->callback();
        std::cout << "C++ lambda reached (no GIL)\n";
    });
}

When lambda function is getting called from framework, the line inside lambda

nb::gil_scoped_acquire gil;

is responsible for the crash.

Critical nanobind error: nanobind::detail::incref_check(): attempted to change the reference count of a Python object while the GIL was not held.

I believe because there are two threads, the python one and the one that's calling the lambda or I could be wrong. How do I solve this problem?

[wjakob]

This makes no sense to me based on the code you provided. You are only acquiring the GIL, where is it being released?

Likely the issue is that you are calling set_graph_evaluation_finish_callback directly from C++ while not holding the GIL. Passing a nb::callable by value will involve reference counting, which causes the issue.