Trying to catch exception from python in c++ hangs

[tvercaut]

I am trying to replicate the exception handling described here:
https://pybind11.readthedocs.io/en/stable/advanced/exceptions.html#handling-exceptions-from-python-in-c

Unfortunately, this hangs on my setup (Mac OS 13.3.1, arm64, pybind11 2.10.4). If I don't try to handle the exception, I get an expected crash:

libc++abi: terminating due to uncaught exception of type pybind11::error_already_set: ModuleNotFoundError: No module named 'fakemodule'

At:
  <string>(2): <module>

zsh: abort      ./pybind11test

Here is the full code to replicate (pybind11test.cpp):

#include <pybind11/embed.h>

#include <iostream>

namespace py = pybind11;

int main() {
  // Calling the following directly leads to the exepected
  // terminating due to uncaught exception of type
  // pybind11::error_already_set: ModuleNotFoundError:
  // No module named 'fakemodule'
  //py::scoped_interpreter guard{};
  //py::exec("import fakemodule");
  try {
    std::cout << "Starting import test" << std::endl;
    py::scoped_interpreter guard{};
    // The following hangs
    py::exec("import fakemodule");
    std::cout << "Done import test" << std::endl;
    return EXIT_SUCCESS;
  } catch (py::error_already_set& e) {
    std::cout << "Caught py::error_already_set exception in main"
              << e.what() << std::endl;
    return EXIT_FAILURE;
  } catch (const std::exception& e) {
    // standard exceptions
    std::cout << "Caught std::exception in main: " << e.what() << std::endl;
    return EXIT_FAILURE;
  } catch (...) {
    // everything else
    std::cout << "Caught unknown exception in main" << std::endl;
    return EXIT_FAILURE;

and corresponding CMakeLists.txt

cmake_minimum_required(VERSION 3.25)
project(pybind11test)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)

# Warnings compiler flags
if(MSVC)
  add_compile_options(/W4)
else()
  add_compile_options(-Wall -Wextra -Wpedantic)
endif()

include(FetchContent)
FetchContent_Declare(
  pybind11
  GIT_REPOSITORY https://github.com/pybind/pybind11
  GIT_TAG v2.10.4
  GIT_SHALLOW TRUE
  GIT_PROGRESS TRUE
  SYSTEM
)
FetchContent_MakeAvailable(pybind11)

add_executable(pybind11test pybind11test.cpp)
target_link_libraries(pybind11test PRIVATE pybind11::embed)

Am I missing something obvious to be able to catch pybind11 exceptions stemming from python?

[jagerman]

Am I missing something obvious to be able to catch pybind11 exceptions stemming from python?

The problem here is that your interpreter is scoped to the try { ... } and so isn't available anymore once you get to the catch, but the catch is trying to reach into python by the what() call and so the interpreter still needs to be available to do anything with the py::error_already_set exception.

If you move the py::scoped_interpreter guard{}; above the try then the code should work as expected.

[tvercaut]

Brilliant. Many thanks. I didn't realise the interpreter still had to be available. Maybe this could be documented here:
https://pybind11.readthedocs.io/en/stable/advanced/exceptions.html#handling-exceptions-from-python-in-c