feat: move Pythonizations implementations to source files

[wdconinc]

This PR moves the implementation of Pythonizations detail functions into a source file. This allows the headers to be included in C++20 modules (#907).

BEGINRELEASENOTES

• Move Pythonizations detail implementations to separate source file

ENDRELEASENOTES

[Copilot]

Pull request overview

This PR refactors the Pythonizations implementation by moving function definitions from inline functions in the header file to a separate source file. This change enables the headers to be included in C++20 modules while maintaining the same functionality.

• Function implementations moved from static inline in header to regular definitions in source file
• Added target_link_libraries(podio PRIVATE Python3::Python) to CMakeLists.txt
• Header file now contains only forward declarations

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
src/Pythonizations.cc	New source file containing implementations of subscript and pythonize_subscript functions
include/podio/detail/Pythonizations.h	Refactored to contain only function declarations, removed inline implementations and <stdexcept> include
src/CMakeLists.txt	Added Pythonizations.cc to core_sources and linked Python3 library

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wdconinc]

Ah, this probably should have been in the Python:: namespace since that's what is added as a dependency in the config

podio/cmake/podioConfig.cmake.in

Lines 28 to 32 in 5d82748

	if(NOT "@REQUIRE_PYTHON_VERSION@" STREQUAL "")
	find_dependency(Python @REQUIRE_PYTHON_VERSION@ COMPONENTS Interpreter Development)
	else()
	find_dependency(Python COMPONENTS Interpreter Development)
	endif()

and in the PODIO_GENERATE_DATAMODEL module:

podio/cmake/podioMacros.cmake

Line 158 in 5d82748

COMMAND ${Python_EXECUTABLE} ${podio_PYTHON_DIR}/podio_class_generator.py ${CLANG_FORMAT_ARG} ${SCHEMA_EVOLUTION_ARG} ${UPSTREAM_EDM_ARG} ${YAML_FILE} ${ARG_OUTPUT_FOLDER} ${datamodel} ${ARG_IO_BACKEND_HANDLERS} ${LANGUAGE_ARG} ${VERSION_ARG} ${OLD_DESCRIPTION_ARG}

[wdconinc]

Then again, the package that is found is Python3...

podio/cmake/podioBuild.cmake

Lines 197 to 200 in 5d82748

	if(NOT PODIO_RELAX_PYVER)
	find_package(Python3 ${REQUIRE_PYTHON_VERSION} EXACT REQUIRED COMPONENTS Development Interpreter)
	else()
	find_package(Python3 ${REQUIRE_PYTHON_VERSION} REQUIRED COMPONENTS Development Interpreter)

[paulgessinger]

I've investigated this a bit after the Mac build started failing in spack for me. The way I see it, this now compiles in a strict dependency on the exact Python runtime lib into the podio library, which means that you need to actually link against python in a way you usually do for embedding the interpreter.

This then causes issues in Spack, I believe because the python build from the package there does not correctly propagate it's own build configuration on macOS, and relies on intl being installed in linux "by accident".

[tmadlener]

Looks like moving these into a dedicated TU and building them into libpodio causes quite a few issues. I am wondering whether we should move the implementation back into Pythonizations.h but marking the functions inline only. That should work for modules, IIUC.

edit: Just read in #907 that Python.h is the issue not the static inline.

---

Some additional information of what we have learned in the meantime about the behavior with and without these changes.

• These changes make builds on macOS fail: see Link against Python3::Module #925
• Prior to these changes it looks like any and all references to these functions were inlined and subsequently removed by the linker as there is no trace of them in the compiled binaries (e.g. readelf -sWC libedm4hep.so | grep cppyy doesn't give any hits). cppyy is still able to find them (potentially because it JIT compiles the collection headers)
  • We were also missing a target_link_libraries(podio PUBLIC Python3::Python). However, given the above that didn't raise an issue.
• Moving the inline definition of __cppyy_pythonize_ from the collection headers to the collection implementation files in code generations makes all symbols appear again.
  • These changes would require to at least have a target_link_libraries(<edmlib> PRIVATE Python3::Python) to be present in the cmake macro that builds the core datamodel lib.

[wdconinc]

Looks like moving these into a dedicated TU and building them into libpodio causes quite a few issues. I am wondering whether we should move the implementation back into Pythonizations.h but marking the functions inline only. That should work for modules, IIUC.

Certainly the use case of modules is tangential here. If this needs to be reverted, we can figure out how to make it work with modules given the additional constraint of header-only Pythonizations.h. That said, I think just inline (not static inline) should be sufficient. The issue was not in the Python.h header in this case (at least not for the python I was testing with).