Cross-module enum class imports fail when classes are split across generated modules

[timosachsenberg]

Problem Description

When using cdef enum class with wrap-attach: directive, autowrap splits the generated code across multiple _pyopenms_N.pyx files for parallel compilation. However, when an enum class is defined in one module (e.g., _pyopenms_2.pyx) but used as a parameter type in a class method that ends up in a different module (e.g., _pyopenms_5.pyx), the necessary cross-module import is missing.

Example

In IonSource.pxd:

cdef extern from "<OpenMS/METADATA/IonSource.h>" namespace "OpenMS":
    cdef cppclass IonSource(MetaInfoInterface):
        void setPolarity(Polarity polarity) except + nogil

cdef extern from "<OpenMS/METADATA/IonSource.h>" namespace "OpenMS::IonSource":
    cdef enum class Polarity "OpenMS::IonSource::Polarity":
        # wrap-attach:
        #    IonSource
        POLNULL, POSITIVE, NEGATIVE, SIZE_OF_POLARITY

Generated Code Issue

• _pyopenms_2.pyx contains: class _PyPolarity(_PyEnum): ...
• _pyopenms_5.pyx contains: assert isinstance(in_0, _PyPolarity), 'arg in_0 wrong type'

But _pyopenms_5.pyx never imports _PyPolarity from _pyopenms_2, causing:

pyopenms/_pyopenms_5.pyx:5313:32: undeclared name not builtin: _PyPolarity

Affected Cases

This affects any enum class used as a function parameter when autowrap places the enum definition and the function in different output modules:

1. SpectrumType (SpectrumSettings) - _PySpectrumType defined in _pyopenms_3, used in _pyopenms_4
2. Polarity (IonSource) - _PyPolarity defined in _pyopenms_2, used in _pyopenms_5
3. MODELTYPE (MZTrafoModel) - _PyMODELTYPE defined in _pyopenms_2, used in _pyopenms_7

Related

This appears related to Cython issue #5609 regarding mixed imports/cimports of enums across modules.

Expected Behavior

Autowrap should either:

1. Generate proper cross-module imports for enum classes used as parameters
2. Keep enum class definitions in the same module as the classes they're attached to
3. Generate a shared module for all enum classes that other modules import from

Environment

• autowrap version: (latest from pip)
• Cython version: 0.29.x / 3.x
• Python: 3.10
• OpenMS: develop branch (PR converting plain enums to enum class)

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Related PRs

#131 - [TEST] example for enums in namespaces [closed]
#212 - Fix handling of wrap-ignore for foreign cimports (#196) [merged]
#221 - Add test for scoped enum forward declaration bug [merged]
#222 - Use IntEnum instead of Enum for enum class wrappers [merged]
#234 - Add test for enum-based overload resolution [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[timosachsenberg]

Root Cause Analysis

After deep investigation, I've identified the exact location of the bug and a proposed fix.

Bug Location

File: autowrap/CodeGenerator.py, method create_foreign_cimports() (lines ~2008-2013)

if resolved.cpp_decl.annotations.get("wrap-attach"):
    # No need to import attached classes as they are
    # usually in the same pxd file and should not be
    # globally exported.
    pass  # <-- BUG: Skips the import entirely

The False Assumption

The comment assumes attached enums are "usually in the same pxd file" and therefore don't need cross-module imports. However, autowrap's module splitting (via chunkIt in pyOpenMS's create_cpp_extension.py) distributes pxd files across output modules alphabetically, so:

• IonSource.pxd (defines Polarity enum) → _pyopenms_2.pyx
• IonSource class methods using Polarity → _pyopenms_5.pyx

Two Missing Import Layers

Layer	What's Needed	Current Behavior
Cython (cimport)	from .module cimport EnumType	Skipped for wrap-attach enums
Python (import)	from .module import _PyEnumName	Never generated for any enum

The Python-level _PyEnumName classes (e.g., _PyPolarity, _PySpectrumType) are used in type assertions:

assert isinstance(in_0, _PyPolarity), 'arg in_0 wrong type'

But no cross-module Python imports are generated for these.

Proposed Fix

Add a new method to generate Python imports for enum classes:

def create_foreign_enum_imports(self):
    """Generate Python imports for enum classes used in type assertions across modules"""
    code = Code()
    for module in self.all_decl:
        mname = "." + module if sys.version_info >= (3, 0) and self.add_relative else module
        if os.path.basename(self.target_path).split(".pyx")[0] != module:
            for resolved in self.all_decl[module]["decls"]:
                if resolved.__class__ in (ResolvedEnum,):
                    if not resolved.wrap_ignore:
                        # Generate Python import for the _Py prefixed enum class
                        py_name = "_Py" + resolved.name
                        code.add("from $mname import $py_name", locals())
    self.top_level_code.append(code)

Call this in create_pyx_file() after create_foreign_cimports():

self.create_foreign_cimports()
self.create_foreign_enum_imports()  # Add this line

Note: This is NOT Cython Issue #5609

The Cython issue #5609 is about import/cimport order sensitivity within a single module. Our issue is about missing imports entirely across modules - a different problem with a different fix location (autowrap, not Cython).

[jpfeuffer]

I guess it makes sense. But I have to say I am not too familiar with the multi module mechanisms.
E.g. when something is compiled and imported as a separate module and when something is just imported from a pxd in the same module. Then of course there is difference in python imports and cimports.

Confusing.