[Feature]: Built-in Cython (.pyx / .pxd) structural parsing support

[ritikpathania]

Problem

Cython (.pyx implementation files and .pxd declaration files) is common in scientific Python, machine-learning, and performance-sensitive Python projects.

code-review-graph currently has no built-in Cython language mapping, so these files are not indexed as part of the normal structural graph.

Proposed solution

The goal is to integrate Cython with CRG's existing structural graph model, including where statically identifiable:

• File nodes for .pyx and .pxd
• Class nodes
• Function nodes for def, cdef, and cpdef
• IMPORTS_FROM edges for Python imports and Cython cimport
• CALLS edges
• Python/Cython module resolution
• incremental rebuild handling

This is intended as structural Tree-sitter analysis, not Cython compiler-level semantic analysis.

While investigating this, we found two viable implementation approaches and would appreciate maintainer guidance before submitting an implementation.

A. Native Cython Tree-sitter grammar

The project currently constrains:

tree-sitter-language-pack>=0.3.0,<1

and the current lockfile uses the pre-1.0 line.

Cython grammar support was added to tree-sitter-language-pack in the 1.x series. With a current 1.x release, get_parser("cython") successfully parses representative Cython constructs including:

• cdef / cpdef
• typed parameters and local declarations
• pointers and memoryviews
• cimport / from ... cimport
• .pxd declarations
• casts and address-of expressions
• nogil, noexcept, and related qualifiers

We also verified that the native grammar exposes the structural nodes needed to integrate Cython through CRG's normal Tree-sitter extraction architecture.

The main concern is dependency scope: moving CRG from tree-sitter-language-pack <1 to its 1.x series is not Cython-only.

In a local experiment against otherwise-pristine CRG, the 1.x dependency changed behavior outside Cython, including:

• exception behavior for invalid custom-language grammar loading; and
• Erlang grammar/AST behavior affecting an existing extraction test.

Because of that wider compatibility surface, we did not want to bundle a language-pack major-version migration into a Cython feature without maintainer guidance.

B. Python-grammar preprocessing fallback

If retaining the existing <1 dependency policy is important, we also have a working proof-of-concept that preprocesses Cython-specific syntax into Python-compatible syntax before parsing with CRG's existing Python grammar.

The fallback has regression coverage for:

• cdef / cpdef
• typed arguments and local declarations
• pointers and memoryviews
• cimport
• decorators and multiline signatures
• nested definitions
• comments and docstrings
• casts and address-of expressions
• source-line preservation
• idempotency
• structural extraction
• Python/Cython module resolution

This avoids a project-wide parser dependency migration, but adds a Cython-specific preprocessing layer and therefore a larger long-term maintenance surface than using the native grammar.

Maintainer guidance requested

Would you prefer that built-in Cython support pursue:

1. the native Cython grammar, together with the necessary tree-sitter-language-pack 1.x compatibility work; or
2. the self-contained preprocessing approach while retaining the current <1 dependency policy?

We're happy to prepare the implementation in whichever direction better fits the project's dependency and parser-maintenance policy.

Affected area

Parser / language support

Alternatives considered

Repository-local custom language configuration:

CRG's custom-language mechanism is useful when the installed tree-sitter-language-pack already contains the required grammar. With the project's current locked pre-1.0 version, however, get_parser("cython") is unavailable, so configuration alone cannot provide Cython parsing.

Cython/compiler frontend:

Using a compiler-level frontend would add substantially more complexity and dependencies than CRG's structural AST graph requires.

Additional context

Locally we evaluated both approaches using representative .pyx and .pxd inputs covering functions/methods, typed arguments and declarations, pointers, memoryviews, imports/cimports, decorators, multiline signatures, nested definitions, comments/docstrings, casts/address-of expressions, source coordinates, and structural graph extraction.

We also tested Python -> Cython, Cython -> Python, and Cython -> Cython module resolution and incremental resolver triggering.

No implementation has been pushed or submitted. This issue is intended to establish the preferred architectural direction before opening a pull request.