Skip to content

Rel 25 12 3#104

Merged
oberstet merged 6 commits intocrossbario:masterfrom
oberstet:rel-25-12-3
Dec 16, 2025
Merged

Rel 25 12 3#104
oberstet merged 6 commits intocrossbario:masterfrom
oberstet:rel-25-12-3

Conversation

@oberstet
Copy link
Contributor

@oberstet oberstet commented Dec 16, 2025

fixes #103 [WIP]

Fixes crossbario#102. The FlatBuffers compiler generates Python code with absolute
imports like `from reflection.Type import Type` which fail when the
module is vendored inside a package. Changed all 18 occurrences to use
relative imports (`from .Type import Type`).

Files modified:
- Schema.py (4 imports)
- Service.py (2 imports)
- Object.py (2 imports)
- Enum.py (3 imports)
- EnumVal.py (1 import)
- Field.py (2 imports)
- RPCCall.py (3 imports)

Note: This work was completed with AI assistance (Claude Code).
Addresses crossbario#102 (permanent fix for vendored flatbuffers imports):

1. Added fix-flatbuffers-reflection-imports recipe
   - Automatically fixes absolute imports to relative imports
   - Now runs automatically after generate-flatbuffers-reflection
   - Uses sed to convert 'from reflection.X' to 'from .X'
   - Verifies fix was successful

2. Added test-reflection recipe and scripts/test_reflection.py
   - Tests importing Schema and all 10 reflection classes
   - Tests loading and parsing reflection.bfbs
   - This is the exact use case cfxdb, wamp-xbr, and crossbar need

The fix survives regeneration because generate-flatbuffers-reflection
now has fix-flatbuffers-reflection-imports as a dependency.

Note: This work was completed with AI assistance (Claude Code).
- Added cpy312 and cpy313 to test matrix (was: cpy311, cpy314, pypy311)
- Now tests: cpy311, cpy312, cpy313, cpy314, pypy311
- Added test-reflection step to verify FlatBuffers reflection imports
- This ensures crossbario#102 fix is tested on all Python versions

Note: This work was completed with AI assistance (Claude Code).
@oberstet
Copy link
Contributor Author

Successfully installed zlmdb-25.12.2
==> Testing flatbuffers reflection imports in cpy312...

========================================================================
  REFLECTION IMPORT TESTS - Verifying vendored reflection module
========================================================================

Python: 3.12.12 (main, Nov 19 2025, 22:46:53) [Clang 21.1.4 ]

Test 1: Verifying 'from zlmdb.flatbuffers.reflection.Schema import Schema' works...
  Schema class: <class 'zlmdb.flatbuffers.reflection.Schema.Schema'>
  ✓ PASS: Schema import works

Test 2: Verifying all reflection class imports work...
  All 10 reflection classes imported successfully:
    - Schema: <class 'zlmdb.flatbuffers.reflection.Schema.Schema'>
    - Object: <class 'zlmdb.flatbuffers.reflection.Object.Object'>
    - Field: <class 'zlmdb.flatbuffers.reflection.Field.Field'>
    - Enum: <class 'zlmdb.flatbuffers.reflection.Enum.Enum'>
    - EnumVal: <class 'zlmdb.flatbuffers.reflection.EnumVal.EnumVal'>
    - Type: <class 'zlmdb.flatbuffers.reflection.Type.Type'>
    - Service: <class 'zlmdb.flatbuffers.reflection.Service.Service'>
    - RPCCall: <class 'zlmdb.flatbuffers.reflection.RPCCall.RPCCall'>
    - KeyValue: <class 'zlmdb.flatbuffers.reflection.KeyValue.KeyValue'>
    - SchemaFile: <class 'zlmdb.flatbuffers.reflection.SchemaFile.SchemaFile'>
  ✓ PASS: All reflection classes import

Test 3: Verifying Schema.GetRootAs() can parse reflection.bfbs...
  Loaded reflection.bfbs (8016 bytes)
  Schema has 10 objects
  Schema has 2 enums
  ✓ PASS: Schema.GetRootAs() works

========================================================================
ALL REFLECTION TESTS PASSED (3/3)
========================================================================

========================================================================
✅ ALL REFLECTION IMPORT TESTS PASSED
========================================================================
(cpy312) oberstet@amd-ryzen5:~/work/wamp/zlmdb$ just test-reflection

Submodule updates:
- .ai: bfb4804 -> d1647dc (polish project README)
- .cicd: 024df86 -> 0954ce5 (add helper git version, fix PR template)

GitHub template fixes:
- Move PR template to root level (.github/pull_request_template.md)
  for GitHub auto-population (was in PULL_REQUEST_TEMPLATE/ subdirectory)
- Remove obsolete .github/ISSUE_TEMPLATE.md (using ISSUE_TEMPLATE/ dir)

Note: This work was completed with AI assistance (Claude Code).
@oberstet
Copy link
Contributor Author

works on github workflows now too, eg https://github.com/crossbario/zlmdb/actions/runs/20283952345/job/58253029147:

...
 Successfully installed zlmdb-25.12.2
==> Testing flatbuffers reflection imports in cpy314...

========================================================================
  REFLECTION IMPORT TESTS - Verifying vendored reflection module
========================================================================

Python: 3.14.2 (main, Dec  9 2025, 19:03:28) [Clang 21.1.4 ]

Test 1: Verifying 'from zlmdb.flatbuffers.reflection.Schema import Schema' works...
  Schema class: <class 'zlmdb.flatbuffers.reflection.Schema.Schema'>
  ✓ PASS: Schema import works

Test 2: Verifying all reflection class imports work...
  All 10 reflection classes imported successfully:
    - Schema: <class 'zlmdb.flatbuffers.reflection.Schema.Schema'>
    - Object: <class 'zlmdb.flatbuffers.reflection.Object.Object'>
    - Field: <class 'zlmdb.flatbuffers.reflection.Field.Field'>
    - Enum: <class 'zlmdb.flatbuffers.reflection.Enum.Enum'>
    - EnumVal: <class 'zlmdb.flatbuffers.reflection.EnumVal.EnumVal'>
    - Type: <class 'zlmdb.flatbuffers.reflection.Type.Type'>
    - Service: <class 'zlmdb.flatbuffers.reflection.Service.Service'>
    - RPCCall: <class 'zlmdb.flatbuffers.reflection.RPCCall.RPCCall'>
    - KeyValue: <class 'zlmdb.flatbuffers.reflection.KeyValue.KeyValue'>
    - SchemaFile: <class 'zlmdb.flatbuffers.reflection.SchemaFile.SchemaFile'>
  ✓ PASS: All reflection classes import

Test 3: Verifying Schema.GetRootAs() can parse reflection.bfbs...
  Loaded reflection.bfbs (8016 bytes)
  Schema has 10 objects
  Schema has 2 enums
  ✓ PASS: Schema.GetRootAs() works

========================================================================
ALL REFLECTION TESTS PASSED (3/3)
========================================================================

========================================================================
✅ ALL REFLECTION IMPORT TESTS PASSED
========================================================================

Adds comprehensive documentation for FlatBuffers integration:

- docs/flatbuffers/index.rst: Overview of FlatBuffers integration,
  explaining why FlatBuffers was chosen (zero-copy access, schema
  evolution, cross-language support, memory efficiency, reflection)

- docs/flatbuffers/wamp-zerocopy-dataplane.rst (issue crossbario#98):
  Architecture and design of the zero-copy data plane for WAMP,
  covering data flow from LMDB through FlatBuffers to WAMP transport,
  design principles, and use cases

- docs/flatbuffers/vendoring-design-and-technology.rst (issue crossbario#99):
  Technical details on native binaries, manylinux compliance, ISA
  compatibility, bundled flatc, and PyPy support

Updates docs/index.rst to include the new flatbuffers section.

Closes: crossbario#98, crossbario#99

Note: This work was completed with AI assistance (Claude Code).
@oberstet oberstet merged commit 9082c4b into crossbario:master Dec 16, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release v25.12.3

1 participant