Skip to content

Commit 8490f1d

Browse files
gopalldbclaude
andauthored
feat(rust): add metadata FFI layer with streaming Arrow pass-through (#281)
## Summary - Adds a `metadata-ffi` feature flag exposing `extern "C"` functions for catalog metadata operations (GetCatalogs, GetSchemas, GetTables, GetColumns, GetPrimaryKeys, GetForeignKeys) - Results stream as Arrow `RecordBatchReader` via the Arrow C Data Interface (`FFI_ArrowArrayStream`), using `ResultReaderAdapter` to bridge `ResultReader` → `RecordBatchReader` without buffering - Includes `ConnectionMetadataService` (thin pass-through to `DatabricksClient`), C FFI handle/error management with `catch_unwind` panic safety, and thread-local error buffer - `list_columns` accepts optional catalog — uses `SHOW COLUMNS IN ALL CATALOGS` server-side when catalog is None/wildcard (no client-side multi-catalog orchestration) ## Files ### New - `ffi/mod.rs`, `ffi/catalog.rs`, `ffi/error.rs`, `ffi/handle.rs` — C FFI surface (9 extern "C" functions) - `metadata/service.rs` — `ConnectionMetadataService` returning streaming `Box<dyn ResultReader>` - `spec/odbc-metadata-ffi-design.md` — Design specification ### Modified - `client/mod.rs` — `list_columns` catalog: `&str` → `Option<&str>` - `client/sea.rs` — Handle optional catalog in `list_columns` - `metadata/sql.rs` — `build_show_columns` supports ALL CATALOGS, returns `String` (not `Result`) - `connection.rs` — Updated `list_columns` call sites - `reader/mod.rs` — Added `ResultReaderAdapter` unit tests ## Data Flow ``` DatabricksClient.list_*() → ExecuteResult.reader → ConnectionMetadataService (pass-through) → export_reader() → ResultReaderAdapter → FFI_ArrowArrayStream → C caller ``` No intermediate `collect_batches` or `concat_batches` — batches stream lazily from network to caller. ## Test plan - [x] All 125 unit tests pass (`cargo test`) - [x] `cargo clippy -- -D warnings` clean - [x] `cargo fmt` clean - [ ] Integration test with ODBC wrapper consuming FFI functions 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 4fcf36b commit 8490f1d

File tree

15 files changed

+1571
-29
lines changed

15 files changed

+1571
-29
lines changed

rust/Cargo.lock

Lines changed: 163 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

rust/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,9 @@ async-trait = "0.1"
6464
# Error handling
6565
thiserror = "2"
6666

67+
# Arrow C Data Interface for metadata FFI
68+
arrow = { version = "57", optional = true, default-features = false, features = ["ffi"] }
69+
6770
[dev-dependencies]
6871
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
6972
wiremock = "0.6"
@@ -72,6 +75,7 @@ adbc_driver_manager = "0.22"
7275
[features]
7376
default = []
7477
ffi = ["adbc_ffi"]
78+
metadata-ffi = ["ffi", "dep:arrow"]
7579

7680
[lib]
7781
name = "databricks_adbc"

0 commit comments

Comments
 (0)