feat(csharp): metadata comparator tool#261
Open
msrathore-db wants to merge 3 commits intofeat/sea-metadata-part4from
Open
feat(csharp): metadata comparator tool#261msrathore-db wants to merge 3 commits intofeat/sea-metadata-part4from
msrathore-db wants to merge 3 commits intofeat/sea-metadata-part4from
Conversation
msrathore-db
force-pushed
the
feat/sea-metadata-part4
branch
from
dd03e93 to
e29ba26
Compare
msrathore-db
force-pushed
the
feat/sea-metadata-part5
branch
from
684f9a9 to
b2fb0c1
Compare
This was referenced Mar 2, 2026
msrathore-db
force-pushed
the
feat/sea-metadata-part5
branch
from
b2fb0c1 to
0ce7a95
Compare
msrathore-db
force-pushed
the
feat/sea-metadata-part4
branch
from
e29ba26 to
54f77e8
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
msrathore-db
force-pushed
the
feat/sea-metadata-part4
branch
from
54f77e8 to
cd97cce
Compare
msrathore-db
force-pushed
the
feat/sea-metadata-part5
branch
from
f55151e to
3a15751
Compare
msrathore-db
changed the title
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tput The committed output was from an old rewritten comparator (73 lines). Replace with the actual generated output (4150 lines) including all 13 tests: GetInfo, GetCatalogs, GetSchemas, GetTables, GetColumns, GetColumnsExtended, GetPrimaryKeys, GetCrossReference, GetTableSchema, GetObjects at all 4 depths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
msrathore-db
added a commit
that referenced
this pull request
Mar 9, 2026
Implement metadata operations for the Statement Execution API (REST) protocol, achieving parity with the existing Thrift implementation. ## Core Implementation - IGetObjectsDataProvider async implementation for SEA in StatementExecutionConnection (GetCatalogs, GetSchemas, GetTables, GetColumns via SHOW commands) - Statement-level metadata routing in StatementExecutionStatement (GetColumns, GetColumnsExtended, GetPrimaryKeys, GetCrossReference, GetTableSchema, GetTableTypes, GetInfo) - SQL command builders (ShowCatalogs/Schemas/Tables/Columns/Keys/ ForeignKeys) with pattern conversion (ADBC % → Databricks *) - ColumnMetadataHelper for computing column metadata from type name strings (SEA-only, Thrift gets these from server) - FlatColumnsResultBuilder for building flat GetColumns results - MetadataUtilities for shared catalog/pattern helpers ## Connection Parameters - EnablePKFK: gate PK/FK queries (default: true) - EnableMultipleCatalogSupport: single vs multi-catalog mode (default: true), matching Thrift DatabricksConnection behavior - DatabricksStatement-specific options silently accepted in SEA ## Design - Shared MetadataSchemaFactory (hiveserver2) for Arrow schemas - Shared GetObjectsResultBuilder for nested GetObjects structure - Async internally, blocks once at ADBC sync boundary - x-databricks-sea-can-run-fully-sync header for metadata queries - Design doc: csharp/doc/sea-metadata-design.md ## Testing - Unit tests for ShowCommands and MetadataUtilities (146 tests) - E2E tests comparing Thrift and SEA parity (17 tests) - Metadata comparator tool with schema type comparison ## Reviewed in - hiveserver2: #21, #22 (merged) - databricks: #257, #258, #259, #260, #261, #282 (reviewed) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
eric-wang-1990
approved these changes
Mar 9, 2026
msrathore-db
added a commit
that referenced
this pull request
Mar 9, 2026
Implement metadata operations for the Statement Execution API (REST) protocol, achieving parity with the existing Thrift implementation. ## Core Implementation - IGetObjectsDataProvider async implementation for SEA in StatementExecutionConnection (GetCatalogs, GetSchemas, GetTables, GetColumns via SHOW commands) - Statement-level metadata routing in StatementExecutionStatement (GetColumns, GetColumnsExtended, GetPrimaryKeys, GetCrossReference, GetTableSchema, GetTableTypes, GetInfo) - SQL command builders (ShowCatalogs/Schemas/Tables/Columns/Keys/ ForeignKeys) with pattern conversion (ADBC % → Databricks *) - ColumnMetadataHelper for computing column metadata from type name strings (SEA-only, Thrift gets these from server) - FlatColumnsResultBuilder for building flat GetColumns results - MetadataUtilities for shared catalog/pattern helpers ## Connection Parameters - EnablePKFK: gate PK/FK queries (default: true) - EnableMultipleCatalogSupport: single vs multi-catalog mode (default: true), matching Thrift DatabricksConnection behavior - DatabricksStatement-specific options silently accepted in SEA ## Design - Shared MetadataSchemaFactory (hiveserver2) for Arrow schemas - Shared GetObjectsResultBuilder for nested GetObjects structure - Async internally, blocks once at ADBC sync boundary - x-databricks-sea-can-run-fully-sync header for metadata queries - Design doc: csharp/doc/sea-metadata-design.md ## Testing - Unit tests for ShowCommands and MetadataUtilities (146 tests) - E2E tests comparing Thrift and SEA parity (17 tests) - Metadata comparator tool with schema type comparison ## Reviewed in - hiveserver2: #21, #22 (merged) - databricks: #257, #258, #259, #260, #261, #282 (reviewed) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This was referenced Mar 9, 2026
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 10, 2026
## What's Changed Implement metadata operations for the Statement Execution API (REST) protocol, achieving parity with the existing Thrift implementation. ## Core Implementation - IGetObjectsDataProvider async implementation for SEA in StatementExecutionConnection (GetCatalogs, GetSchemas, GetTables, GetColumns via SHOW commands) - Statement-level metadata routing in StatementExecutionStatement (GetColumns, GetColumnsExtended, GetPrimaryKeys, GetCrossReference, GetTableSchema, GetTableTypes, GetInfo) - SQL command builders (ShowCatalogs/Schemas/Tables/Columns/Keys/ ForeignKeys) with pattern conversion (ADBC % → Databricks *) - ColumnMetadataHelper for computing column metadata from type name strings (SEA-only, Thrift gets these from server) - FlatColumnsResultBuilder for building flat GetColumns results - MetadataUtilities for shared catalog/pattern helpers ## Connection Parameters - EnablePKFK: gate PK/FK queries (default: true) - EnableMultipleCatalogSupport: single vs multi-catalog mode (default: true), matching Thrift DatabricksConnection behavior - DatabricksStatement-specific options silently accepted in SEA ## Design - Shared MetadataSchemaFactory (hiveserver2) for Arrow schemas - Shared GetObjectsResultBuilder for nested GetObjects structure - Async internally, blocks once at ADBC sync boundary - x-databricks-sea-can-run-fully-sync header for metadata queries - Design doc: csharp/doc/sea-metadata-design.md ## Testing - Unit tests for ShowCommands and MetadataUtilities (146 tests) - E2E tests comparing Thrift and SEA parity (17 tests) - Metadata comparator tool with schema type comparison ## Reviewed in - hiveserver2: #21, #22 (merged) - databricks: #257, #258, #259, #260, #261, #282 (reviewed) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes only.
Summary
Standalone metadata comparator tool under
csharp/tools/MetadataComparator/.Compares Thrift vs SEA for all metadata operations and generates
metadata_comparison_output.txt.Usage:
DATABRICKS_CONFIG_FILE=config.json dotnet run🤖 Generated with Claude Code