fix(csharp): preserve schema for empty result sets in SEA mode (PECO-2949)#325
fix(csharp): preserve schema for empty result sets in SEA mode (PECO-2949)#325eric-wang-1990 merged 2 commits intomainfrom
Conversation
CI Failure AnalysisThe failing test Root causes in the test (both pre-existing)1. Bug in // Current (wrong): asserts ActivitySourceName IS empty, but message says non-empty
Assert.True(string.IsNullOrEmpty(tc.ActivitySourceName), "expecting non-empty ActivitySourceName");
activitySourceName = tc.ActivitySourceName; // Never reached — exception is caughtShould be 2. Bug in await stream.CopyToAsync(_currentFileStream);
await stream.FlushAsync(); // Flushes the source MemoryStream, not _currentFileStream!The file stream is never explicitly flushed after writes — only on Neither issue is introduced by this PR. The This comment was generated with GitHub MCP. |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When executing a query against an empty table via the Statement Execution API, the ResultManifest always contains column schema even when totalRowCount is 0. Previously, CreateReader fell through to EmptyArrowArrayStream which always returned new Schema.Builder().Build() (zero fields), discarding the manifest schema entirely. Follow the JDBC driver pattern (DatabricksResultSetMetaData is always built from ResultManifest independently of data presence) by: - Refactoring GetSchemaFromManifest into a null-returning TryGetSchemaFromManifest helper - Extracting schema from the manifest in the no-data branch and passing it to EmptyArrowArrayStream - Making EmptyArrowArrayStream accept an optional Schema parameter Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
5513ab2 to
0397659
Compare
Problem
When executing a query against an empty table via the Statement Execution API (SEA / REST mode), the
ResultManifestalways contains column schema even whentotalRowCountis 0. However,CreateReaderfell through toEmptyArrowArrayStreamwhich always returnednew Schema.Builder().Build()(zero fields), silently discarding the manifest schema.This means callers of
IArrowArrayStream.Schemareceived a schema with no columns for any empty-table query in SEA mode, making it impossible to inspect column metadata without actually having data rows.Root Cause
EmptyArrowArrayStreamwas schema-unaware — it hardcoded an empty schema regardless of context. TheCreateReaderbranching logic only extracted schema from the manifest in the CloudFetch and inline-data paths, not in the empty-result fallback path.Fix
Follow the JDBC driver pattern where
DatabricksResultSetMetaDatais always constructed fromResultManifestindependently of data presence:GetSchemaFromManifestinto a null-returningTryGetSchemaFromManifesthelperCreateReader, extract schema from manifest and pass it toEmptyArrowArrayStreamEmptyArrowArrayStreamaccept an optionalSchemaparameter (defaults to empty schema for the null-manifest / DDL case)Tests
Added
StatementExecutionEmptyResultSchemaTestswith 3 unit tests:ExecuteQuery_EmptyTable_SchemaContainsCorrectColumns— column names preserved for empty tableExecuteQuery_EmptyTable_ArrowTypesAreMappedCorrectly— INT/BIGINT/STRING/BOOLEAN/DOUBLE/DATE/TIMESTAMP all map correctlyExecuteQuery_NullManifest_ReturnsEmptySchema— null manifest (DDL) still returns empty schema without errorKnown CI Failure (Pre-existing)
TelemetryTests.CanEnableFileTracingExporterViaEnvVariable(exporterName: "adbcfile")fails with thec078a8ecversion of the hiveserver2 submodule (currentmainbaseline). This failure is unrelated to this PR — the same test passes in PR #282 which updates the submodule toe42efb47. Once PR #282 merges, this failure will be resolved.The failure is caused by a bug in
TelemetryTests.csline 69:Assert.True(string.IsNullOrEmpty(tc.ActivitySourceName))should beAssert.False. The assertion exception is swallowed by the catch block, leavingactivitySourceNameas""and the trace file unclosed/unflushed before the length check.Closes PECO-2949