docs: add Thrift protocol test suite design

[eric-wang-1990]

Add comprehensive design document for ADBC driver Thrift protocol test suite.

Key features:

• Language-agnostic test specifications (~300 test cases across 16 categories)
• Standalone Go proxy server for failure injection testing
• Multi-language support strategy (C#, Java, C++, Go)
• Extractable design for future common repository

Initial implementation targets C# ADBC driver with plans to extend to:

• Java (JDBC driver)
• C++ (ODBC driver)
• Go (ADBC driver)

The design enables comprehensive testing of:

• Session lifecycle and management
• Statement execution (sync/async)
• Metadata operations
• Arrow format and compression
• CloudFetch results and failure scenarios
• Parameterized queries
• Error handling and recovery
• Concurrency and edge cases

Directory structure:

• docs/designs/thrift-protocol-tests/ - Design and specifications
• test-infrastructure/proxy-server/ - Standalone proxy (to be implemented)

Related to runtime Thrift tests and ADBC E2E tests, but focuses on comprehensive protocol compliance and failure scenario testing.

What's Changed

Please fill in a description of the changes here.

This contains breaking changes.

Closes #NNN.

[lidavidm]

Note that there's https://github.com/adbc-drivers/validation

[eric-wang-1990]

Use mermaid graph

[eric-wang-1990]

Not necessarily Go ADBC, there is a Go driver.

[eric-wang-1990]

Yes part should not be in the readme but should be in PR description.

[eric-wang-1990]

Remove this part, put in PR description

[eric-wang-1990]

remove this

[eric-wang-1990]

Change to mermaid graph

[eric-wang-1990]

Remove the Initial Lang column as it is not important

[eric-wang-1990]

10 weeks too long, put it into a 2 week sprint, same for other places

[eric-wang-1990]

Review Feedback Addressed

Thank you for the review! I've addressed all the comments:

@lidavidm's feedback on validation repository

✅ Added a "Relationship to Existing Validation" section explaining how this Thrift protocol test suite complements the existing adbc-drivers/validation suite. They serve different purposes:

• Validation suite: ADBC API feature validation and SQL query correctness
• This suite: Databricks Thrift protocol compliance and failure scenario testing

My own review comments

✅ Mermaid graph (line 43): Converted ASCII diagram to Mermaid format
✅ Go driver (line 85): Changed "Go ADBC" to "Go driver" for accuracy
✅ For Reviewers section (line 130): Removed - this belongs in PR description
✅ Timeline section (line 175): Removed - this belongs in PR description

All changes pushed in commit 2d0b4c2.

[eric-wang-1990]

Additional Review Feedback Addressed

Addressed additional comments on design.md:

✅ Mermaid graph in design.md (line 130): Converted ASCII diagram to Mermaid format for consistency
✅ Removed 'Initial Lang' column: Removed from test categories table as it's not important for the design

Changes pushed in commit f8ab173.

[jadewang-db]

I know this can be generated by AI easily. but wondering if we can define some format, that can avoid AI code gen for each test cases?

[jadewang-db]

is yaml the best way injection failures? or we actually create a injection api on the proxy serivce, which might be more flexible?

just like how we set mock

[eric-wang-1990]

Enhanced Proxy Server Design with Production Failure Scenarios

Added comprehensive failure scenario configuration based on real Databricks customer issues discovered through internal Glean search.

What's Added

24 production-validated failure scenarios with JIRA references across 6 categories:

Category	Count	Key Issues
🔴 CloudFetch Failures	6	PECOBLR-1131, ES-1624602, BL-13239, ES-1539484
🔴 Connection Resets	4	BL-13580, BL-14202, ES-1657027, ES-1498241
🔴 Session Management	5	ES-610899, ES-1661289, XTA-11040, ES-1608485
🟡 Protocol Issues	3	PECO-2524, BL-14014, ES-1559149
🟢 Rate Limiting	2	Generic timeout scenarios
🟢 Network Failures	2	Generic network issues

Top Priority (P0) Scenarios

1. cloudfetch_expired_link (PECOBLR-1131)

   • Most common CloudFetch issue in OSS JDBC driver

2. connection_reset_during_fetch (BL-13580)

   • Data loss in large JDBC result sets

3. invalid_session_handle (ES-610899)

   • Query failures when session invalidated

4. cloudfetch_timeout (BL-13239)

   • Timeout without clear error during CloudFetch

Why This Matters

• Real-world validated: Every scenario is based on actual customer tickets
• Traceable: JIRA IDs link test failures back to original issues
• Prioritized: P0/P1/P2 priorities guide implementation order
• Actionable: Detailed trigger configurations ready for proxy implementation

This ensures the test suite catches issues that have actually impacted production customers, not just theoretical edge cases.

Changes pushed in commit dffa378.

[eric-wang-1990]

Performance test can be added later.

[eric-wang-1990]

remove this, we do not have a go adbc driver for databricks

[lidavidm]

We do: apache/arrow-adbc#3325

[lidavidm]

(It needs to be ported over here though.)

[eric-wang-1990]

Addressed @jadewang-db Review Feedback

Thanks for the excellent design questions! I've enhanced the design to address both concerns:

1. Test Specification Format (Line 168)

Question: Can we avoid AI code gen dependency?

Solution: Added structured YAML format option alongside Markdown:

# Machine-readable test specification
test_id: TEST-SESSION-001
name: Basic OpenSession
steps:
  - action: open_session
    params:
      credentials: "{{test_credentials}}"
  - action: assert_not_null
    target: session_handle

Benefits:

• ✅ Code generation tools can parse YAML directly
• ✅ Validation tools can check test completeness
• ✅ Markdown still available for documentation
• ✅ No AI dependency for test implementation

2. Failure Injection Approach (Line 294)

Question: Is YAML best? Or should we use an injection API?

Solution: Support both approaches (similar to mocking frameworks):

YAML for static scenarios (90% of cases):

failure_scenarios:
  - name: "cloudfetch_expired_link"
    trigger: "after_requests"
    count: 1
    action: "expire_cloud_link"

HTTP API for dynamic scenarios (10% of cases):

var proxy = new ProxyControlClient("http://localhost:8081");
await proxy.ConfigureFailure(new FailureScenario {
    Name = "dynamic_failure",
    Trigger = new AfterRequestsTrigger { Count = 1 },
    Action = new ExpireCloudLinkAction()
});

Proxy API Endpoints:

• POST /api/v1/failures - Configure failure
• GET /api/v1/failures - List active failures
• DELETE /api/v1/failures/{name} - Clear specific failure

Workflow Example:

1. Start proxy: go run main.go --config cloudfetch-failures.yaml
2. Run tests: dotnet test --filter Category=CloudFetch
3. Proxy injects failures based on YAML config
4. Or use API for dynamic scenarios during test execution

Benefits:

• ✅ Simple YAML for most cases (version controlled, easy to review)
• ✅ Flexible API for complex/dynamic scenarios
• ✅ Similar to mocking frameworks (familiar pattern)
• ✅ Best of both worlds

Changes pushed in commit b7f4c81.

[eric-wang-1990]

Improved Design: Hybrid Approach (No Restart Needed!)

Great catch! Restarting the proxy for each test would indeed be impractical. I've updated the design to use a hybrid approach:

The Problem You Identified

❌ Pure YAML approach: Restart proxy for each test configuration
❌ Too slow and complicated

The Solution: Hybrid Approach

YAML defines scenarios (version controlled, reviewable):

# all-scenarios.yaml - loaded once at startup
failure_scenarios:
  - name: "cloudfetch_expired_link"
    trigger: "after_requests"
    action: "expire_cloud_link"
  - name: "session_timeout_premature"
    trigger: "after_duration"
    action: "invalidate_session"
  # ... all 24 scenarios defined

API controls activation (no restart needed):

var proxy = new ProxyControlClient("http://localhost:8081");

[Fact]
public async Task Test1()
{
    // Enable scenario from YAML
    await proxy.EnableScenario("cloudfetch_expired_link");
    // Run test...
    await proxy.DisableScenario("cloudfetch_expired_link");
}

[Fact]
public async Task Test2()
{
    // Different scenario, same proxy server!
    await proxy.EnableScenario("session_timeout_premature");
    // Run test...
    await proxy.DisableScenario("session_timeout_premature");
}

Key Advantages

✅ Start proxy ONCE: go run main.go --config all-scenarios.yaml
✅ No restart between tests
✅ Fast execution: No startup overhead
✅ Clean isolation: Each test enables/disables its scenarios
✅ YAML benefits: Version controlled, reviewed, documented
✅ API flexibility: Dynamic control without complexity

Updated API Endpoints

• POST /api/v1/scenarios/{name}/enable - Enable YAML-defined scenario
• POST /api/v1/scenarios/{name}/disable - Disable scenario
• GET /api/v1/scenarios - List all defined scenarios
• GET /api/v1/scenarios/active - List currently active scenarios

This gives us the best of both worlds: declarative YAML definitions with imperative API control.

Changes pushed in commit d7730dc.

[jadewang-db]

where we can get the detail list of test cases?

[eric-wang-1990]

I refactored to add those in the /specs folder

[eric-wang-1990]

This seem can be removed since we already have detailed tests in the specs/README.md

[eric-wang-1990]

The thrift_method should have a name as well like GetCatalogs, etc.

[eric-wang-1990]

This needs to be reworked, we should not pre-defined the types but just use equations/formulas.
E.g. for testing a GetColumns metadata call we want to verify:

1. no error throw
2. The result schema matching our golden results