docs: add OAuth implementation sprint plan

3-task breakdown covering foundation + HTTP client changes,
M2M provider, and U2M provider with full test coverage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: vikrantpuppala

rust/docs/designs/oauth-sprint-plan.md

@@ -0,0 +1,118 @@
+# Sprint Plan: OAuth U2M + M2M Implementation
+
+**Sprint dates:** 2026-03-07 to 2026-03-20
+**Sprint goal:** Implement complete OAuth 2.0 authentication (U2M + M2M) for the Rust ADBC driver as described in the [OAuth design doc](oauth-u2m-m2m-design.md).
+
+---
+
+## Story
+
+**Title:** Implement OAuth U2M and M2M authentication
+
+**Description:**
+Add OAuth 2.0 support to the Rust ADBC driver, covering:
+- M2M (Client Credentials) flow for service principal authentication
+- U2M (Authorization Code + PKCE) flow for interactive browser-based login
+- Shared infrastructure: OIDC discovery, token lifecycle management, file-based token caching
+- Integration with `Database` config and `DatabricksHttpClient` (two-phase init via `OnceLock`)
+- ODBC-aligned numeric config values (`AuthMech`/`Auth_Flow` scheme)
+
+**Acceptance Criteria:**
+- [ ] M2M flow works end-to-end: `Database` config -> `new_connection()` -> `get_auth_header()` returns valid Bearer token from client credentials exchange
+- [ ] U2M flow works end-to-end: browser launch -> callback server captures code -> PKCE token exchange -> cached token reused on subsequent connections
+- [ ] Token refresh state machine works: FRESH (no-op) -> STALE (background refresh) -> EXPIRED (blocking refresh)
+- [ ] File-based token cache persists U2M tokens across connections with correct permissions (0o600)
+- [ ] `DatabricksHttpClient` supports two-phase init: created without auth, auth set later via `OnceLock`
+- [ ] All config options from the design doc are parseable via `set_option()`
+- [ ] Invalid config combinations fail with clear error messages at `new_connection()`
+- [ ] `cargo fmt`, `cargo clippy -- -D warnings`, `cargo test` all pass
+
+---
+
+## Sub-Tasks
+
+### Task 1: Foundation + HTTP Client Changes
+
+**Scope:** Shared OAuth infrastructure and modifications to existing code to support two-phase auth initialization.
+
+**Files to create:**
+- `src/auth/oauth/mod.rs` — module root, re-exports
+- `src/auth/oauth/token.rs` — `OAuthToken` struct with `is_expired()`, `is_stale()`, JSON serialization
+- `src/auth/oauth/oidc.rs` — `OidcEndpoints` struct, `discover()` function hitting `{host}/oidc/.well-known/oauth-authorization-server`
+- `src/auth/oauth/cache.rs` — `TokenCache` with `load()`/`save()`, SHA-256 hashed filenames, 0o600 permissions
+- `src/auth/oauth/token_store.rs` — `TokenStore` with `RwLock<Option<OAuthToken>>`, `AtomicBool` for refresh coordination, FRESH/STALE/EXPIRED state machine
+
+**Files to modify:**
+- `Cargo.toml` — add `oauth2 = "5"`, `sha2 = "0.10"`, `open = "5"`, `dirs = "5"`
+- `src/client/http.rs` — change `auth_provider` from `Arc<dyn AuthProvider>` to `OnceLock<Arc<dyn AuthProvider>>`, add `set_auth_provider()`, update `execute()` to read from `OnceLock`
+- `src/database.rs` — add `AuthMechanism`/`AuthFlow` enums (with `TryFrom<i64>`), new config fields (`auth_mechanism`, `auth_flow`, `auth_client_id`, `auth_client_secret`, `auth_scopes`, `auth_token_endpoint`, `auth_redirect_port`), `set_option()`/`get_option_string()` for all new keys
+- `src/auth/mod.rs` — add `pub mod oauth;`, remove old `pub use oauth::OAuthCredentials`, delete `src/auth/oauth.rs` (replaced by directory module)
+
+**Tests:**
+- `token.rs`: `test_token_fresh_not_expired`, `test_token_stale_threshold`, `test_token_expired_within_buffer`, `test_token_serialization_roundtrip`
+- `oidc.rs`: `test_discover_workspace_endpoints`, `test_discover_invalid_response`, `test_discover_http_error`
+- `cache.rs`: `test_cache_key_deterministic`, `test_cache_save_load_roundtrip`, `test_cache_missing_file`, `test_cache_file_permissions`, `test_cache_corrupted_file`
+- `token_store.rs`: `test_store_fresh_token_no_refresh`, `test_store_expired_triggers_blocking_refresh`, `test_store_concurrent_refresh_single_fetch`, `test_store_stale_returns_current_token`
+- `http.rs`: `test_execute_without_auth_works_before_auth_set`, `test_execute_fails_before_auth_set`, `test_execute_succeeds_after_auth_set`, `test_set_auth_provider_twice_panics_or_errors`
+- `database.rs`: `test_set_auth_mechanism_valid`, `test_set_auth_mechanism_invalid`, `test_set_auth_flow_valid`, `test_set_auth_flow_invalid`, `test_new_connection_missing_mechanism`, `test_new_connection_oauth_missing_flow`, `test_new_connection_pat_missing_token`
+
+**Definition of done:** All foundation modules compile, unit tests pass, `DatabricksHttpClient` two-phase init works, database config parsing works for all auth options.
+
+---
+
+### Task 2: M2M Provider (Client Credentials)
+
+**Scope:** Implement the M2M OAuth flow using `oauth2::BasicClient` for client credentials exchange.
+
+**Files to create:**
+- `src/auth/oauth/m2m.rs` — `ClientCredentialsProvider` implementing `AuthProvider`
+
+**Files to modify:**
+- `src/auth/oauth/mod.rs` — add `pub mod m2m;`, re-export `ClientCredentialsProvider`
+- `src/auth/mod.rs` — re-export `ClientCredentialsProvider`
+- `src/database.rs` — wire `AuthFlow::ClientCredentials` match arm in `new_connection()` to create `ClientCredentialsProvider`
+
+**Implementation details:**
+- Construct `oauth2::BasicClient` from OIDC-discovered endpoints
+- Implement `oauth2` HTTP adapter that routes through `DatabricksHttpClient::execute_without_auth()`
+- Use `TokenStore` for token lifecycle (no disk cache for M2M)
+- `get_auth_header()` → `TokenStore::get_or_refresh()` → `client.exchange_client_credentials()` if needed
+
+**Tests:**
+- `m2m.rs`: `test_m2m_token_exchange`, `test_m2m_auto_refresh`, `test_m2m_oidc_discovery`
+- Wiremock integration (`tests/`): `test_m2m_full_flow_discovery_and_token_exchange`, `test_m2m_token_refresh_on_expiry`, `test_m2m_discovery_failure_propagates`
+- Database validation: `test_new_connection_client_credentials_missing_secret`
+
+**Definition of done:** M2M flow works end-to-end from `Database` config through `new_connection()` to `get_auth_header()`. Wiremock tests verify the full OIDC discovery -> token exchange -> refresh cycle.
+
+---
+
+### Task 3: U2M Provider (Authorization Code + PKCE)
+
+**Scope:** Implement the U2M OAuth flow with browser-based login, PKCE, callback server, and token caching.
+
+**Files to create:**
+- `src/auth/oauth/callback.rs` — `CallbackServer` with localhost HTTP listener, state validation, timeout
+- `src/auth/oauth/u2m.rs` — `AuthorizationCodeProvider` implementing `AuthProvider`
+
+**Files to modify:**
+- `src/auth/oauth/mod.rs` — add `pub mod callback;`, `pub mod u2m;`, re-export `AuthorizationCodeProvider`
+- `src/auth/mod.rs` — re-export `AuthorizationCodeProvider`
+- `src/database.rs` — wire `AuthFlow::Browser` match arm in `new_connection()` to create `AuthorizationCodeProvider`
+
+**Implementation details:**
+- On creation: try loading cached token via `TokenCache::load()`
+- If cached token has valid `refresh_token`: store in `TokenStore`, refresh on first `get_auth_header()` if stale/expired
+- If no cache: generate PKCE via `PkceCodeChallenge::new_random_sha256()`, build auth URL via `client.authorize_url()`, start `CallbackServer`, launch browser via `open::that()`, wait for callback, exchange code via `client.exchange_code().set_pkce_verifier()`
+- Save token to cache after every successful acquisition/refresh
+- Refresh uses `client.exchange_refresh_token()` through `DatabricksHttpClient::execute_without_auth()`
+- If refresh token is expired, fall back to full browser flow
+
+**Tests:**
+- `callback.rs`: `test_callback_captures_code`, `test_callback_validates_state`, `test_callback_timeout`
+- `u2m.rs`: `test_u2m_refresh_token_flow`, `test_u2m_cache_hit`, `test_u2m_cache_miss_with_expired_refresh`
+- Wiremock integration: `test_u2m_refresh_token_full_flow`
+- Database validation: `test_new_connection_token_passthrough_missing_token`
+- E2E (ignored): `test_m2m_end_to_end`, `test_u2m_end_to_end`
+
+**Definition of done:** U2M flow works end-to-end. Token cache persists across connections. Refresh path works via wiremock tests. Browser flow verified manually via `#[ignore]` E2E test.

rust/docs/designs/oauth-u2m-m2m-design.md

@@ -180,7 +180,7 @@ pub(crate) struct TokenStore {
 ```
 
 **Contract:**
-- `get_or_refresh(refresh_fn)`: Returns a valid token. If STALE, spawns background refresh via `std::thread::spawn` and returns current token. If EXPIRED, blocks caller until refresh completes.
+- `get_or_refresh(refresh_fn)`: Returns a valid token. If STALE, spawns background refresh via `tokio::spawn` and returns current token. If EXPIRED, blocks caller until refresh completes.
 - Thread-safe: `RwLock` for read-heavy access, `AtomicBool` to prevent concurrent refresh.
 - Only one refresh runs at a time; concurrent callers receive the current (stale) token.
 
@@ -291,29 +291,29 @@ sequenceDiagram
     participant DB as Database
     participant M2M as ClientCredsProvider
     participant OIDC as OIDC Discovery
-    participant TE as Token Endpoint
+    participant TokenEP as Token Endpoint
 
     App->>DB: new_connection()
     DB->>M2M: new(host, client_id, client_secret, scopes)
     M2M->>OIDC: GET {host}/oidc/.well-known/oauth-authorization-server
     OIDC-->>M2M: OidcEndpoints
 
-    Note over App,TE: First get_auth_header() call
+    Note over App,TokenEP: First get_auth_header() call
     App->>M2M: get_auth_header()
-    M2M->>TE: client.exchange_client_credentials()
-    TE-->>M2M: access_token (no refresh_token)
+    M2M->>TokenEP: client.exchange_client_credentials()
+    TokenEP-->>M2M: access_token (no refresh_token)
     M2M-->>App: "Bearer {access_token}"
 
-    Note over App,TE: Subsequent calls (token FRESH)
+    Note over App,TokenEP: Subsequent calls (token FRESH)
     App->>M2M: get_auth_header()
     M2M-->>App: "Bearer {cached_token}"
 
-    Note over App,TE: Token becomes STALE
+    Note over App,TokenEP: Token becomes STALE
     App->>M2M: get_auth_header()
     M2M->>M2M: Spawn background refresh
     M2M-->>App: "Bearer {current_token}"
-    M2M->>TE: client.exchange_client_credentials() (background)
-    TE-->>M2M: new access_token
+    M2M->>TokenEP: client.exchange_client_credentials() (background)
+    TokenEP-->>M2M: new access_token
 ```
 
 ### Token Exchange (M2M)
@@ -362,65 +362,47 @@ impl TokenCache {
 
 ## Configuration Options
 
-Following the Databricks ODBC driver's two-level authentication scheme, configuration uses `AuthMech` (mechanism) and `Auth_Flow` (OAuth flow type) as the primary selectors. Both are **required** -- no auto-detection.
+Authentication is configured via a single `databricks.auth.type` string option that selects the authentication method. This replaces the ODBC-style two-level `AuthMech`/`Auth_Flow` numeric scheme with self-describing string values.
 
-### Rust Enums
+### Rust Enum
 
 ```rust
-/// Authentication mechanism -- top-level selector.
-/// Config values match the ODBC driver's AuthMech numeric codes.
+/// Authentication type -- single selector for the authentication method.
 #[derive(Debug, Clone, PartialEq)]
-#[repr(u8)]
-pub enum AuthMechanism {
-    /// Personal access token (no OAuth). Config value: 0
-    Pat = 0,
-    /// OAuth 2.0 -- requires AuthFlow to select the specific flow. Config value: 11
-    OAuth = 11,
-}
-
-/// OAuth authentication flow -- selects the specific OAuth grant type.
-/// Config values match the ODBC driver's Auth_Flow numeric codes.
-/// Only applicable when AuthMechanism is OAuth.
-#[derive(Debug, Clone, PartialEq)]
-#[repr(u8)]
-pub enum AuthFlow {
-    /// Use a pre-obtained OAuth access token directly. Config value: 0
-    TokenPassthrough = 0,
-    /// M2M: client credentials grant for service principals. Config value: 1
-    ClientCredentials = 1,
-    /// U2M: browser-based authorization code + PKCE. Config value: 2
-    Browser = 2,
+pub enum AuthType {
+    /// Personal access token.
+    AccessToken,
+    /// M2M: client credentials grant for service principals.
+    OAuthM2m,
+    /// U2M: browser-based authorization code + PKCE.
+    OAuthU2m,
 }
 ```
 
 ### Authentication Selection
 
 | Option | Type | Values | Required | Description |
 |--------|------|--------|----------|-------------|
-| `databricks.auth.mechanism` | Int | `0` (PAT), `11` (OAuth) | **Yes** | Authentication mechanism (matches ODBC `AuthMech`) |
-| `databricks.auth.flow` | Int | `0` (token passthrough), `1` (client credentials), `2` (browser) | **Yes** (when mechanism=`11`) | OAuth flow type (matches ODBC `Auth_Flow`) |
+| `databricks.auth.type` | String | `access_token`, `oauth_m2m`, `oauth_u2m` | **Yes** | Authentication method |
 
-**Values aligned with ODBC driver:**
-
-| `mechanism` | `flow` | ODBC `AuthMech` | ODBC `Auth_Flow` | Description |
-|-------------|--------|-----------------|-------------------|-------------|
-| `0` | -- | -- | -- | Personal access token |
-| `11` | `0` | 11 | 0 | Pre-obtained OAuth access token |
-| `11` | `1` | 11 | 1 | M2M: service principal |
-| `11` | `2` | 11 | 2 | U2M: browser-based auth code + PKCE |
+| Value | Description |
+|-------|-------------|
+| `access_token` | Personal access token |
+| `oauth_m2m` | M2M: client credentials for service principals |
+| `oauth_u2m` | U2M: browser-based authorization code + PKCE |
 
 ### Credential and OAuth Options
 
 | Option | Type | Default | Required For | Description |
 |--------|------|---------|-------------|-------------|
-| `databricks.access_token` | String | -- | mechanism=`0`, flow=`0` | Access token (PAT or OAuth) |
-| `databricks.auth.client_id` | String | `"databricks-cli"` (flow=`2`) | flow=`1` (required), flow=`2` (optional) | OAuth client ID |
-| `databricks.auth.client_secret` | String | -- | flow=`1` | OAuth client secret |
-| `databricks.auth.scopes` | String | `"all-apis offline_access"` (flow=`2`), `"all-apis"` (flow=`1`) | No | Space-separated OAuth scopes |
+| `databricks.access_token` | String | -- | `access_token` | Personal access token |
+| `databricks.auth.client_id` | String | `"databricks-cli"` (`oauth_u2m`) | `oauth_m2m` (required), `oauth_u2m` (optional) | OAuth client ID |
+| `databricks.auth.client_secret` | String | -- | `oauth_m2m` | OAuth client secret |
+| `databricks.auth.scopes` | String | `"all-apis offline_access"` (`oauth_u2m`), `"all-apis"` (`oauth_m2m`) | No | Space-separated OAuth scopes |
 | `databricks.auth.token_endpoint` | String | Auto-discovered via OIDC | No | Override OIDC-discovered token endpoint |
 | `databricks.auth.redirect_port` | String | `"8020"` | No | Localhost port for browser callback server |
 
-Both `mechanism` and `flow` are mandatory -- no auto-detection. This makes configuration explicit and predictable, matching the ODBC driver's approach where `AuthMech` and `Auth_Flow` are always specified.
+`databricks.auth.type` is mandatory -- no auto-detection. This makes configuration explicit and predictable.
 
 ---
 
@@ -447,9 +429,9 @@ The `AuthProvider::get_auth_header()` trait method is synchronous, but OAuth tok
 // 1. Create HTTP client (no auth yet)
 let http_client = Arc::new(DatabricksHttpClient::new(self.http_config.clone())?);
 
-// 2. Create auth provider based on mechanism + flow enums
+// 2. Create auth provider based on auth type
 //    (see database.rs section below for full match logic)
-let auth_provider: Arc<dyn AuthProvider> = /* match on AuthMechanism/AuthFlow */;
+let auth_provider: Arc<dyn AuthProvider> = /* match on AuthType */;
 
 // 3. Set auth on HTTP client
 http_client.set_auth_provider(auth_provider);
@@ -488,11 +470,10 @@ impl DatabricksHttpClient {
 
 | Scenario | Error Kind | Behavior |
 |----------|-----------|----------|
-| Missing `databricks.auth.mechanism` | `invalid_argument()` | Fail at `new_connection()` |
-| Missing `databricks.auth.flow` when mechanism=`11` | `invalid_argument()` | Fail at `new_connection()` |
-| Invalid numeric value for mechanism or flow | `invalid_argument()` | Fail at `set_option()` |
-| Missing `client_id` or `client_secret` for flow=`1` | `invalid_argument()` | Fail at `new_connection()` |
-| Missing `access_token` for mechanism=`0` or flow=`0` | `invalid_argument()` | Fail at `new_connection()` |
+| Missing `databricks.auth.type` | `invalid_argument()` | Fail at `new_connection()` |
+| Invalid value for `databricks.auth.type` | `invalid_argument()` | Fail at `set_option()` |
+| Missing `client_id` or `client_secret` for `oauth_m2m` | `invalid_argument()` | Fail at `new_connection()` |
+| Missing `access_token` for `access_token` type | `invalid_argument()` | Fail at `new_connection()` |
 | OIDC discovery HTTP failure | `io()` | Fail at provider creation |
 | Token endpoint returns error | `io()` | Fail at `get_auth_header()` |
 | Browser callback timeout (120s) | `io()` | Fail at provider creation |
@@ -518,65 +499,39 @@ Two-phase auth initialization via `OnceLock` (see [Concurrency Model](#concurren
 
 **New fields on `Database`:**
 ```rust
-auth_mechanism: Option<AuthMechanism>,
-auth_flow: Option<AuthFlow>,
-auth_client_id: Option<String>,
-auth_client_secret: Option<String>,
-auth_scopes: Option<String>,
-auth_token_endpoint: Option<String>,
-auth_redirect_port: Option<u16>,
+auth_config: AuthConfig,  // groups all auth-related options
 ```
 
-`set_option` parses numeric config values into the enums:
+`set_option` parses the auth type string:
 ```rust
-"databricks.auth.mechanism" => {
-    let v = Self::parse_int_option(&value)
-        .ok_or_else(|| /* error: expected integer */)?;
-    self.auth_mechanism = Some(AuthMechanism::try_from(v)?);  // 0 -> Pat, 11 -> OAuth
-}
-"databricks.auth.flow" => {
-    let v = Self::parse_int_option(&value)
-        .ok_or_else(|| /* error: expected integer */)?;
-    self.auth_flow = Some(AuthFlow::try_from(v)?);  // 0 -> TokenPassthrough, 1 -> ClientCredentials, 2 -> Browser
+"databricks.auth.type" => {
+    let v = value.as_ref();
+    self.auth_config.auth_type = Some(AuthType::try_from(v)?);
+    // "access_token" -> AccessToken, "oauth_m2m" -> OAuthM2m, "oauth_u2m" -> OAuthU2m
 }
 ```
 
-**Modified `new_connection()`:** Two-phase initialization with enum matching:
+**Modified `new_connection()`:** Two-phase initialization with auth type matching:
 
 ```rust
 // Phase 1: Create HTTP client (no auth yet)
 let http_client = Arc::new(DatabricksHttpClient::new(self.http_config.clone())?);
 
-// Phase 2: Create auth provider based on mechanism + flow
-let mechanism = self.auth_mechanism.as_ref()
-    .ok_or_else(|| /* error: databricks.auth.mechanism is required */)?;
+// Phase 2: Create auth provider based on auth type
+let auth_type = self.auth_config.validate(&self.access_token)?;
 
-let auth_provider: Arc<dyn AuthProvider> = match mechanism {
-    AuthMechanism::Pat => {
+let auth_provider: Arc<dyn AuthProvider> = match auth_type {
+    AuthType::AccessToken => {
         let token = self.access_token.as_ref()
-            .ok_or_else(|| /* error: access_token required for mechanism=0 */)?;
+            .ok_or_else(|| /* error: access_token required */)?;
         Arc::new(PersonalAccessToken::new(token))
     }
-    AuthMechanism::OAuth => {
-        let flow = self.auth_flow.as_ref()
-            .ok_or_else(|| /* error: databricks.auth.flow required when mechanism=11 */)?;
-        match flow {
-            AuthFlow::TokenPassthrough => {
-                // No auto-refresh -- token is used as-is until it expires.
-                // Matches ODBC behavior where expired tokens require the caller
-                // to provide a new token via SQLSetConnectAttr.
-                let token = self.access_token.as_ref()
-                    .ok_or_else(|| /* error: access_token required for flow=0 */)?;
-                Arc::new(PersonalAccessToken::new(token))
-            }
-            AuthFlow::ClientCredentials => Arc::new(
-                ClientCredentialsProvider::new(host, client_id, client_secret, http_client.clone())?
-            ),
-            AuthFlow::Browser => Arc::new(
-                AuthorizationCodeProvider::new(host, client_id, http_client.clone())?
-            ),
-        }
-    }
+    AuthType::OAuthM2m => Arc::new(
+        ClientCredentialsProvider::new(host, client_id, client_secret, http_client.clone())?
+    ),
+    AuthType::OAuthU2m => Arc::new(
+        AuthorizationCodeProvider::new(host, client_id, http_client.clone())?
+    ),
 };
 
 // Phase 3: Wire auth into HTTP client