|
| 1 | +# Authentication Types Reference |
| 2 | + |
| 3 | +This document provides a comprehensive reference for all authentication types (`auth_type`) supported by the Databricks SDK for Python. |
| 4 | + |
| 5 | +## Authentication Types Table |
| 6 | + |
| 7 | +| Auth Type | Description | Required Parameters | Optional Parameters | Environment Variables | |
| 8 | +|-----------|-------------|---------------------|---------------------|----------------------| |
| 9 | +| `pat` | Personal Access Token authentication - the most common method for programmatic access | `host`, `token` | - | `DATABRICKS_HOST`, `DATABRICKS_TOKEN` | |
| 10 | +| `basic` | Basic HTTP authentication using username and password (primarily for AWS) | `host`, `username`, `password` | `account_id` (for account-level operations) | `DATABRICKS_HOST`, `DATABRICKS_USERNAME`, `DATABRICKS_PASSWORD`, `DATABRICKS_ACCOUNT_ID` | |
| 11 | +| `oauth-m2m` | OAuth 2.0 Machine-to-Machine (service principal) authentication | `host`, `client_id`, `client_secret` | `scopes`, `authorization_details` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET` | |
| 12 | +| `external-browser` | OAuth 2.0 authentication flow using local browser for user login | `host`, `auth_type='external-browser'` | `client_id`, `client_secret` | `DATABRICKS_HOST`, `DATABRICKS_AUTH_TYPE`, `DATABRICKS_CLIENT_ID` | |
| 13 | +| `databricks-cli` | Uses tokens from the Databricks CLI (`databricks auth login`) | `host` | `account_id` (for account-level), `databricks_cli_path` | `DATABRICKS_HOST`, `DATABRICKS_ACCOUNT_ID`, `DATABRICKS_CLI_PATH` | |
| 14 | +| `azure-client-secret` | Azure Active Directory (AAD) Service Principal authentication | `azure_client_id`, `azure_client_secret`, `azure_tenant_id` | `host`, `azure_workspace_resource_id`, `azure_environment` | `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_TENANT_ID`, `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_ENVIRONMENT` | |
| 15 | +| `azure-cli` | Uses credentials from Azure CLI (`az login`) | `host` (or `azure_workspace_resource_id`) | `azure_tenant_id` | `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_TENANT_ID` | |
| 16 | +| `github-oidc` | GitHub Actions OIDC authentication (workload identity federation) | `host`, `client_id` | `token_audience`, `account_id` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_TOKEN_AUDIENCE`, `DATABRICKS_ACCOUNT_ID` | |
| 17 | +| `github-oidc-azure` | GitHub Actions OIDC for Azure Databricks workspaces | `host`, `azure_client_id` | `azure_tenant_id` | `DATABRICKS_HOST`, `ARM_CLIENT_ID`, `ARM_TENANT_ID` | |
| 18 | +| `azure-devops-oidc` | Azure DevOps Pipelines OIDC authentication | `host`, `client_id` | `token_audience`, `account_id` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `SYSTEM_ACCESSTOKEN` | |
| 19 | +| `google-credentials` | Google Cloud service account authentication using credentials JSON | `host`, `google_credentials` | - | `DATABRICKS_HOST`, `GOOGLE_CREDENTIALS` | |
| 20 | +| `google-id` | Google Cloud authentication using service account impersonation | `host`, `google_service_account` | - | `DATABRICKS_HOST`, `DATABRICKS_GOOGLE_SERVICE_ACCOUNT` | |
| 21 | +| `metadata-service` | Authentication using Databricks-hosted metadata service | `host`, `metadata_service_url` | - | `DATABRICKS_HOST`, `DATABRICKS_METADATA_SERVICE_URL` | |
| 22 | +| `runtime` | Auto-detected authentication when running in Databricks Runtime (notebooks, jobs) | _(auto-detected)_ | - | `DATABRICKS_RUNTIME_VERSION` (auto-set) | |
| 23 | +| `runtime-oauth` | OAuth authentication for Databricks Runtime with fine-grained permissions | `scopes` | `authorization_details` | `DATABRICKS_RUNTIME_VERSION` (auto-set) | |
| 24 | +| `model-serving` | Auto-detected authentication when running in Databricks Model Serving environment | _(auto-detected)_ | - | `IS_IN_DB_MODEL_SERVING_ENV` or `IS_IN_DATABRICKS_MODEL_SERVING_ENV` (auto-set) | |
| 25 | +| `env-oidc` | OIDC token from environment variable | `host` | `oidc_token_env`, `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN`, `DATABRICKS_OIDC_TOKEN_ENV`, `DATABRICKS_CLIENT_ID` | |
| 26 | +| `file-oidc` | OIDC token from file path | `host`, `oidc_token_filepath` | `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN_FILE`, `DATABRICKS_CLIENT_ID` | |
| 27 | + |
| 28 | +## Common Parameters Across All Auth Types |
| 29 | + |
| 30 | +These parameters can be used with any authentication type: |
| 31 | + |
| 32 | +| Parameter | Description | Environment Variable | |
| 33 | +|-----------|-------------|---------------------| |
| 34 | +| `http_timeout_seconds` | HTTP request timeout (default: 60 seconds) | - | |
| 35 | +| `retry_timeout_seconds` | Total retry timeout (default: 300 seconds / 5 minutes) | - | |
| 36 | +| `debug_truncate_bytes` | Truncate debug logs above this size (default: 96 bytes) | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | |
| 37 | +| `debug_headers` | Enable debug logging of HTTP headers (default: false) | `DATABRICKS_DEBUG_HEADERS` | |
| 38 | +| `rate_limit` | Maximum requests per second to Databricks API | `DATABRICKS_RATE_LIMIT` | |
| 39 | +| `skip_verify` | Skip SSL certificate verification (not recommended) | - | |
| 40 | + |
| 41 | +## Usage Examples |
| 42 | + |
| 43 | +### Personal Access Token (PAT) |
| 44 | +```python |
| 45 | +from databricks.sdk import WorkspaceClient |
| 46 | + |
| 47 | +w = WorkspaceClient( |
| 48 | + host="https://your-workspace.cloud.databricks.com", |
| 49 | + token="dapi1234567890abcdef" |
| 50 | +) |
| 51 | +``` |
| 52 | + |
| 53 | +### OAuth Machine-to-Machine (Service Principal) |
| 54 | +```python |
| 55 | +from databricks.sdk import WorkspaceClient |
| 56 | + |
| 57 | +w = WorkspaceClient( |
| 58 | + host="https://your-workspace.cloud.databricks.com", |
| 59 | + client_id="your-client-id", |
| 60 | + client_secret="your-client-secret" |
| 61 | +) |
| 62 | +``` |
| 63 | + |
| 64 | +### External Browser (OAuth for Users) |
| 65 | +```python |
| 66 | +from databricks.sdk import WorkspaceClient |
| 67 | + |
| 68 | +w = WorkspaceClient( |
| 69 | + host="https://your-workspace.cloud.databricks.com", |
| 70 | + auth_type="external-browser" |
| 71 | +) |
| 72 | +``` |
| 73 | + |
| 74 | +### Azure Service Principal |
| 75 | +```python |
| 76 | +from databricks.sdk import WorkspaceClient |
| 77 | + |
| 78 | +w = WorkspaceClient( |
| 79 | + host="https://adb-1234567890.azuredatabricks.net", |
| 80 | + azure_client_id="your-azure-client-id", |
| 81 | + azure_client_secret="your-azure-client-secret", |
| 82 | + azure_tenant_id="your-azure-tenant-id" |
| 83 | +) |
| 84 | +``` |
| 85 | + |
| 86 | +### Databricks CLI |
| 87 | +```python |
| 88 | +from databricks.sdk import WorkspaceClient |
| 89 | + |
| 90 | +# Assumes you've run: databricks auth login --host https://your-workspace.cloud.databricks.com |
| 91 | +w = WorkspaceClient( |
| 92 | + host="https://your-workspace.cloud.databricks.com" |
| 93 | +) |
| 94 | +``` |
| 95 | + |
| 96 | +### GitHub Actions OIDC |
| 97 | +```python |
| 98 | +from databricks.sdk import WorkspaceClient |
| 99 | + |
| 100 | +# In GitHub Actions with OIDC configured |
| 101 | +w = WorkspaceClient( |
| 102 | + host="https://your-workspace.cloud.databricks.com", |
| 103 | + client_id="your-databricks-oauth-client-id" |
| 104 | +) |
| 105 | +``` |
| 106 | + |
| 107 | +### Google Cloud Credentials |
| 108 | +```python |
| 109 | +from databricks.sdk import WorkspaceClient |
| 110 | + |
| 111 | +w = WorkspaceClient( |
| 112 | + host="https://your-workspace.gcp.databricks.com", |
| 113 | + google_credentials="/path/to/service-account-key.json" |
| 114 | +) |
| 115 | +``` |
| 116 | + |
| 117 | +### Runtime (in Databricks Notebooks) |
| 118 | +```python |
| 119 | +from databricks.sdk import WorkspaceClient |
| 120 | + |
| 121 | +# No credentials needed when running in Databricks Runtime |
| 122 | +w = WorkspaceClient() |
| 123 | +``` |
| 124 | + |
| 125 | +## Authentication Priority Order |
| 126 | + |
| 127 | +When no `auth_type` is explicitly specified, the SDK attempts authentication methods in this order: |
| 128 | + |
| 129 | +1. `pat` - Personal Access Token |
| 130 | +2. `basic` - Username/Password |
| 131 | +3. `metadata-service` - Metadata Service (if URL provided) |
| 132 | +4. `oauth-m2m` - OAuth Service Principal |
| 133 | +5. `env-oidc` - Environment OIDC token |
| 134 | +6. `file-oidc` - File-based OIDC token |
| 135 | +7. `github-oidc` - GitHub OIDC |
| 136 | +8. `azure-client-secret` - Azure Service Principal |
| 137 | +9. `github-oidc-azure` - GitHub OIDC for Azure |
| 138 | +10. `azure-cli` - Azure CLI |
| 139 | +11. `azure-devops-oidc` - Azure DevOps OIDC |
| 140 | +12. `external-browser` - Browser-based OAuth |
| 141 | +13. `databricks-cli` - Databricks CLI |
| 142 | +14. `runtime-oauth` - Databricks Runtime OAuth |
| 143 | +15. `runtime` - Databricks Runtime native |
| 144 | +16. `google-credentials` - Google Cloud credentials |
| 145 | +17. `google-id` - Google Cloud ID |
| 146 | +18. `model-serving` - Model Serving environment |
| 147 | + |
| 148 | +You can override this order by explicitly setting the `auth_type` parameter. |
| 149 | + |
| 150 | +## Notes |
| 151 | + |
| 152 | +- **Auto-detected auth types** (`runtime`, `runtime-oauth`, `model-serving`): These are automatically detected based on environment variables and don't require explicit configuration. |
| 153 | +- **Azure authentication**: When using Azure-specific auth types, if `host` is not provided but `azure_workspace_resource_id` is, the SDK will automatically resolve the workspace URL. |
| 154 | +- **OIDC authentication**: OIDC-based methods (`github-oidc`, `azure-devops-oidc`, `env-oidc`, `file-oidc`) use token exchange to obtain Databricks tokens from external identity providers. |
| 155 | +- **Scopes**: OAuth-based methods support the `scopes` parameter for fine-grained access control (e.g., `scopes="clusters sql"`). |
| 156 | + |
| 157 | +## See Also |
| 158 | + |
| 159 | +- [Main README Authentication Section](../README.md#authentication) |
| 160 | +- [OAuth Documentation](./oauth.md) |
| 161 | +- [Azure AD Authentication](./azure-ad.md) |
| 162 | +- [Databricks Authentication Documentation](https://docs.databricks.com/dev-tools/auth.html) |
0 commit comments