Skip to content

Commit a2ec565

Browse files
Add native Azure DevOps OIDC authentication support (#512)
Added native support for Azure DevOps OIDC authentication to allow authentication from Azure DevOps pipeline's environment. ## Testing Tested on an Azure DevOps project. Used the SDK to authenticate with a Databrick's workspace using Azure DevOps OIDC 1) Created a demo Java files that uses the SDK to create a Workspace Client: This will test if the OIDC authentication is working <img width="1914" height="988" alt="Screenshot 2025-09-24 at 12 52 23" src="https://github.com/user-attachments/assets/964acc19-e903-4a06-bee4-005d9d2a4696" /> 2) Created a pipeline to run the demo file. Set the necessary environment variables. (System.AccessToken is slightly different, it needs to be exported through pipeline syntax: https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken) <img width="1913" height="988" alt="Screenshot 2025-09-24 at 12 52 48" src="https://github.com/user-attachments/assets/8d8b682f-dac5-4e79-8f1d-fb8c03959a3b" /> 3) The pipeline runs successfully indicating that authentication succeeded. <img width="961" height="494" alt="Screenshot 2025-09-24 at 12 58 17" src="https://github.com/user-attachments/assets/aaf44560-6984-4e63-a849-38a54a3dfd28" /> --------- Co-authored-by: Renaud Hartert <[email protected]>
1 parent 0e52976 commit a2ec565

File tree

7 files changed

+442
-151
lines changed

7 files changed

+442
-151
lines changed

NEXT_CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
### New Features and Improvements
66

7+
* Add native support for Azure DevOps OIDC authentication.
8+
79
### Bug Fixes
810

911
### Documentation

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,10 +116,11 @@ Depending on the Databricks authentication method, the SDK uses the following in
116116

117117
### Databricks native authentication
118118

119-
By default, the Databricks SDK for Java initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks Workload Identity Federation (WIF) authentication using OIDC (`auth_type="github-oidc"` argument).
119+
By default, the Databricks SDK for Java initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Workload Identity Federation (WIF). See [Supported WIF](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider) for the supported JWT token providers.
120120

121121
- For Databricks token authentication, you must provide `host` and `token`; or their environment variable or `.databrickscfg` file field equivalents.
122122
- For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file.
123+
- For Azure DevOps OIDC authentication, the `token_audience` is irrelevant as the audience is always set to `api://AzureADTokenExchange`. Also, the `System.AccessToken` pipeline variable required for OIDC request must be exposed as the `SYSTEM_ACCESSTOKEN` environment variable, following [Pipeline variables](https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken)
123124

124125
| Argument | Description | Environment variable |
125126
|--------------|-------------|-------------------|

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DefaultCredentialsProvider.java

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,16 @@ private void addOIDCCredentialsProviders(DatabricksConfig config) {
125125
config.getActionsIdTokenRequestUrl(),
126126
config.getActionsIdTokenRequestToken(),
127127
config.getHttpClient())));
128+
129+
// Try to create Azure DevOps token source - if environment variables are missing,
130+
// skip this provider gracefully.
131+
try {
132+
namedIdTokenSources.add(
133+
new NamedIDTokenSource(
134+
"azure-devops-oidc", new AzureDevOpsIDTokenSource(config.getHttpClient())));
135+
} catch (DatabricksException e) {
136+
LOG.debug("Azure DevOps OIDC provider not available: {}", e.getMessage());
137+
}
128138
// Add new IDTokenSources and ID providers here. Example:
129139
// namedIdTokenSources.add(new NamedIDTokenSource("custom-oidc", new CustomIDTokenSource(...)));
130140

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
package com.databricks.sdk.core.oauth;
2+
3+
import com.databricks.sdk.core.DatabricksException;
4+
import com.databricks.sdk.core.http.HttpClient;
5+
import com.databricks.sdk.core.http.Request;
6+
import com.databricks.sdk.core.http.Response;
7+
import com.databricks.sdk.core.utils.Environment;
8+
import com.fasterxml.jackson.databind.ObjectMapper;
9+
import com.fasterxml.jackson.databind.node.ObjectNode;
10+
import com.google.common.base.Strings;
11+
import java.io.IOException;
12+
13+
/**
14+
* AzureDevOpsIDTokenSource retrieves JWT Tokens from Azure DevOps Pipelines. This class implements
15+
* the IDTokenSource interface and provides a method for obtaining ID tokens specifically from Azure
16+
* DevOps Pipeline environment.
17+
*
18+
* <p>This implementation relies on the <a
19+
* href="https://learn.microsoft.com/en-us/rest/api/azure/devops/distributedtask/oidctoken/create">Azure
20+
* DevOps OIDC token API</a>.
21+
*/
22+
public class AzureDevOpsIDTokenSource implements IDTokenSource {
23+
/* Access token for authenticating with Azure DevOps API */
24+
private final String azureDevOpsAccessToken;
25+
/* Team Foundation Collection URI (e.g., https://dev.azure.com/organization) */
26+
private final String azureDevOpsTeamFoundationCollectionUri;
27+
/* Plan ID for the current pipeline run */
28+
private final String azureDevOpsPlanId;
29+
/* Job ID for the current pipeline job */
30+
private final String azureDevOpsJobId;
31+
/* Team Project ID where the pipeline is running */
32+
private final String azureDevOpsTeamProjectId;
33+
/* Host type (e.g., "build", "release") */
34+
private final String azureDevOpsHostType;
35+
/* HTTP client for making requests to Azure DevOps */
36+
private final HttpClient httpClient;
37+
/* Environment for reading configuration values */
38+
private final Environment environment;
39+
/* JSON mapper for parsing response data */
40+
private static final ObjectMapper mapper = new ObjectMapper();
41+
42+
/**
43+
* Constructs a new AzureDevOpsIDTokenSource by reading environment variables. This constructor
44+
* implements fail-early validation - if any required environment variables are missing, it will
45+
* throw a DatabricksException immediately.
46+
*
47+
* @param httpClient The HTTP client to use for making requests
48+
* @throws DatabricksException if any required environment variables are missing
49+
*/
50+
public AzureDevOpsIDTokenSource(HttpClient httpClient) {
51+
this(httpClient, createDefaultEnvironment());
52+
}
53+
54+
/**
55+
* Constructs a new AzureDevOpsIDTokenSource with a custom environment. This constructor is
56+
* primarily used for testing to inject mock environment variables.
57+
*
58+
* @param httpClient The HTTP client to use for making requests
59+
* @param environment The environment to read configuration from
60+
* @throws DatabricksException if httpClient is null or any required environment variables are
61+
* missing
62+
*/
63+
protected AzureDevOpsIDTokenSource(HttpClient httpClient, Environment environment) {
64+
if (httpClient == null) {
65+
throw new DatabricksException("HttpClient cannot be null");
66+
}
67+
this.httpClient = httpClient;
68+
this.environment = environment;
69+
70+
this.azureDevOpsAccessToken = validateEnvironmentVariable("SYSTEM_ACCESSTOKEN");
71+
this.azureDevOpsTeamFoundationCollectionUri =
72+
validateEnvironmentVariable("SYSTEM_TEAMFOUNDATIONCOLLECTIONURI");
73+
this.azureDevOpsPlanId = validateEnvironmentVariable("SYSTEM_PLANID");
74+
this.azureDevOpsJobId = validateEnvironmentVariable("SYSTEM_JOBID");
75+
this.azureDevOpsTeamProjectId = validateEnvironmentVariable("SYSTEM_TEAMPROJECTID");
76+
this.azureDevOpsHostType = validateEnvironmentVariable("SYSTEM_HOSTTYPE");
77+
}
78+
79+
/** Creates a default Environment using system environment variables. */
80+
private static Environment createDefaultEnvironment() {
81+
String pathEnv = System.getenv("PATH");
82+
String[] pathArray =
83+
pathEnv != null ? pathEnv.split(java.io.File.pathSeparator) : new String[0];
84+
return new Environment(System.getenv(), pathArray, System.getProperty("os.name"));
85+
}
86+
87+
/**
88+
* Validates that an environment variable is present and not empty.
89+
*
90+
* @param varName The environment variable name
91+
* @return The environment variable value
92+
* @throws DatabricksException if the environment variable is missing or empty
93+
*/
94+
private String validateEnvironmentVariable(String varName) {
95+
String value = environment.get(varName);
96+
if (Strings.isNullOrEmpty(value)) {
97+
if (varName.equals("SYSTEM_ACCESSTOKEN")) {
98+
throw new DatabricksException(
99+
String.format(
100+
"Missing environment variable %s, if calling from Azure DevOps Pipeline, please set this env var following https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken",
101+
varName));
102+
}
103+
throw new DatabricksException(
104+
String.format(
105+
"Missing environment variable %s, likely not calling from Azure DevOps Pipeline",
106+
varName));
107+
}
108+
return value;
109+
}
110+
111+
/**
112+
* Retrieves an ID token from Azure DevOps Pipelines. This method makes an authenticated request
113+
* to Azure DevOps to obtain a JWT token that can later be exchanged for a Databricks access
114+
* token.
115+
*
116+
* <p>Note: The audience parameter is ignored for Azure DevOps OIDC tokens as they have a
117+
* hardcoded audience for Azure AD integration.
118+
*
119+
* @param audience Ignored for Azure DevOps OIDC tokens
120+
* @return An IDToken object containing the JWT token value
121+
* @throws DatabricksException if the token request fails
122+
*/
123+
@Override
124+
public IDToken getIDToken(String audience) {
125+
126+
// Build Azure DevOps OIDC endpoint URL.
127+
String requestUrl =
128+
String.format(
129+
"%s/%s/_apis/distributedtask/hubs/%s/plans/%s/jobs/%s/oidctoken?api-version=7.2-preview.1",
130+
azureDevOpsTeamFoundationCollectionUri,
131+
azureDevOpsTeamProjectId,
132+
azureDevOpsHostType,
133+
azureDevOpsPlanId,
134+
azureDevOpsJobId);
135+
136+
Request req =
137+
new Request("POST", requestUrl)
138+
.withHeader("Authorization", "Bearer " + azureDevOpsAccessToken)
139+
.withHeader("Content-Type", "application/json");
140+
141+
Response resp;
142+
try {
143+
resp = httpClient.execute(req);
144+
} catch (IOException e) {
145+
throw new DatabricksException(
146+
"Failed to request ID token from Azure DevOps at " + requestUrl + ": " + e.getMessage(),
147+
e);
148+
}
149+
150+
if (resp.getStatusCode() != 200) {
151+
throw new DatabricksException(
152+
"Failed to request ID token from Azure DevOps: status code "
153+
+ resp.getStatusCode()
154+
+ ", response body: "
155+
+ resp.getBody().toString());
156+
}
157+
158+
ObjectNode jsonResp;
159+
try {
160+
jsonResp = mapper.readValue(resp.getBody(), ObjectNode.class);
161+
} catch (IOException e) {
162+
throw new DatabricksException(
163+
"Failed to parse Azure DevOps OIDC token response: " + e.getMessage(), e);
164+
}
165+
166+
// Azure DevOps returns {"oidcToken":"***"} format, not {"value":"***"} like GitHub Actions.
167+
if (!jsonResp.has("oidcToken")) {
168+
throw new DatabricksException("Azure DevOps OIDC token response missing 'oidcToken' field");
169+
}
170+
171+
String tokenValue = jsonResp.get("oidcToken").textValue();
172+
if (Strings.isNullOrEmpty(tokenValue)) {
173+
throw new DatabricksException("Received empty OIDC token from Azure DevOps");
174+
}
175+
return new IDToken(tokenValue);
176+
}
177+
}

databricks-sdk-java/src/main/java/com/databricks/sdk/core/oauth/GitHubOidcTokenSupplier.java

Lines changed: 0 additions & 79 deletions
This file was deleted.

databricks-sdk-java/src/main/java/com/databricks/sdk/core/oauth/GithubOidcCredentialsProvider.java

Lines changed: 0 additions & 71 deletions
This file was deleted.

0 commit comments

Comments
 (0)