Multiple authentication methods in ENV fail

[joshbrumpton-octopus]

Describe the bug

The dbt-databricks adapter fails when multiple Databricks authentication credentials are present in the environment, even when profiles.yml explicitly configures a specific auth method.

When profiles.yml is configured to use OAuth (client_id and client_secret), the adapter still fails if DATABRICKS_TOKEN exists in the environment, despite not being referenced in the profile.

The root cause is the Databricks SDK's strict validation which rejects any configuration where multiple auth methods are detected. I've raised a related issue there: databricks/databricks-sdk-py#1183

Steps To Reproduce

1. Set up profiles.yml with OAuth authentication:

my_project:
  target: dev
  outputs:
    dev:
      type: databricks
      host: my-workspace.cloud.databricks.com
      http_path: /sql/1.0/warehouses/xxxxx
      client_id: "{{ env_var('DATABRICKS_CLIENT_ID') }}"
      client_secret: "{{ env_var('DATABRICKS_CLIENT_SECRET') }}"
      catalog: my_catalog
      schema: my_schema

2. Set environment variables including an unused DATABRICKS_TOKEN:

export DATABRICKS_TOKEN="dapi..."
export DATABRICKS_CLIENT_ID="my-client-id"
export DATABRICKS_CLIENT_SECRET="my-client-secret"

3. Run any dbt command:

dbt debug

Expected behavior

When auth is explicitly configured in profiles.yml, the adapter should use only those credentials and not fail due to unrelated environment variables.

Screenshots and log output

  File "/home/circleci/project/.venv/lib/python3.11/site-packages/dbt/adapters/databricks/credentials.py", line 130, in __post_init__
    self._credentials_manager = DatabricksCredentialManager.create_from(self)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/circleci/project/.venv/lib/python3.11/site-packages/dbt/adapters/databricks/credentials.py", line 280, in create_from
    return DatabricksCredentialManager(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 12, in __init__
  File "/home/circleci/project/.venv/lib/python3.11/site-packages/dbt/adapters/databricks/credentials.py", line 337, in __post_init__
    self._config = self.authenticate_with_pat()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/circleci/project/.venv/lib/python3.11/site-packages/dbt/adapters/databricks/credentials.py", line 293, in authenticate_with_pat
    return Config(
           ^^^^^^^
  File "/home/circleci/project/.venv/lib/python3.11/site-packages/databricks/sdk/config.py", line 190, in __init__
    raise ValueError(message) from e
ValueError: validate: more than one authorization method configured: oauth and pat. Config: host=https://*****************************************, token=***, client_id=************************************, client_secret=***. Env: DATABRICKS_TOKEN, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET

System information

dbt-databricks version

1.10.3

python version

Python 3.11

Additional context

This is particularly problematic in CI/CD environments where multiple Databricks credentials exist. e.g. when migrating to OAuth. The explicit configuration should take precedence over environment and not penalise an arguably cluttered env.

Current workaround:

unset DATABRICKS_TOKEN
dbt run

Possible fixes in dbt-databricks while waiting for upstream SDK changes:

1. Temporarily unset conflicting env vars before initializing the SDK Config object
2. Pass an auth_type parameter to the SDK (if/when supported) to restrict credential scanning