Skip to content

JWT expires on long-running dbt jobs #482

@RenanGreca

Description

@RenanGreca

Expected behavior

When using dbt-trino with the jwt authentication method, we would like dbt tasks such as dbt run to succeed regardless of the number of models and/or query execution times.

Actual behavior

If a single call to e.g. dbt run lasts longer than the JWT expiration (e.g. 60 minutes), the token expires and Trino returns "permission denied" errors.

Steps To Reproduce

In my case, I have a programmatic call to dbt, which uses details from a profiles.yml file to authenticate with Trino. This profiles.yml refers to an environment variable containing the token. Even if the variable is refreshed mid-run, the dbt-trino connector only uses the token set at the start of execution.

I have identified two possible avenues for mitigation:

  1. In my code, I patched the TrinoJwtCredentials.trino_auth method to generate a new token each time, rather than using the initial one. This works for me, but can't be a general solution because each identity provider will have a different token generation procedure. This is also prone to break if the dbt-trino connector changes. Perhaps we could add an interface for implementing custom token generators more cleanly.

Sample:

from dbt.adapters.trino.connections import TrinoJwtCredentials

def trino_auth(self):
    """Overrides TrinoJwtCredentials.trino_auth from the dbt-trino connector."""
    self.jwt_token = my_access_token_generator()
    return trino.auth.JWTAuthentication(self.jwt_token)

TrinoJwtCredentials.trino_auth = trino_auth
  1. In TrinoJwtCredentials, we could add a property for the name of an environment variable containing the JWT. Then, in trino_auth, we could read from that environment variable. Then it remains the user's responsibility to refresh tokens periodically.

Sample:

class TrinoJwtCredentials(TrinoCredentials):
    jwt_env_var: str | None
    ...

    def trino_auth(self):
        if (self.jwt_env_var):
            token = os.getenv(self.jwt_env_var)
            return trino.auth.JWTAuthentication(token)

        return trino.auth.JWTAuthentication(self.jwt_token)

Log output/Screenshots

No response

Operating System

Ubuntu 24.04

dbt version

1.9.1

Trino Server version

436

Python version

3.10

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions