Vertex AI Generative: ACCESS_TOKEN_EXPIRED

[Fov6363]

Client

Vertex AI Generative (Go SDK, streaming)

Environment

• OS / Container: K8s on cloud service
• Runtime: Go 1.24.9
• Deployment: Kubernetes (long-lived pods)
• Auth: Service Account JSON via GOOGLE_APPLICATION_CREDENTIALS

Code and Dependencies

  package main

  import (
  	"context"
  	"fmt"

  	"google.golang.org/genai"
  )

  func main() {
  	ctx := context.Background()
  	client, err := genai.NewClient(ctx, &genai.ClientConfig{
  		Backend:  genai.BackendVertexAI,
  		Project:  "PROJECT_ID",
  		Location: "REGION",
  	})
  	if err != nil {
  		panic(err)
  	}

  	iter := client.Models.GenerateContentStream(ctx, "MODEL",
  		[]*genai.Content{
  			genai.NewContentFromText("Please produce a long response.", genai.RoleUser),
  		},
  		&genai.GenerateContentConfig{},
  	)

  	for resp, err := range iter {
  		if err != nil {
  			fmt.Printf("stream error: %v\n", err)
  			return
  		}
  		_ = resp
  	}
  }

go.mod

module rpc

go 1.24.9

require (
google.golang.org/genai v1.40.0
google.golang.org/grpc v1.74.2
cloud.google.com/go/auth v0.16.2 // indirect
cloud.google.com/go v0.121.2 // indirect
// ... (other deps)
)

Expected behavior

Streaming should survive access token refresh; the SDK should renew tokens and keep the
active stream alive without returning UNAUTHENTICATED.

Actual behavior

When the access token expires during an already-established stream, the SDK returns
UNAUTHENTICATED with reason:ACCESS_TOKEN_EXPIRED and the stream ends.

Screenshots

Error 401, Message: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project., Status: UNAUTHENTICATED, Details: [map[@type:type.googleapis.com/google.rpc.ErrorInfo domain:googleapis.com metadata:map[method:google.cloud.aiplatform.v1beta1.PredictionService.StreamGenerateContent service:aiplatform.googleapis.com] reason:ACCESS_TOKEN_EXPIRED]]

Additional context

• It's not always possible to reproduce the error. Observing the logs, there are occasionally a few concentrated error events over 5 days.
• Using service account JSON credentials via ADC (GOOGLE_APPLICATION_CREDENTIALS).
• Stream duration can exceed token lifetime.
• Error example: UNAUTHENTICATED: Request had invalid authentication credentials.
  reason:ACCESS_TOKEN_EXPIRED.

[quartzmo]

@Fov6363 Thanks for reporting this.

[quartzmo]

Labeling as feature request while I investigate the expected behavior proposed in the description.

[quartzmo]

This Module is Deprecated

Please note from the Vertex AI README.md:

Note

Starting on June 24, 2025, the cloud.google.com/go/vertexai/genai package is deprecated and will be removed on June 24, 2026, except cloud.google.com/go/vertexai/genai/tokenizer.
New users are encouraged to use the Google GenAI Go SDK available at google.golang.org/genai.
For information about migrating to the Google Gen AI SDK, see Vertex AI SDK migration guide.

gRPC Streaming

Regarding the stream ending due to token expiration, unfortunately I believe this is the expected behavior for the underlying gRPC transport, although I understand it presents challenges for you with long-running streams.

Technical Details

gRPC streams in the Go SDK (and gRPC in general) establish the authentication context—sending the OAuth2 access token—only once at the beginning of the stream creation (in the initial HTTP/2 headers). The underlying gRPC client does not support refreshing the access token mid-stream for an active RPC.

If a stream remains open longer than the validity of the access token (typically 1 hour), and the backend service validates the token for subsequent messages or periodically checks its validity, the service will terminate the connection with UNAUTHENTICATED when the token expires.

We can see this in vertexai/genai/client.go. The GenerateContentStream method returns an iterator that wraps the raw gRPC stream:

// vertexai/genai/client.go

func (iter *GenerateContentResponseIterator) Next() (*GenerateContentResponse, error) {
    // ...
    resp, err := iter.sc.Recv() // Direct call to the gRPC stream
    // ...
    if err != nil {
        return nil, err // Error returned directly to the user
    }
    // ...
}

The Next() method delegates directly to iter.sc.Recv(). There is no middleware or logic within the generated client to catch UNAUTHENTICATED, refresh credentials, and transparently recreate the stream.

Why automatic resumption is not supported

Automatically "resuming" a stream is not safe for the SDK to perform generically because it implies application-level state management that the SDK cannot assume:

• Vertex AI / GenAI: If a generation stream fails halfway through, a transparent "retry" would effectively restart the generation from scratch (re-sending the prompt). This would result in duplicate content or a completely different response from the model. The SDK does not have the context to ask the model to "continue generation from token X".
• Speech-to-Text: Simply reconnecting a stream would break the continuity of the audio session. To resume correctly, the client would need to re-send unacknowledged audio or manage offset parameters, which requires complex buffering logic specific to the application's consistency requirements.

Recommended Workaround

For streams that are expected to exceed the token lifetime (approx. 1 hour), you need to implement application-level retry logic:

1. Catch the Error: Monitor your stream iteration for UNAUTHENTICATED errors.
2. Restart: If this error occurs, close the current stream and initiate a new stream request.
3. Automatic Refresh: When you call GenerateContentStream (or the equivalent method) again, the SDK's underlying transport will automatically fetch a fresh, valid access token for the new request.
4. Resume Context: (Application specific) You will need to adjust your request to resume from the logical stopping point (e.g., for a Chat application, append the partial response you received to the chat history before starting the new request).

I hope this clarifies the architecture limitations here.