[core] periodically reload Ray service account tokens by andrewsykim · Pull Request #60778 · ray-project/ray

andrewsykim · 2026-02-05T18:21:24Z

Description

Ray token auth (with k8s) loads a projected Kubernetes service account token in /var/run/secrets/ray.io/serviceaccount/token. However, this token expires every hour by default. This PR updates Ray to periodically reload this token every 5 minutes to ensure expired service tokens are not cached.

Related issues

Link related issues: "Fixes #1234", "Closes #1234", or "Related to #1234".

Additional information

Optional: Add implementation details, API changes, usage examples, screenshots, etc.

gemini-code-assist

Code Review

This pull request introduces a mechanism to periodically reload Ray service account tokens, which is a crucial improvement for Kubernetes deployments where tokens expire. The implementation correctly uses std::chrono::steady_clock to track the token's age and invalidate the cache when necessary. My review focuses on improving code maintainability by addressing duplicated logic. Specifically, the token expiration check is duplicated in GetToken and TryLoadToken and should be refactored into a single helper function. I've also noted a minor style improvement for a constant declaration and a misleading comment that should be clarified.

gemini-code-assist · 2026-02-05T18:22:32Z

src/ray/rpc/authentication/authentication_token_loader.cc

    "or store the token in any file and set RAY_AUTH_TOKEN_PATH to point to it, "
    "or set the RAY_AUTH_TOKEN environment variable.";

+const int32_t kRaySATokenTTLSeconds = 300;


For a compile-time constant like this, it's preferable to use constexpr instead of const. This allows the value to be used in more compile-time contexts and makes the intent clearer.

Suggested change

const int32_t kRaySATokenTTLSeconds = 300;

constexpr int32_t kRaySATokenTTLSeconds = 300;

gemini-code-assist · 2026-02-05T18:22:32Z

src/ray/rpc/authentication/authentication_token_loader.cc

+  // If k8s token auth is enabled, revoke cached token as Kubelet
+  // will expire and auto rotate new service account tokens every hour by default.
+  // Use 5 minutes as a default as users can configure the expiration time.
+  if (IsK8sTokenAuthEnabled()) {
+    if (cached_token_ && std::chrono::steady_clock::now() - last_load_time_ >
+                             std::chrono::seconds(kRaySATokenTTLSeconds)) {
+      cached_token_ = nullptr;
+    }
+  }


This logic for expiring the cached token is duplicated in TryLoadToken (lines 100-108). To improve maintainability and avoid future inconsistencies, consider extracting it into a private helper function that can be called from both methods after acquiring the lock.

Additionally, the comment on line 60 is a bit misleading as it suggests the expiration time is user-configurable, while kRaySATokenTTLSeconds is a constant. If configurability is not intended, consider updating the comment to something like: // Use a 5-minute interval for reloading to ensure the token is fresh, as Kubelet rotates service account tokens every hour by default.

I think this is fine to ignore, there's already some duplication between GetToken and TryLoadToken, not sure it's worth refactoring

gemini-code-assist · 2026-02-05T18:22:32Z

src/ray/rpc/authentication/authentication_token_loader.cc

+  // If k8s token auth is enabled, revoke cached token as Kubelet
+  // will expire and auto rotate new service account tokens every hour by default.
+  // Use 5 minutes as a default as users can configure the expiration time.
+  if (IsK8sTokenAuthEnabled()) {
+    if (cached_token_ && std::chrono::steady_clock::now() - last_load_time_ >
+                             std::chrono::seconds(kRaySATokenTTLSeconds)) {
+      cached_token_ = nullptr;
+    }
+  }


This is a duplicate of the token expiration logic found in GetToken. As mentioned in my other comment, this should be extracted to a private helper function to avoid code duplication and improve maintainability.

I think this is fine to ignore, there's already some duplication between GetToken and TryLoadToken, not sure it's worth refactoring

sampan-s-nayak · 2026-02-06T04:36:47Z

src/ray/rpc/authentication/authentication_token_loader.cc

    "or store the token in any file and set RAY_AUTH_TOKEN_PATH to point to it, "
    "or set the RAY_AUTH_TOKEN environment variable.";

+constexpr int kRaySATokenTTLSeconds = 300;


is it possible that the token is loaded right before it is rotated by k8s? in this case all RPC calls using the cached token would fail for 5min.

k8s serviceaccount tokens are JWT's and JWT tokens usually have an exp field which gives us an idea of the tokens validity. can we rely on this instead of a blanket 5min TTL?

https://kubernetes.io/docs/concepts/security/service-accounts/#authenticating-credentials

also lets add a unit test for this

In the case of the default 1 hour expiration, I believe a new token will be automounted well before 5 minutes. But agree that checking the expiration in the token is better. I believe Go clients watch for filesystem changes to the token path but I wasn't srue how to implement that in C++ :)

Will look into this a bit and try to add unit tests

@sampan-s-nayak added logic to check expiration in the JWT token, but kept a buffer so we attempt reload at least 5 minute before the expiration. Also added a unit test

src/ray/rpc/authentication/authentication_token_loader.h

src/ray/rpc/authentication/tests/authentication_token_loader_test.cc

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

src/ray/rpc/authentication/authentication_token_loader.cc

…iration when k8s token auth is enabled Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

src/ray/rpc/authentication/authentication_token_loader.cc

cursor · 2026-02-09T17:27:01Z

src/ray/rpc/authentication/authentication_token_loader.cc

+      if (exp) {
+        cached_token_expiration_time_ =
+            *exp - std::chrono::seconds(kRaySATokenExpirationBufferSeconds);
+      } else {


Token thrashing when remaining lifetime under buffer

High Severity

When a K8s token has less than 5 minutes remaining before expiration, cached_token_expiration_time_ is set to a past time (token's exp minus 300 seconds). This causes every subsequent GetToken() call to immediately invalidate the cached token and reload it from the filesystem. In busy systems making frequent RPC calls, this results in excessive filesystem reads until the token is actually rotated by Kubernetes, defeating the caching mechanism and potentially causing performance degradation.

This won't be a problem as Kubernetes enforces a minimum expiration time of 10 minutes

src/ray/rpc/authentication/tests/authentication_token_loader_test.cc

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

src/ray/rpc/authentication/authentication_token_loader.cc

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

andrewsykim requested a review from a team as a code owner February 5, 2026 18:21

gemini-code-assist bot reviewed Feb 5, 2026

View reviewed changes

ray-gardener bot added the community-contribution Contributed by the community label Feb 5, 2026

andrewsykim force-pushed the ray-reload-cached-token branch from ba63f2a to b1d659f Compare February 5, 2026 21:30

andrewsykim assigned edoakes and sampan-s-nayak Feb 5, 2026

edoakes added the go add ONLY when ready to merge, run all tests label Feb 5, 2026

sampan-s-nayak reviewed Feb 6, 2026

View reviewed changes

andrewsykim force-pushed the ray-reload-cached-token branch from b1d659f to fe96dcc Compare February 6, 2026 19:24

cursor bot reviewed Feb 6, 2026

View reviewed changes

src/ray/rpc/authentication/authentication_token_loader.h Outdated Show resolved Hide resolved

cursor bot reviewed Feb 6, 2026

View reviewed changes

src/ray/rpc/authentication/tests/authentication_token_loader_test.cc Show resolved Hide resolved

andrewsykim force-pushed the ray-reload-cached-token branch from 86f2c22 to 2125d90 Compare February 9, 2026 14:11

andrewsykim added 6 commits February 9, 2026 15:47

[core] periodically reload Ray service account tokens

290ba21

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

use constexpr int instead of const int32_t

e88e1e2

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

add initial code to check expiration time in JWT tokens

8b941ba

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

remove unused variable last_load_time

c0cad6a

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

add buffer to JWT expiration time

8d83ebf

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

set expiration time to buffer + 2 seconds

c855f26

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

andrewsykim force-pushed the ray-reload-cached-token branch from 2125d90 to 17f987a Compare February 9, 2026 15:47

cursor bot reviewed Feb 9, 2026

View reviewed changes

src/ray/rpc/authentication/authentication_token_loader.cc Show resolved Hide resolved

andrewsykim force-pushed the ray-reload-cached-token branch from 17f987a to c855f26 Compare February 9, 2026 16:51

log warnings on failure to parse JWT tokens and only call GetTokenExp…

b729301

…iration when k8s token auth is enabled Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

cursor bot reviewed Feb 9, 2026

View reviewed changes

adjust expiration time to reduce liklihood of flaky test

7b8caa3

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

cursor bot reviewed Feb 9, 2026

View reviewed changes

src/ray/rpc/authentication/authentication_token_loader.cc Outdated Show resolved Hide resolved

update JWT token parsing to use WebSafeBase64Unescape

8e8d391

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

	const int32_t kRaySATokenTTLSeconds = 300;
	constexpr int32_t kRaySATokenTTLSeconds = 300;

Conversation

andrewsykim commented Feb 5, 2026

Description

Related issues

Additional information

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot Feb 9, 2026

Choose a reason for hiding this comment

Token thrashing when remaining lifetime under buffer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants