Skip to content

Implement token caching for OIDC validation #523

@stmh

Description

@stmh

Issue

OIDC token validation makes an HTTP request to /oauth/userinfo on every API request, causing:

  • High latency (network round-trip on every auth check)
  • Increased load on OIDC provider
  • Potential rate limiting from provider
  • Poor API performance

Location

scotty/src/oauth/device_flow.rs:221-256

Current Code

pub async fn validate_oidc_token(&self, access_token: &str) -> Result<OidcUser, OAuthError> {
    let user_url = format!("{}/oauth/userinfo", self.oidc_issuer_url);
    let response = self.http_client.inner()
        .get(&user_url)
        .bearer_auth(access_token)
        .send().await?;  // HTTP call on EVERY validation!
    // ...
}

Recommendation

Implement in-memory token cache with TTL:

use std::sync::Arc;
use tokio::sync::RwLock;
use std::collections::HashMap;
use std::time::{SystemTime, Duration};

struct TokenCacheEntry {
    user: OidcUser,
    expires_at: SystemTime,
}

pub struct TokenCache {
    cache: Arc<RwLock<HashMap<String, TokenCacheEntry>>>,
}

impl TokenCache {
    pub async fn get_or_validate<F>(
        &self,
        token: &str,
        validator: F,
    ) -> Result<OidcUser, OAuthError>
    where
        F: Future<Output = Result<OidcUser, OAuthError>>,
    {
        // Check cache first
        if let Some(entry) = self.cache.read().await.get(token) {
            if SystemTime::now() < entry.expires_at {
                return Ok(entry.user.clone());
            }
        }
        
        // Cache miss or expired - validate and cache
        let user = validator.await?;
        let entry = TokenCacheEntry {
            user: user.clone(),
            expires_at: SystemTime::now() + Duration::from_secs(300), // 5 min TTL
        };
        self.cache.write().await.insert(token.to_string(), entry);
        Ok(user)
    }
}

⚠️ CRITICAL: Token Cache Cleanup

DO NOT FORGET to implement cache entry cleanup to prevent memory leaks! Add:

  1. Background task to remove expired entries (every 5-10 minutes)
  2. Max cache size with LRU eviction policy
  3. Manual invalidation on token revocation events

Consider using moka or mini-moka crate for built-in TTL and LRU eviction.

Performance Impact

  • Reduces validation latency from ~100-500ms to <1ms (cache hit)
  • Reduces OIDC provider load by 95%+
  • Improves API response times significantly

Related Beads

  • scotty-42
  • scotty-41 (add cache hit/miss metrics)

Priority

Medium - performance optimization

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions