fix: evict prompt cache on NotFound error

[Betensis]

Problem

When a prompt label is removed in the Langfuse UI and the cached entry expires, get_prompt kept returning the stale prompt. The refresh worker logged a 404 NotFoundError but left the cache entry intact, so the SDK never fell back to another label or the unlabeled version. This bug is described in the issue linked below.

Technical details

• Catch NotFoundError in _fetch_prompt_and_update_cache, log a warning and delete the cache entry before re‑raising the error.
• Add PromptCache.delete(key) to remove a single cached entry and PromptCache.invalidate(prompt_name) to drop all entries for a prompt.
• Add test_evict_prompt_cache_entry_when_refresh_returns_not_found and update fixtures to initialise _resources with PromptCache.

Original issue: langfuse/langfuse#10138

---

Important

Fixes prompt cache eviction on NotFoundError by updating cache handling logic and adding tests for verification.

• Behavior:
  • In client.py, catch NotFoundError in _fetch_prompt_and_update_cache, log a warning, and delete the cache entry before re-raising the error.
  • Add PromptCache.delete(key) to remove a single cached entry and PromptCache.invalidate(prompt_name) to drop all entries for a prompt.
• Tests:
  • Add test_evict_prompt_cache_entry_when_refresh_returns_not_found in test_prompt.py to verify cache eviction on NotFoundError.
  • Update fixtures in test_prompt.py to initialize _resources with PromptCache.

^{This description was created by for 3c32bb0. You can customize this summary. It will automatically update as commits are pushed.}

Disclaimer: Experimental PR review

Greptile Summary

• Fixes stale prompt cache bug by evicting cache entries when background refresh receives 404 NotFoundError from API
• Adds PromptCache.delete() method to remove single cache entries when prompts are deleted in Langfuse UI

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The fix is well-targeted with proper exception handling, safe cache deletion logic using dict.pop(key, None), and comprehensive test coverage verifying the cache eviction flow
• No files require special attention

Important Files Changed

Filename	Overview
langfuse/_client/client.py	Added NotFoundError exception handling to evict cache entries when prompts are deleted from Langfuse UI
tests/test_prompt.py	Added comprehensive test for cache eviction on NotFoundError and updated fixture with _resources initialization

Sequence Diagram

sequenceDiagram
    participant User
    participant Langfuse as "Langfuse.get_prompt()"
    participant Cache as "PromptCache"
    participant Worker as "Background Worker"
    participant API as "Langfuse API"

    User->>Langfuse: "get_prompt(name)"
    Langfuse->>Cache: "get(cache_key)"
    Cache-->>Langfuse: "cached_prompt (expired)"
    Langfuse->>Cache: "add_refresh_prompt_task()"
    Langfuse-->>User: "return stale prompt"
    
    Cache->>Worker: "schedule refresh task"
    Worker->>Langfuse: "call _fetch_prompt_and_update_cache()"
    Langfuse->>API: "prompts.get(name, label)"
    API-->>Langfuse: "NotFoundError (404)"
    Langfuse->>Cache: "delete(cache_key)"
    
    User->>Langfuse: "get_prompt(name, fallback)"
    Langfuse->>Cache: "get(cache_key)"
    Cache-->>Langfuse: "None"
    Langfuse->>API: "prompts.get(name, label)"
    API-->>Langfuse: "NotFoundError (404)"
    Langfuse-->>User: "return fallback prompt"

Loading

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile
React with 👍 or 👎 to share your feedback on this new summary format}

[Copilot]

Pull Request Overview

This PR fixes a bug where deleted prompts remained in the cache indefinitely after expiration. When a prompt label is removed in Langfuse UI and the cached entry expires, the SDK now properly evicts the stale cache entry upon receiving a 404 NotFoundError, allowing fallback mechanisms to work correctly.

Key Changes:

• Added PromptCache.delete(key) method to remove individual cache entries when a prompt is not found
• Modified error handling in _fetch_prompt_and_update_cache to catch NotFoundError, log a warning, evict the cache entry, and re-raise the error
• Enhanced test fixtures to properly initialize _resources with PromptCache and added comprehensive test coverage for the eviction behavior

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
langfuse/_utils/prompt_cache.py	Adds delete method to safely remove individual cache entries using pop(key, None)
langfuse/_client/client.py	Adds NotFoundError exception handling to evict cache entries before re-raising the error
tests/test_prompt.py	Updates fixture to initialize PromptCache and adds comprehensive test verifying cache eviction on NotFoundError during background refresh

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.