Skip to content

Conversation

@GangGreenTemperTatum
Copy link
Collaborator

@GangGreenTemperTatum GangGreenTemperTatum commented Jul 2, 2025

add cache retry logic for unsupported providers

#20

when caching is enabled but the LLM provider doesn't support it, detect cache_control errors and automatically retry without caching instead of failing the entire challenge run

Key Changes:

  • add cache retry logic for unsupported providers

Added:

  • add cache retry logic for unsupported providers

Generated Summary:

  • Added new error handling in the chat execution flow to detect caching-related errors when "cache_control" appears in the error message.
  • Integrated logic to disable caching (setting cache to False) and retry the chat request using a new pipeline.
  • Implemented specific exception handling for various API errors (InternalServerError, BadRequestError, Timeout, ServiceUnavailableError, APIConnectionError) during the retry.
  • Included logging and metric tracking (e.g., "cache_unsupported", "failed_chats") to monitor caching issues and retry outcomes.

This summary was generated with ❤️ by rigging

@GangGreenTemperTatum GangGreenTemperTatum merged commit fface44 into main Jul 2, 2025
4 checks passed
@GangGreenTemperTatum GangGreenTemperTatum deleted the ads/eng-2222-chore-improve-airtbench-agent-harness-when-caching-is-not branch July 2, 2025 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants