💡 [IMPROVEMENT]: improve AIRTBench agent harness when caching is not enabled in the rigging chat pipeline

[GangGreenTemperTatum]

Implementation PR

NA atm

Reference Issues

ENG-2156

https://docs.dreadnode.io/open-source/rigging/topics/pipelines#defining-catchable-errors

AIRTBench-Code/airtbench/main.py

Lines 605 to 611 in 717ccf7

	pipeline: rg.ChatPipeline | None = (
	generator.wrap(backoff_wrapper)
	.chat(
	[{"role": "system", "content": system_prompt}, {"role": "user", "content": prompt}],
	)
	.cache("latest" if args.enable_cache else False)
	)

Summary

these kinds of errors prevent successful runs since we are not handling caching complications from upstream inference all too-well in the current harness

2025-06-10 08:41:01.432 | DEBUG    | callisto.kernel:interrupt:710 - Kernel 05967d2d-7881-4815-8164-ab776e203d9a interrupted
2025-06-10 08:41:01.432 | WARNING  | __main__:run_step:472 - |- Execution Timeout
08:41:01.433     Challenge voyager - step 76/100
08:41:01.434       Chat with litellm!claude-3-7-sonnet-20250219,timeout=240
2025-06-10 08:41:01.574 | WARNING  | __main__:run_step:291 - |- Chat failed: litellm.BadRequestError: AnthropicException - {"type":"error","error":{"type":"invalid_request_error","message":"messages.6.content.1.text: cache_control cannot be set for empty text blocks"}}
08:41:01.574     Challenge voyager - step 77/100
08:41:01.576       Chat with litellm!claude-3-7-sonnet-20250219,timeout=240
2025-06-10 08:41:05.587 | INFO     | __main__:run_step:348 - |- 1 executions, restart=False, give_up=False
2025-06-10 08:41:05.587 | INFO     | __main__:run_step:394 - |- Executing:

Basic Example

handle caching better when not available and perform some retry logic or catch statements

Drawbacks

NA

Unresolved questions

NA

[GangGreenTemperTatum]

closed by #40