proposal: make Max Iteration Limit visible to the LLM and have a Graceful Termination Message

[VascoSch92]

Summary

The max_iterations parameter acts as a hard server-side ceiling on the number of LLM calls per conversation, but:

1. The LLM is never informed of the limit. The system prompt contains zero mention of iteration budgets, step limits, or remaining turns. The agent has no way to plan its work within the budget.
2. When the limit is (or would be) hit, there is no graceful degradation. The agent doesn't receive a warning as it approaches the ceiling, and there is no mechanism to prompt it to wrap up and provide a best-effort answer before being cut off.

What the code does when the limit is hit

From local_conversation.py:

1. max_iterations_per_run is set on ConversationState (default 500) and tracked at line 594 as a local iteration counter.
2. The run loop (lines 596-689) increments iteration after each agent.step() call and checks if iteration >= self.max_iteration_per_run.
3. When the limit is reached, it hard-stops with an ERROR status and emits a ConversationErrorEvent with a MaxIterationsReached error.
4. The iteration count is never passed to the agent — not in agent.step(), not in the system prompt, not in the context. The agent has zero visibility into how many steps it has left.
5. There is no warning injection as the agent approaches the limit — no "you have N iterations remaining" message.

The result: the conversation abruptly terminates with an error event. Any work the agent has done but not yet synthesized into a final answer is lost. The instance scores as failed.

The Problem

Without budget awareness, the agent can't prioritize

If the agent knew it had N steps remaining, it could:

• Plan ahead: tackle the most critical parts first
• Wrap up early: provide a best-effort answer when time is running low
• Avoid rabbit holes: skip expensive exploration paths when budget is tight
• Manage subagent delegation: avoid spawning subagents late in the budget (subagent calls with max_iteration_per_run: null have no limit either, risking budget exhaustion)

Without this information, an agent at step 498/500 will happily start a 50-step research task and get killed mid-way.

Without a graceful termination message, answers are lost

When the limit is hit:

• The run loop in local_conversation.py fires a ConversationErrorEvent with MaxIterationsReached and sets status to ERROR
• The LLM never gets a chance to synthesize its findings into a final answer
• All the work done up to that point is wasted — the instance scores as a hard error (not even "incorrect" — just failed)
• This is especially costly for tasks where the agent has gathered the right data but hasn't yet formatted the answer

The error type is indistinguishable from real errors

MaxIterationsReached produces the same ERROR status as infrastructure failures, OOM crashes, or API errors. Downstream reporting (e.g., output_errors.jsonl, ERROR_LOGS.txt) treats iteration exhaustion identically to a crash. This makes it hard to:

• Distinguish "ran out of budget" from "something broke"
• Measure how often the limit is the bottleneck
• Tune max_iterations based on data

Proposed Solutions

Option A: Communicate the budget to the LLM (recommended)

Add iteration budget info to the system prompt and/or inject warnings as the agent approaches the limit.

In the system prompt:

You have a budget of {max_iterations} steps for this task.
Plan your approach to complete the task within this budget.

Warning injection at ~80% and ~95% budget:

[SYSTEM] You have used {current}/{max_iterations} steps.
{remaining} steps remaining. Begin wrapping up and provide your best answer.

Option B: Graceful termination message (minimum fix)

If we don't want to expose the budget to the LLM (to avoid gaming or conservative behavior), at least inject a final message when the limit is about to be hit:

[SYSTEM] You are about to reach the maximum number of steps allowed.
This is your FINAL step. Provide your best answer NOW based on everything
you have gathered so far.

This gives the agent one last turn to produce an answer instead of being silently killed.

Option C: Return a distinct status instead of ERROR (minimum observability fix)

Instead of emitting a generic ConversationErrorEvent with ERROR status, return a dedicated status like MAX_ITERATIONS_REACHED or BUDGET_EXHAUSTED. This would:

• Let eval harnesses distinguish "ran out of steps" from real crashes
• Enable data-driven tuning of max_iterations
• Allow retry logic to treat budget exhaustion differently (e.g., retry with higher limit vs. retry with same config)

In local_conversation.py, this could be as simple as:

# Instead of:
raise ConversationError("MaxIterationsReached")

# Use a distinct status:
self.state.status = ConversationStatus.ITERATION_LIMIT
return ConversationIterationLimitEvent(
    iteration=iteration,
    max_iterations=self.max_iteration_per_run,
    last_agent_action=last_action,  # preserve what the agent was doing
)

Option D: All of the above (ideal)

• Light budget awareness in the system prompt (without exact numbers): "You have a limited number of steps. Work efficiently and be prepared to provide a best-effort answer if prompted."
• Hard warning injection at 90-95% budget with exact remaining steps
• Final-chance message at the last step
• Distinct ITERATION_LIMIT status for observability

Additional Consideration: Subagent Iteration Limits

The max_iteration_per_run field for subagents is currently null (unlimited). This means a subagent could theoretically consume the entire remaining budget of the parent conversation. Consider:

• Setting a default max_iteration_per_run for subagents (e.g., 20-50 steps)
• Or deducting subagent steps from the parent's budget

Impact

Helpful for:

• Harder benchmarks (SWE-bench, complex GAIA Level 3 tasks)
• Lower max_iterations settings used for cost control
• Tasks that involve extensive web research or multi-step reasoning
• Heavy subagent usage patterns

The fix is low-effort (a few lines in the system prompt + a budget check before each LLM call) but high-impact — it turns a silent failure into a recoverable situation.

[enyst]

Re: Option C

Let eval harnesses distinguish "ran out of steps" from real crashes

This should definitely be the case, IMHO. Isn't there a way now? We used to distinguish and had some reporting for how many instances ran out of iterations.

---

Note: afaik codex-cli doesn't have a max iterations. It goes on until it decides to answer the user and beyond.

It might be worth thinking about... 🤔 There could be issues with cost and with instances running forever -ish, but maybe we could consider simply making it 1k...

---

Re: the other options

Just to note, we used to have a reminder message to the LLM with remaining iterations.

I'm not sure this is a system prompt issue, the reminder was appended to the last user message / observation.

[enyst]

Maybe worth noting, whatever behavior we'd like for evals, for the user in normal agent use I do think the expected behavior is not to have a reminder, and is not to stop completely: instead, a new user message would clear the error and the agent continues for another round.

[all-hands-bot]

[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you please take a look and add the appropriate priority label when you have a chance?

[VascoSch92]

Maybe worth noting, whatever behavior we'd like for evals, for the user in normal agent use I do think the expected behavior is not to have a reminder, and is not to stop completely: instead, a new user message would clear the error and the agent continues for another round.

This is a good point. However, it could be nice to have an appropriate error for this kind of situations .Because now we return just a generic one.

[VascoSch92]

I just wanted to point out a situation that I encountered today with an eval, precisely with Commit0

These are the results:

Instance	Pass Rate	Passed / Total	End Status
deprecated	100.0%	171 / 171	finished_no_finish_tool
tinydb	100.0%	201 / 201	status=running
voluptuous	100.0%	149 / 149	finish_tool
wcwidth	94.9%	37 / 39	finished_no_finish_tool
portalocker	75.0%	30 / 40	status=running
pyjwt	74.0%	194 / 262	status=running
minitorch	57.1%	84 / 147	finish_tool
cookiecutter	47.2%	175 / 371	finish_tool
marshmallow	37.8%	465 / 1229	status=running
imapclient	36.3%	97 / 267	status=running
chardet	20.2%	77 / 381	finish_tool
babel	7.5%	44 / 589	finished_no_finish_tool
jinja	1.3%	10 / 800	status=running
cachetools	FAILED	—	eval harness error
parsel	FAILED	—	eval harness error
simpy	FAILED	—	eval harness error

The following instances ended with status=running, meaning the agent exhausted all 500 iterations without calling the finish tool:

• imapclient, jinja, marshmallow, portalocker, pyjwt, tinydb

Notably, tinydb achieved 100% pass rate but still didn't finish cleanly — the agent kept working despite all tests already passing.

Moreover, jinja (1.3%) — Only 10 out of 800 tests passed. The agent was still running when it hit the iteration limit, suggesting it could not make meaningful progress on this large codebase.

[enyst]

@VascoSch92 The End Status here is from the SDK side, that is, the ConversationErrorEvent or so, or is it from the agent-server's REST or maybe from a WebSocket event?

@OpenHands Try to find the answer to this question, look at the run-eval workflows in this repo, also clone yourself the benchmarks/ repo from the same org, and look how results are received and processed from commit0 instances.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[VascoSch92]

@enyst your agent never returned

[enyst]

The End Status here is from the SDK side, that is, the ConversationErrorEvent or so, or is it from the agent-server's REST or maybe from a WebSocket event?

@OpenHands Try to find the answer to this question, look at the run-eval workflows in this repo, also clone yourself the benchmarks/ repo from the same org, and look how results are received and processed from commit0 instances.