Stamp last_active in streaming agent loop to prevent heartbeat false-positives#1090
Open
chrisyoung2005 wants to merge 1 commit intoRightNow-AI:mainfrom
Open
Stamp last_active in streaming agent loop to prevent heartbeat false-positives#1090chrisyoung2005 wants to merge 1 commit intoRightNow-AI:mainfrom
chrisyoung2005 wants to merge 1 commit intoRightNow-AI:mainfrom
Conversation
…positives Fixes RightNow-AI#1089 run_agent_loop_streaming skipped the touch_agent() call that the non-streaming run_agent_loop performs before every LLM request. On slow local inference (e.g. Ollama qwen3.5:35b, multi-minute generations), last_active went stale and the heartbeat monitor flagged the agent as unresponsive, triggering crash recovery mid-stream. With multiple agents sharing one Ollama instance, queued agents appeared frozen while the active one generated. Mirror the non-streaming behavior: stamp last_active immediately before stream_with_retry so the heartbeat window covers the full LLM call. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Author
|
Note on failing checks: Format, Clippy, and Security Audit are pre-existing failures on None of the diffs flagged by
The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1089
What
run_agent_loop_streamingwas missing thetouch_agent()call that the non-streamingrun_agent_loopperforms before every LLM request. This causeslast_activeto go stale during slow streaming generations, and the heartbeat monitor flags the agent as unresponsive mid-stream — triggering a crash-recovery cycle.Why
With a slow local backend (e.g. Ollama
qwen3.5:35bgenerating for minutes, especially under contention from multiple agents sharing one Ollama instance), the agent appears "frozen" to the user. Under the hood, the kernel has already marked it unresponsive, killed the loop, and is restarting it — which re-queues the request, making the problem worse when multiple agents pile on.The non-streaming path handles this correctly at
crates/openfang-runtime/src/agent_loop.rs:446-449:The streaming path did not have the equivalent, so
last_activewas only updated between iterations (after streaming finished), not before the long-running call.Fix
Mirror the non-streaming behavior in
run_agent_loop_streaming, immediately beforestream_with_retry:Minimal 6-line change, no behavior change for agents that fit inside the heartbeat window.
agent_id_stris already in scope at this point in the function.Verification
cargo fmt -p openfang-runtime -- --check— cleancargo clippy -p openfang-runtime --all-targets -- -D warnings— cleancargo test -p openfang-runtime— 929 passed, 0 failedRepro path on local Ollama: with
heartbeat.default_timeout_secsbelow actual per-iteration generation time, streaming agents get killed mid-response and re-spawned in a loop. With this patch applied, the same config runs to completion.🤖 Generated with Claude Code