-
Notifications
You must be signed in to change notification settings - Fork 217
fix: send hook_config to server in RemoteConversation #2115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 20 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
e286b4b
fix: send hook_config to server in RemoteConversation
xingyaoww 80ba8c6
test: add test for hook_config being sent to server
xingyaoww a790935
feat: Add HookExecutionEvent for hook observability
xingyaoww 8acdc2f
Update examples/02_remote_agent_server/01_convo_with_local_agent_serv…
xingyaoww 16caa43
Apply suggestion from @xingyaoww
xingyaoww a8eae38
feat: Add hook_config to ConversationState and SystemPromptEvent
xingyaoww 2444bd1
test: add stop hook and verify HookExecutionEvent for both hooks
openhands-agent 9990f4e
Merge branch 'main' into fix/remote-conversation-hook-config
xingyaoww ecacb88
fix: avoid circular import when rebuilding SystemPromptEvent
openhands-agent 008a9d7
fix: truncate HookExecutionEvent logs to 50k chars
openhands-agent 988b89b
docs: fix RemoteConversation PostToolUse hook comment
openhands-agent 071702b
test: update example to demonstrate on_stop hook with syntax validation
xingyaoww 69f37a7
test: fix agent instruction to demonstrate full on_stop hook cycle
xingyaoww fd8a2af
Merge main into fix/remote-conversation-hook-config
xingyaoww 32c781e
Add example run log after merge from main
xingyaoww 4c14b6e
Add JSON mode example run log
xingyaoww cfc7445
Rename pre_commit_check.sh to pycompile_check.sh
xingyaoww 70ebee0
Remove old log files from .pr folder
xingyaoww ce7f1e9
Merge branch 'main' into fix/remote-conversation-hook-config
xingyaoww e346ea2
fix: make all hooks server-only in RemoteConversation
openhands-agent ee3a7c0
refactor: remove hook_config from SystemPromptEvent
openhands-agent aa0129f
chore: Remove PR-only artifacts [automated]
e68d1ee
Merge branch 'main' into fix/remote-conversation-hook-config
xingyaoww File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # On-Stop Hook Demo Summary | ||
|
|
||
| ## Test Scenario | ||
|
|
||
| The example (`examples/02_remote_agent_server/01_convo_with_local_agent_server.py`) was modified to demonstrate the `on_stop` hook feature: | ||
|
|
||
| 1. **Hook Configuration**: A Stop hook is configured to run `pre_commit_check.sh`, which validates Python syntax in the workspace | ||
| 2. **Agent Instruction**: The agent is asked to create a Python file with a syntax error and then finish | ||
| 3. **Expected Flow**: | ||
| - Agent creates broken Python file | ||
| - Agent tries to finish | ||
| - Stop hook runs syntax check, finds error, returns `deny` with feedback | ||
| - Agent receives feedback and continues | ||
| - Agent attempts to fix the error | ||
| - Cycle repeats until successful or max retries | ||
|
|
||
| ## Evidence from Test Runs | ||
|
|
||
| ### Hook Configuration Sent to Server ✅ | ||
|
|
||
| From the logs: | ||
| ``` | ||
| ConversationStateUpdate(key=hook_config, value={'pre_tool_use': [], 'post_tool_use': [], 'user_prompt_submit': [], 'session_start': [], 'session_end': [], 'stop': [{'matcher': '*', 'hooks': [{'type': 'command', 'command': '/mnt/data/software-agent-sdk/examples/02_remote_agent_server/hook_scripts/pre_commit_check.sh', 'timeout': 60, 'async_': False}]}]}) | ||
| ``` | ||
|
|
||
| ### Stop Hook Denied Stopping ✅ | ||
|
|
||
| From server logs: | ||
| ``` | ||
| "Stop hook denied stopping: Blocked by hook" | ||
| "Stop hook denied agent stopping" | ||
| ``` | ||
|
|
||
| ### Feedback Sent to Agent ✅ | ||
|
|
||
| From client logs: | ||
| ``` | ||
| [Stop hook feedback] | ||
| SyntaxError: invalid syntax | ||
| ``` | ||
|
|
||
| ### Agent Continued Running After Denial ✅ | ||
|
|
||
| State transitions observed: | ||
| ``` | ||
| execution_status: running -> finished -> running -> finished -> running -> ... | ||
| ``` | ||
|
|
||
| This pattern repeated 5+ times, showing the hook successfully denying the stop and the agent continuing to work. | ||
|
|
||
| ## Key Findings | ||
|
|
||
| 1. **The `hook_config` IS being properly sent to the server** in the conversation creation payload | ||
| 2. **The Stop hook IS executing on the server side** when the agent tries to finish | ||
| 3. **The hook denial IS being communicated** back to the agent as feedback | ||
| 4. **The agent IS continuing to run** after receiving the denial feedback | ||
| 5. **The agent successfully fixes issues and completes** - the full cycle works end-to-end | ||
|
|
||
| ## Successful Test Run (17:29) | ||
|
|
||
| The full cycle was demonstrated: | ||
| 1. ✅ Agent created `test_broken.py` with syntax error | ||
| 2. ✅ Stop hook denied stopping ("Blocked by hook") | ||
| 3. ✅ Agent received feedback about the error | ||
| 4. ✅ Agent fixed the syntax error in `test_broken.py` | ||
| 5. ✅ Agent finished successfully | ||
|
|
||
| From the logs: | ||
| ``` | ||
| Create a Python file called 'test_broken.py' | ||
| [File test_broken.py edited with 1 changes.] | ||
| Stop hook denied stopping: Blocked by hook | ||
| I'll check and correct the syntax error in test_broken.py. | ||
| [File test_broken.py edited with 1 changes.] (fix applied) | ||
| finished! | ||
| ``` | ||
|
|
||
| ## Files Modified | ||
|
|
||
| - `examples/02_remote_agent_server/01_convo_with_local_agent_server.py` - Updated to use Stop hook | ||
| - `examples/02_remote_agent_server/hook_scripts/pre_commit_check.sh` - New script for syntax validation | ||
|
|
||
| ## Log Files | ||
|
|
||
| - `example_run_output.log` - First test run | ||
| - `example_run_output2.log` - Second test run | ||
| - `example_run_output3.log` - Third test run with retry logic | ||
| - `test_run_20260302_172931.log` - Final successful test run with complete cycle | ||
|
|
||
| ## Conclusion | ||
|
|
||
| The PR's fix to send `hook_config` to the server in RemoteConversation is working correctly. The Stop hook demonstrates the complete feedback loop where: | ||
| 1. Hooks run on the server | ||
| 2. Hook results (allow/deny) affect agent behavior | ||
| 3. Feedback from denied hooks is sent back to the agent | ||
| 4. The agent continues working to address the feedback | ||
| 5. **The agent successfully completes after fixing issues** ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
27 changes: 27 additions & 0 deletions
27
examples/02_remote_agent_server/hook_scripts/pycompile_check.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| #!/bin/bash | ||
| # Stop hook: Run Python syntax check on all .py files in the workspace | ||
| # Returns deny if any Python file has syntax errors, with the error output as feedback | ||
| # | ||
| # This hook validates that the agent hasn't broken any Python files. | ||
| # Environment variable CHECK_DIR can override the default working directory. | ||
|
|
||
| CHECK_DIR="${CHECK_DIR:-.}" | ||
|
|
||
| # Find all Python files and check for syntax errors | ||
| ERRORS="" | ||
| while IFS= read -r -d '' file; do | ||
| # Run python syntax check | ||
| result=$(python3 -m py_compile "$file" 2>&1) | ||
xingyaoww marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| if [ $? -ne 0 ]; then | ||
| ERRORS="${ERRORS}\n${result}" | ||
| fi | ||
| done < <(find "$CHECK_DIR" -name "*.py" -print0 2>/dev/null) | ||
|
|
||
| if [ -n "$ERRORS" ]; then | ||
| # Escape the output for JSON | ||
| ESCAPED_OUTPUT=$(echo -e "$ERRORS" | head -50 | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()))') | ||
| echo "{\"decision\": \"deny\", \"additionalContext\": $ESCAPED_OUTPUT}" | ||
| exit 2 | ||
| fi | ||
|
|
||
| exit 0 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Suggestion: This example has grown significantly in complexity. The original showed basic RemoteConversation usage (~50 lines). Now it's a comprehensive hook testing harness (~350 lines) that deliberately creates broken Python files to test stop hook denial and retry logic.
For a "01_" introductory example, consider splitting this into:
01_convo_with_local_agent_server.py- basic usage (~50 lines)02_hooks_with_stop_validation.py- comprehensive hook testing (current content)The comprehensive demonstration is valuable for showing hook capabilities, but might be better as a separate, clearly-labeled advanced example.
Not blocking - this is an organizational suggestion to improve discoverability for users learning the basics.