feat: implement parallel tool execution for better performance#352
feat: implement parallel tool execution for better performance#352cleyesode wants to merge 3 commits intoNano-Collective:mainfrom
Conversation
|
Hey @cleyesode! Thanks for working on this! Parallel tool execution is a great idea for performance :D I have some thoughts about the current approach that I think need to be addressed before merging - let me know what you think? Race conditions from unconditional parallelism The main issue is that all tool calls are parallelised without considering dependencies between them. LLMs frequently issue tool calls with implicit ordering - e.g., If we want parallel execution, we'd need to either analyse tool dependencies (e.g., only parallelise read-only tools on different paths) or let the caller opt in for known-safe batches like Fire-and-forget Changing Non-deterministic conversation state
Smaller items:
I'd suggest either scoping this to only parallelise provably-independent tools, or keeping sequential execution as the default with an opt-in parallel path for safe combinations. Happy to discuss approaches! :D |
|
Understood, I may have rushed. The implementation seeks to resolve additional missing tool errors encountered within my own niche use-cases for contemporaneous bash. More testing needed. Good to have excellent code owner to tell me when to pump brakes. I'll circle back on this. |
Hey, no problem :D this is genuinely a good improvement, just with some edge-cases to think about! Hopefully this is something we can still do! |
|
Hey @will-lamerton , just dropping in to keep this idea alive. I will work through this properly in the coming week. As a 100% local LLM, resolving tool calling function errors which may sometimes get overlooked by ai sdk users is my top priority. This PR is on hold while I retool xml parser and introduce response-formatter and streaming. Stay tuned |
Summary
Successfully implemented parallel tool execution for the Nanocoder agent. The changes enable multiple git tools (git_status, git_log, git_diff) to execute simultaneously instead of sequentially.
Key Changes
1. Parallel Execution Architecture
2. Performance Impact
3. Test Coverage
Usage Example
The agent can now call multiple git tools in a single response:
```json
{
"tool_calls": [
{"name": "git_status"},
{"name": "git_log"},
{"name": "git_diff"}
]
}
```
All three tools will execute simultaneously, returning results faster than before.
Implementation Details
The implementation maintains full backward compatibility and all existing error handling/validation logic. Tools that require confirmation (`needsApproval: true`) are never affected by parallel execution - they still require user approval before running.
Files Changed