fix: add tool count validation to prevent API errors when enabling many functions

[theonlypal]

This PR fixes the 500 Internal Server Error when enabling all Gmail functions in the playground.

What changed

• Added MAX_TOOLS = 64 constant based on empirical OpenAI API limits
• Added early validation in /chat route returning HTTP 400 with actionable message
• Added defensive check in openai_chat_stream with warning log

Why it failed before

OpenAI's API has undocumented limits on tool count and total token size of tool definitions. When users enable all functions for an app (Gmail has 24 functions), the request exceeds these limits and returns a cryptic "There was an issue with your request" error that surfaces as a 500.

Why this fix works

Validates tool count before making the API call. Returns a 400 with a clear message telling users to reduce selected functions. The limit of 64 is conservative - actual limit is around 128 or ~200k tokens of tool definitions.

Risk assessment

• Low risk: validation-only change
• No behavioral change for requests under the limit
• Existing requests that would fail now fail earlier with a useful message

Closes #337

---

To make this production-safe, the remaining work is:

• Implement intelligent tool selection/prioritization when limit exceeded
• Add UI feedback showing tool count approaching limit
• Consider chunked/multi-turn approaches for complex tool sets
• Make MAX_TOOLS configurable via environment variable
• Add integration tests for limit boundary conditions

This is ~3-5 days.
I can own this as a short paid engagement; otherwise this PR stands on its own.

---

Summary by cubic

Validate tool count to prevent cryptic OpenAI errors when too many functions are enabled. Adds a conservative MAX_TOOLS=64 and returns a clear 400 with guidance; addresses #337.

• Bug Fixes
  • Early check in /chat: 400 with message if tools > MAX_TOOLS.
  • Defensive validation and warning in openai_chat_stream.
  • Avoids failed API calls when enabling many Gmail functions or apps.

^{Written for commit 2487903. Summary will update on new commits.}

Summary by CodeRabbit

• Bug Fixes
  • Added validation to enforce a maximum limit of 64 tools per request. Requests exceeding this limit will return a clear error message with details.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[vercel]

Someone is attempting to deploy a commit to the Proxy Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a conservative tool usage limit (MAX_TOOLS = 64) to prevent OpenAI API failures when excessive tools are selected. Validation is implemented at the route handler level (returns 400 HTTPException) and the prompt module level (raises ValueError with logging) to catch the issue early and return appropriate error responses.

Changes

Cohort / File(s)	Summary
Tool Limit Enforcement backend/aci/server/agent/prompt.py	Adds MAX_TOOLS constant (64), implements validation in openai_chat_stream to check tool count and raise ValueError if exceeded, updates function docstring
Route-Level Guard backend/aci/server/routes/agent.py	Imports HTTPException and MAX_TOOLS, adds runtime check in handle_chat to raise 400 HTTPException with descriptive message before constructing StreamingResponse

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 A limit most wise, the tools now confined,
Sixty-four is the number we've carefully lined,
No more shall the API error and crash,
Early guards catch the tools with a dash! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding tool count validation to prevent API errors from enabling many functions.
Linked Issues check	✅ Passed	The PR directly addresses issue #337 by implementing tool count validation with a MAX_TOOLS limit to prevent the openai.APIError that occurs when enabling many Gmail functions.
Out of Scope Changes check	✅ Passed	All changes are scoped to tool count validation: introducing MAX_TOOLS constant, adding validation in routes and stream functions, and updating documentation as needed.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6e1e651 and 2487903.

📒 Files selected for processing (2)

• backend/aci/server/agent/prompt.py
• backend/aci/server/routes/agent.py

🧰 Additional context used

📓 Path-based instructions (1)

backend/**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

backend/**/*.py: Backend Python code must be compatible with Python 3.12+
Backend must use ruff for formatting and linting
Backend must use mypy for type checking

Files:

• backend/aci/server/agent/prompt.py
• backend/aci/server/routes/agent.py

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: cubic · AI code reviewer

🔇 Additional comments (3)

backend/aci/server/routes/agent.py (1)

66-71: LGTM! Early validation with actionable error message.

The validation correctly guards against exceeding OpenAI's tool limits before initiating the stream. Returning HTTP 400 with a descriptive message is appropriate for input validation failures.

backend/aci/server/agent/prompt.py (2)

15-18: Well-documented constant with clear rationale.

The comment helpfully explains the empirical basis for the limit and the conservative approach taken.

---

81-88: Defensive check is appropriate, but note the async generator context.

The validation is a good safety net for direct callers of this function. However, since this is an async generator, the check executes when streaming begins (not when StreamingResponse is instantiated). If this code path were reached, the ValueError would propagate during streaming and result in an unhandled exception.

This is acceptable because the route handler in agent.py validates first and prevents this scenario. The warning log appropriately signals this should-never-happen condition.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

No issues found across 2 files