fix: Limit max response middleware body size with get_body_limit() instead of usize::MAX

[michaelfeil]

Signed-off-by: michaelfeil me@michaelfeil.eu

Overview:

I am hunting down a memory bug. This is super uncritical, but using usize::max here is not appropriate for error messages.

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

Summary by CodeRabbit

• Bug Fixes
  • Improved error response handling by implementing configurable limits on HTTP response body sizes, enhancing system stability and performance when processing error responses.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi michaelfeil! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

Walkthrough

The pull request modifies HTTP response body limit handling in the OpenAI service middleware. Specifically, it replaces an unlimited body size constraint (usize::MAX) with a configurable limit obtained from get_body_limit() when reading error responses from HTTP 422 status codes.

Changes

Cohort / File(s)	Summary
HTTP Response Body Limit Configuration lib/llm/src/http/service/openai.rs	Replaced unlimited body size limit (usize::MAX) with configurable limit from get_body_limit() for HTTP 422 error response handling

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 A single line hops into place,
From infinite bounds to config grace,
The error response finds its true measure,
No longer unbounded—a sensible treasure!

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The description covers the overview and context (memory bug investigation), but lacks specific details about which files were changed and doesn't properly fill the template's 'Details' and 'Where should the reviewer start' sections.	Complete the 'Details' section with specific changes made, fill 'Where should the reviewer start' to guide reviewers to openai.rs, and replace the placeholder issue reference with an actual GitHub issue number.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The pull request title accurately describes the main change: replacing usize::MAX with a configurable get_body_limit() for constraining the body size limit when reading error responses in the HTTP service middleware.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[rmccorm4]

LGTM, thanks for the contribution @michaelfeil !

[rmccorm4]

/ok to test e5e615e