🔥 feat: Add StreamResponseBody support for the Client #3711
Conversation
|
Thanks for opening this pull request! 🎉 Please check out our contributing guidelines. If you need help or want to chat with us, join us on Discord https://gofiber.io/discord |
WalkthroughAdds response-body streaming support across the client stack: new Client StreamResponseBody/SetStreamResponseBody, Response.BodyStream()/IsStreaming, transport-layer getters/setters and propagation for standard/host/lb transports, Save() streaming paths, request BodyStream forwarding, core response ownership fixes, tests, and docs. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Client
participant Transport
participant Core
participant fasthttp as "fasthttp.Client"
participant Response
User->>Client: SetStreamResponseBody(true)
Client->>Transport: SetStreamResponseBody(true)
Transport->>fasthttp: set internal streaming flag(s)
User->>Client: Do(request)
Client->>Core: executeRequest(...)
Core->>fasthttp: Do()/DoDeadline(request)
fasthttp-->>Core: *fasthttp.Response
alt request contains BodyStream
Core->>fasthttp: SetBodyStream(contentLen, reader)
end
Core-->>Client: build Response (wrap RawResponse)
User->>Response: IsStreaming()
Response-->>User: true/false
alt streaming response
User->>Response: BodyStream()
Response-->>User: io.Reader (stream)
User->>Response: Save(dest)
Response->>Response: io.Copy(BodyStream -> dest)
else non-streaming response
User->>Response: Body()
Response-->>User: []byte
User->>Response: Save(dest)
Response->>Response: io.Copy(bytes.Reader -> dest)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @Abhirup-99, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a significant enhancement to the client by adding support for streaming HTTP response bodies. This allows consumers to process large responses or server-sent events incrementally, preventing excessive memory consumption. The changes include new configuration options at both the client and individual request levels, along with the necessary logic to facilitate this streaming behavior and comprehensive test coverage.
Highlights
- Client-level Streaming Configuration: Introduced a streamResponseBody field and corresponding SetStreamResponseBody and StreamResponseBody methods to the Client struct, enabling global control over response body streaming.
- Request-level Streaming Override: Added a streamResponseBody field and methods to the Request struct, allowing per-request configuration that can override the client's default streaming setting.
- Dynamic Streaming Execution: Modified the core request execution logic to dynamically set and restore the underlying HTTP client's streaming option based on the active request or client configuration.
- Streamed Body Access: Implemented a BodyStream() method on the Response struct, providing an io.Reader interface for efficient, stream-based consumption of response bodies.
- Comprehensive Test Coverage: Added extensive unit tests to validate the new streaming functionality, covering scenarios like server-sent events, large responses, default settings, overrides, and method chaining.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
I will update the documentation if the approach seems sound enough. |
There was a problem hiding this comment.
Pull Request Overview
This pull request adds support for streaming HTTP response bodies in the client library, enabling responses to be read as streams instead of being fully loaded into memory. This is particularly useful for handling large responses or server-sent events.
- Added streaming configuration at both client and request levels with override capability
- Implemented
BodyStream()method on Response struct to provide streaming access - Added comprehensive test coverage for various streaming scenarios
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| client/client.go | Adds streamResponseBody field and configuration methods to Client struct |
| client/request.go | Adds per-request streaming configuration with client-level fallback |
| client/core.go | Updates request execution to handle streaming option setting and restoration |
| client/response.go | Implements BodyStream method for streaming response body access |
| client/client_test.go | Comprehensive tests for client-level streaming configuration and behavior |
| client/response_test.go | Tests for response streaming functionality including large data handling |
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
gaby
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (4)
client/request.go (1)
594-605: Effective value resolution is correct and robustFalls back to the client setting when unset; default false if no client is attached yet. Consider documenting this precedence in the method comment for discoverability.
client/client_test.go (3)
1750-1767: Optional: make SSE handler actually stream (flush between events) for stronger signalRight now the handler writes three messages and returns. This can still pass even if the server buffers and sends a single chunk. Consider using SetBodyStreamWriter and flushing between writes to exercise true streaming behavior. Example:
app.Get("/sse", func(c fiber.Ctx) error { c.Set("Content-Type", "text/event-stream") c.Set("Cache-Control", "no-cache") c.Set("Connection", "keep-alive") c.Context().SetBodyStreamWriter(func(w *bufio.Writer) { msgs := []string{"data: message 1\n\n", "data: message 2\n\n", "data: message 3\n\n"} for _, m := range msgs { _, _ = w.WriteString(m) _ = w.Flush() time.Sleep(10 * time.Millisecond) } }) return nil })This reduces false positives where streaming isn’t actually occurring.
1836-1841: Strengthen fallback contract when streaming is disabled by reading BodyStreamYou assert BodyStream() is non-nil, but don’t verify it yields the same content as Body() in the fallback path. Read it and compare.
Apply this diff:
body := resp.Body() require.Equal(t, "Hello, World!", string(body)) bodyStream := resp.BodyStream() require.NotNil(t, bodyStream) + streamed, err := io.ReadAll(bodyStream) + require.NoError(t, err) + require.Equal(t, body, streamed)
1854-1883: Please add a follow-up request and Reset() behavior test to guard against per-request overrides leakingYou’ve already verified the getter/inheritance logic for StreamResponseBody, but we’re still missing two important checks in client/client_test.go around lines 1854–1883:
- A second request on the same client after an override, to ensure the per-request override doesn’t persist.
- Exercising
Request.Reset()to confirm it clears any per-request streaming flag and falls back to the client default.Suggested diff to insert after the existing assertions:
req2 := client.R().SetStreamResponseBody(false) require.False(t, req2.StreamResponseBody()) + // Follow-up call: ensure the override didn’t leak into actual behavior. + resp2b, err := req2.Get("http://" + addr + "/test") + require.NoError(t, err) + defer resp2b.Close() + require.Equal(t, "Hello, World!", string(resp2b.Body())) clientWithStreaming := New().SetStreamResponseBody(true) req3 := clientWithStreaming.R() require.True(t, req3.StreamResponseBody()) // Should inherit from client req4 := client.R(). SetStreamResponseBody(true). SetTimeout(time.Second * 5). SetStreamResponseBody(false) require.False(t, req4.StreamResponseBody()) + + // After Reset, per-request override should be cleared and revert to client setting (false). + reqReset := client.R().SetStreamResponseBody(true) + reqReset.Reset() + require.False(t, reqReset.StreamResponseBody())This will catch regressions where the underlying fasthttp flag isn’t properly restored and confirm Reset() clears the override.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (6)
client/client.go(2 hunks)client/client_test.go(1 hunks)client/core.go(1 hunks)client/request.go(3 hunks)client/response.go(1 hunks)client/response_test.go(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
client/response_test.go (1)
client/client.go (2)
Get(706-708)New(647-652)
client/client_test.go (1)
client/client.go (2)
New(647-652)Get(706-708)
🔇 Additional comments (6)
client/request.go (3)
70-72: Good choice: tri-state per-request overrideUsing a pointer to bool for streamResponseBody cleanly distinguishes unset from explicit true/false. LGTM.
606-612: Chainable setter is fineNo issues.
679-679: Reset clears the override as expectedResetting streamResponseBody to nil ensures subsequent requests inherit the client default. LGTM.
client/client_test.go (3)
1737-1745: LGTM: basic client-level toggle coverage is solidCovers default false, enable, disable. Clear, fast, and deterministic.
1793-1819: LGTM: large-response streaming vs non-streaming parityGood end-to-end validation that BodyStream() delivers identical data to Body() for large payloads, with streaming enabled/disabled. Using 1 MiB is a reasonable balance for CI.
1846-1852: LGTM: chainable method precedenceConfirms last-set-wins across chained setters with unrelated setters in between. Clear and sufficient.
|
@Abhirup-99 The tests are failing. You can run them locally by running |
Abhirup-99
force-pushed
the
add-response
branch
from
a725b97 to
9c12637
Compare
|
@gaby fixed the failing tests |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3711 +/- ##
==========================================
+ Coverage 91.67% 91.76% +0.09%
==========================================
Files 119 119
Lines 10177 10242 +65
==========================================
+ Hits 9330 9399 +69
+ Misses 536 534 -2
+ Partials 311 309 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (3)
client/client.go (2)
538-557: Reset should also clear streamResponseBody to default falseClient.Reset clears most flags but leaves streamResponseBody unchanged, leading to surprising behavior post-reset.
func (c *Client) Reset() { c.fasthttp = &fasthttp.Client{} c.baseURL = "" c.timeout = 0 c.userAgent = "" c.referer = "" c.retryConfig = nil c.debug = false + c.streamResponseBody = false
58-61: Guard temporary per-request flips of fasthttp.Client.StreamResponseBody with a dedicated mutex on ClientIf core executes by toggling the shared fasthttp.Client.StreamResponseBody between requests, concurrent requests can race. Add a small private mutex dedicated to this toggle. This keeps the public API unchanged and avoids overloading c.mu, which guards other state.
Apply:
type Client struct { logger log.CommonLogger fasthttp *fasthttp.Client @@ - timeout time.Duration - mu sync.RWMutex - debug bool - streamResponseBody bool + timeout time.Duration + mu sync.RWMutex + debug bool + streamResponseBody bool + // protects temporary flips of fasthttp.Client.StreamResponseBody during per-request execution + streamRespMu sync.Mutex }To confirm whether the toggle approach is used (vs. cloning a temp fasthttp.Client), run:
#!/bin/bash # Inspect how StreamResponseBody is applied during execution. rg -n -C2 --type=go 'StreamResponseBody' client/core.go client | sed -n '1,200p'client/response_test.go (1)
598-611: Fix flakiness: copy Body() before closing resp1normalBody aliases resp1’s internal buffer; after Close(), pooled buffers may be reused, causing nondeterministic failures. Copy the bytes before closing.
- defer resp1.Close() - normalBody := resp1.Body() + defer resp1.Close() + normalBody := append([]byte(nil), resp1.Body()...)
🧹 Nitpick comments (2)
client/client.go (2)
439-451: API shape LGTM; document precedence explicitly in the commentThe getter/setter are clear and chainable. Add one line noting that a request-level setting overrides the client-level default for discoverability.
// SetStreamResponseBody enables or disables response body streaming. // When enabled, the response body can be read as a stream using BodyStream() // instead of being fully loaded into memory. This is useful for large responses -// or server-sent events. +// or server-sent events. Note: a per-request setting (Request.SetStreamResponseBody) +// overrides this client-level default. func (c *Client) SetStreamResponseBody(enable bool) *Client { c.streamResponseBody = enable return c }
563-575: Optional: plumb StreamResponseBody through Config for convenienceMany helpers accept Config; adding an optional pointer makes it easy to set streaming per call without constructing a Request manually.
type Config struct { Ctx context.Context //nolint:containedctx // It's needed to be stored in the config. Body any Header map[string]string Param map[string]string Cookie map[string]string PathParam map[string]string FormData map[string]string UserAgent string Referer string File []*File Timeout time.Duration MaxRedirects int + // Optional per-request override; when set, overrides the client default. + StreamResponseBody *bool } @@ if cfg.MaxRedirects != 0 { req.SetMaxRedirects(cfg.MaxRedirects) } + if cfg.StreamResponseBody != nil { + req.SetStreamResponseBody(*cfg.StreamResponseBody) + } + if cfg.Body != nil { req.SetJSON(cfg.Body) return }Also applies to: 577-635
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (6)
client/client.go(2 hunks)client/client_test.go(1 hunks)client/core.go(2 hunks)client/request.go(3 hunks)client/response.go(1 hunks)client/response_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- client/response.go
- client/client_test.go
- client/core.go
🔇 Additional comments (5)
client/response_test.go (1)
542-596: Solid coverage for basic and large streaming pathsThe test server and the two subtests validate BodyStream() on small and larger responses. Clear and deterministic.
client/request.go (4)
70-72: *Good choice using bool to represent tri-state (unset vs true/false)This enables request-level override while preserving client default. No action needed.
594-605: Precedence and fallback logic reads correctlyReturns request override when set; otherwise falls back to client-level; defaults to false if no client. Matches the intended design.
606-612: Setter is fine; pointer escape is safeUsing &enable is idiomatic (escapes to heap). Chainable API is consistent with others.
679-679: Reset correctly clears the per-request overrideResetting streamResponseBody to nil ensures the request returns to default behavior. Good.
Abhirup-99
force-pushed
the
add-response
branch
from
9c12637 to
4d32c17
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
client/core.go (1)
132-141: Minor duplication: unify Do vs DoRedirects callsitesThe four callsites using
fastHTTPClient.Do/DoRedirectsunder retry and non-retry paths duplicate logic. A tiny helper makes it harder to regress:- if cfg != nil { - err = retry.NewExponentialBackoff(*cfg).Retry(func() error { - if shouldRedirect { - return fastHTTPClient.DoRedirects(reqv, respv, c.req.maxRedirects) - } - return fastHTTPClient.Do(reqv, respv) - }) - } else { - if shouldRedirect { - err = fastHTTPClient.DoRedirects(reqv, respv, c.req.maxRedirects) - } else { - err = fastHTTPClient.Do(reqv, respv) - } - } +call := func() error { + if shouldRedirect { + return fastHTTPClient.DoRedirects(reqv, respv, c.req.maxRedirects) + } + return fastHTTPClient.Do(reqv, respv) +} +if cfg != nil { + err = retry.NewExponentialBackoff(*cfg).Retry(call) +} else { + err = call() +}Low impact, but improves readability.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (6)
client/client.go(2 hunks)client/client_test.go(1 hunks)client/core.go(2 hunks)client/request.go(3 hunks)client/response.go(1 hunks)client/response_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
- client/response.go
- client/client.go
- client/request.go
- client/response_test.go
- client/client_test.go
🧰 Additional context used
🧬 Code graph analysis (1)
client/core.go (1)
client/client.go (1)
Client(32-62)
🔇 Additional comments (3)
client/core.go (3)
84-118: Kudos: avoids flipping shared fasthttp.Client.StreamResponseBodyNot mutating the shared client’s flag sidesteps the prior data race concerns. Good move. If you adopt the dual-client approach suggested above, you’ll keep this benefit while restoring pooling.
149-151: No action needed: CopyTo + ReleaseResponse ordering is safeVerified that fasthttp.Response.CopyTo performs a synchronous copy of the body into resp.RawResponse (rather than merely reassigning a stream pointer), and once CopyTo returns, you can immediately call ReleaseResponse(respv) without affecting the destination’s BodyStream. The existing code at client/core.go (lines 149–151) is correct and requires no changes.
88-118: Verify comprehensive cloning offasthttp.Clientfields
I wasn’t able to locateclient/core.goin the repository, but manually copying each exported field fromfasthttp.Clientis brittle and prone to omissions (e.g., new fields likeMaxConnWaitTimeout,RetryIf, etc., won’t be copied and can silently change behavior). Please verify that you:
- Mirror all exported fields when creating a temporary client for per-request overrides.
- Centralize this logic in a helper (e.g., reflection-based) to automatically pick up new fields.
- Add a unit test that fails if any exported field in
fasthttp.Clientisn’t present in the clone.- Or consider toggling
StreamResponseBodyon the shared client under a lock instead of cloning.Let me know if you’d like a PR-ready helper and reflection-based test snippet for this.
Abhirup-99
force-pushed
the
add-response
branch
from
4d32c17 to
28342fa
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (3)
client/response.go (2)
92-101: Tighten BodyStream docs: lifetime, connection reuse, and Body() emptiness when streamingThe code is fine. Please clarify the contract and call order to prevent misuse with pooled responses and fasthttp’s connection reuse.
Apply this doc tweak:
-// BodyStream returns the response body as a stream reader. -// Note: When using BodyStream(), the response body is not copied to memory, -// so calling Body() afterwards may return an empty slice. +// BodyStream returns an io.Reader for the response body. +// If streaming was enabled (via Request.SetStreamResponseBody, Client.SetStreamResponseBody, +// or the underlying fasthttp.Client.StreamResponseBody), the reader yields bytes directly from +// the connection without buffering in memory. +// +// Important: +// - Consume the reader before calling Response.Close(). +// - Read the stream to EOF to allow connection reuse by the underlying client. +// - When streaming is enabled, Body() (and helpers like String/JSON/XML/CBOR/Save) may be empty; +// prefer io.Copy with BodyStream() for large responses. +// If streaming is not enabled, BodyStream returns a bytes.Reader over Body(). func (r *Response) BodyStream() io.Reader {
150-165: Save() ignores streaming and closes caller’s writer — stream from BodyStream() and don’t close external writers
- Using Body() makes Save() write an empty file/output when streaming is enabled.
- Closing an io.Writer that the function didn’t open is surprising and can break callers.
Apply this minimal change:
- if _, err = io.Copy(outFile, bytes.NewReader(r.Body())); err != nil { + if _, err = io.Copy(outFile, r.BodyStream()); err != nil { return fmt.Errorf("failed to write response body to file: %w", err) } @@ - case io.Writer: - if _, err := io.Copy(p, bytes.NewReader(r.Body())); err != nil { + case io.Writer: + if _, err := io.Copy(p, r.BodyStream()); err != nil { return fmt.Errorf("failed to write response body to io.Writer: %w", err) } - defer func() { - if pc, ok := p.(io.WriteCloser); ok { - _ = pc.Close() //nolint:errcheck // not needed - } - }() return nilFollow-up (optional): consider adding a short note in Save()’s doc that it respects streaming and doesn’t close external writers.
client/response_test.go (1)
606-629: Avoid retaining Body()’s internal buffer across Close; also assert Body() is empty when streamingTo prevent future flakiness with pooled buffers and to codify the streaming contract, copy the non-streamed body and add an assertion that Body() is empty when streaming is enabled.
Apply:
- defer resp1.Close() - normalBody := resp1.Body() + defer resp1.Close() + normalBody := append([]byte(nil), resp1.Body()...) // copy before pool reuse @@ - streamedBody, err := io.ReadAll(resp2.BodyStream()) + streamedBody, err := io.ReadAll(resp2.BodyStream()) require.NoError(t, err) require.Equal(t, normalBody, streamedBody) + // streaming mode shouldn't buffer Body() + require.Len(t, resp2.Body(), 0)
🧹 Nitpick comments (1)
client/response_test.go (1)
542-629: Add a streaming Save() regression test (uses BodyStream and does not close external writer)Given the Save() fix, add coverage to ensure we don’t regress: the writer must not be closed by Save(), and content must be streamed correctly when streaming is enabled.
Insert this new subtest at the end of Test_Response_BodyStream (before the final closing brace):
+ t.Run("Save streams and does not close external writer", func(t *testing.T) { + t.Parallel() + + server := startTestServer(t, func(app *fiber.App) { + app.Get("/stream", func(c fiber.Ctx) error { + return c.SendString("streaming data") + }) + }) + defer server.stop() + + client := New().SetDial(server.dial()).SetStreamResponseBody(true) + resp, err := client.Get("http://example.com/stream") + require.NoError(t, err) + defer resp.Close() + + type trackingBuf struct { + bytes.Buffer + closed bool + } + // implements io.WriteCloser to verify Save() does not close it + var w = &trackingBuf{} + // wrap with NopCloser-like shim + writer := struct { + io.Writer + io.Closer + }{ + Writer: w, + Closer: io.CloserFunc(func() error { + w.closed = true + return nil + }), + } + + // Save to writer; it must not close it + err = resp.Save(writer) + require.NoError(t, err) + require.Equal(t, "streaming data", w.String()) + require.False(t, w.closed, "Save must not close external writers") + })If you prefer, we can place a similar subtest under Test_Response_Save.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (6)
client/client.go(2 hunks)client/client_test.go(1 hunks)client/core.go(2 hunks)client/request.go(3 hunks)client/response.go(1 hunks)client/response_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- client/client.go
- client/request.go
- client/core.go
- client/client_test.go
🧰 Additional context used
🧬 Code graph analysis (1)
client/response_test.go (1)
client/client.go (2)
Get(706-708)New(647-652)
Abhirup-99
force-pushed
the
add-response
branch
from
28342fa to
ddbb831
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
client/response.go (2)
92-101: Tighten BodyStream contract; clarify lifetime and fallback behaviorDoc is still a bit vague about ordering/lifetime and the fallback to an in-memory reader. Propose expanding the comment to state the contract explicitly and reduce surprises.
-// BodyStream returns the response body as a stream reader. -// Note: When using BodyStream(), the response body is not copied to memory, -// so calling Body() afterwards may return an empty slice. +// BodyStream returns an io.Reader for the response body. +// +// If streaming was enabled (via Request.SetStreamResponseBody, Client.SetStreamResponseBody, +// or the underlying fasthttp.Client.StreamResponseBody), the reader yields data directly from +// the connection without buffering. Otherwise, it returns a bytes.Reader over Body(). +// +// Important: +// - Consume the reader before calling Response.Close(). +// - When streaming is enabled, Body() (and helpers like String/JSON/XML/CBOR/Save) may observe +// an empty body. Prefer io.Copy with BodyStream() for large responses. func (r *Response) BodyStream() io.Reader { if stream := r.RawResponse.BodyStream(); stream != nil { return stream } // If streaming is not enabled, return a bytes.Reader from the regular body return bytes.NewReader(r.RawResponse.Body()) }
161-172: Make Save() always stream via BodyStream(); also stream when saving to a fileThe writer branch partially streams, but the file path branch still buffers via Body(). Since BodyStream() already falls back to a bytes.Reader when streaming isn’t enabled, you can unconditionally copy from BodyStream() in both branches. This avoids large in-memory buffers and simplifies the code.
@@ - if _, err = io.Copy(outFile, bytes.NewReader(r.Body())); err != nil { + if _, err = io.Copy(outFile, r.BodyStream()); err != nil { return fmt.Errorf("failed to write response body to file: %w", err) } @@ - case io.Writer: - var err error - if r.IsStreaming() { - _, err = io.Copy(p, r.BodyStream()) - } else { - _, err = io.Copy(p, bytes.NewReader(r.Body())) - } - - if err != nil { - return fmt.Errorf("failed to write response body to writer: %w", err) - } - - return nil + case io.Writer: + if _, err := io.Copy(p, r.BodyStream()); err != nil { + return fmt.Errorf("failed to write response body to writer: %w", err) + } + return nilFollow-up: consider adding/adjusting tests to cover Save(filePath) with streaming enabled (e.g., large response) to ensure we don’t regress. I can draft that if helpful.
client/core.go (1)
84-120: Per-request allocating a new fasthttp.Client loses connection pooling and adds substantial latency/TLS overheadConstructing a fresh
*fasthttp.Clientwhenever the per-request streaming flag differs discards the shared connection pool and TLS session cache, causing extra TCP/TLS handshakes and reduced throughput under mixed workloads. This will regress performance for SSE/large downloads that toggle streaming on/off across requests.Consider maintaining two long-lived internal clients (streaming vs non-streaming) on
Clientand routing each request accordingly. Here's a suggested implementation:Add to
client/client.go:+// In Client struct: +streamingFHC *fasthttp.Client // lazily initialized clone with StreamResponseBody=true + +// pickFastHTTPClient returns a pooled client configured for the requested streaming mode. +func (c *Client) pickFastHTTPClient(stream bool) *fasthttp.Client { + c.mu.RLock() + defer c.mu.RUnlock() + + if stream == c.streamResponseBody { + return c.fasthttp + } + // Lazily build the streaming variant once and reuse it. + if stream && c.streamingFHC != nil { + return c.streamingFHC + } + return c.initStreamingClientLocked(stream) +} + +// initStreamingClientLocked clones exported config into a new fasthttp.Client. +// c.mu must be held by the caller. +func (c *Client) initStreamingClientLocked(stream bool) *fasthttp.Client { + o := c.fasthttp + clone := &fasthttp.Client{ + Dial: o.Dial, + DialDualStack: o.DialDualStack, + TLSConfig: o.TLSConfig, + MaxConnsPerHost: o.MaxConnsPerHost, + MaxIdleConnDuration: o.MaxIdleConnDuration, + MaxConnDuration: o.MaxConnDuration, + ReadTimeout: o.ReadTimeout, + WriteTimeout: o.WriteTimeout, + ReadBufferSize: o.ReadBufferSize, + WriteBufferSize: o.WriteBufferSize, + MaxResponseBodySize: o.MaxResponseBodySize, + NoDefaultUserAgentHeader: o.NoDefaultUserAgentHeader, + DisableHeaderNamesNormalizing: o.DisableHeaderNamesNormalizing, + DisablePathNormalizing: o.DisablePathNormalizing, + MaxIdemponentCallAttempts: o.MaxIdemponentCallAttempts, + Name: o.Name, + ConfigureClient: o.ConfigureClient, + StreamResponseBody: stream, + } + if stream { + c.streamingFHC = clone + } + return clone +}Then simplify this code block:
- // Determine which client to use - create a new one if StreamResponseBody differs - var fastHTTPClient *fasthttp.Client - requestStreamResponseBody := c.req.StreamResponseBody() - c.client.mu.RLock() - clientStream := c.client.streamResponseBody - original := c.client.fasthttp - - if requestStreamResponseBody != clientStream { - // Request setting differs from client setting, create a temporary client - - fastHTTPClient = &fasthttp.Client{ - Dial: original.Dial, - DialDualStack: original.DialDualStack, - TLSConfig: original.TLSConfig, - MaxConnsPerHost: original.MaxConnsPerHost, - MaxIdleConnDuration: original.MaxIdleConnDuration, - MaxConnDuration: original.MaxConnDuration, - ReadTimeout: original.ReadTimeout, - WriteTimeout: original.WriteTimeout, - ReadBufferSize: original.ReadBufferSize, - WriteBufferSize: original.WriteBufferSize, - MaxResponseBodySize: original.MaxResponseBodySize, - NoDefaultUserAgentHeader: original.NoDefaultUserAgentHeader, - DisableHeaderNamesNormalizing: original.DisableHeaderNamesNormalizing, - DisablePathNormalizing: original.DisablePathNormalizing, - MaxIdemponentCallAttempts: original.MaxIdemponentCallAttempts, - Name: original.Name, - ConfigureClient: original.ConfigureClient, - - // Request-specific override - StreamResponseBody: requestStreamResponseBody, - } - } else { - fastHTTPClient = original - } - c.client.mu.RUnlock() + // Determine which client to use without sacrificing connection pooling + requestStreamResponseBody := c.req.StreamResponseBody() + fastHTTPClient := c.client.pickFastHTTPClient(requestStreamResponseBody)This keeps pooling intact and avoids repeated allocation while maintaining thread safety.
🧹 Nitpick comments (1)
client/response.go (1)
103-106: IsStreaming semantics are OK; consider documenting that it reflects fasthttp.BodyStream presenceMinor: IsStreaming simply mirrors RawResponse.BodyStream() != nil. A short note in the comment that this reports whether the underlying fasthttp response is streaming (not whether data remains unread) would avoid confusion.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (6)
client/client.go(2 hunks)client/client_test.go(1 hunks)client/core.go(2 hunks)client/request.go(3 hunks)client/response.go(2 hunks)client/response_test.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- client/response_test.go
- client/client_test.go
- client/client.go
- client/request.go
🧰 Additional context used
🧬 Code graph analysis (1)
client/core.go (1)
client/client.go (1)
Client(32-62)
🔇 Additional comments (1)
client/core.go (1)
133-142: LGTM!The routing to use
fastHTTPClientinstead of the hardcoded client is correct and properly handles both retry and non-retry paths with redirects.
|
Ok thx, will check your pr tomorrow again |
|
the CI failures don't seem related to my changes |
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
client/client_test.go (1)
2352-2356: Consider trimming redundant default/ getter subtests
"default value"and"getter with standard client without setter"both assert that a freshly created client returnsfalsefromStreamResponseBody(), so they are functionally identical. Keeping just one would reduce noise without losing coverage.Also applies to: 2403-2408
🧹 Nitpick comments (1)
ctx_test.go (1)
3951-3954: Per-callTestConfigusage forapp.Testlooks correctSwitching these upload tests to
app.Test(req, TestConfig{Timeout: 5 * time.Second})is consistent with the variadicTestConfigAPI and gives them more generous time budgets without changing behavior otherwise (defaultFailOnTimeoutremainstrue). If you want to make the intent crystal clear and match patterns used elsewhere in this file, you could optionally spell outFailOnTimeout: trueand/or add adefer resp.Body.Close()after therequire.NoErrorto mirror nearby tests, but the current change is functionally sound as-is.Also applies to: 3993-3996
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
client/client_test.go(1 hunks)client/response.go(3 hunks)ctx_test.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- client/response.go
🧰 Additional context used
📓 Path-based instructions (2)
**/*_test.go
📄 CodeRabbit inference engine (AGENTS.md)
When adding Go tests, always invoke
t.Parallel()at the start of each test and subtest to maximize concurrency
Files:
client/client_test.goctx_test.go
**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
Prefer
github.com/gofiber/utils/v2helpers (for example,utils.Trim) when performing common operations such as string manipulation, whenever it is practical and appropriate for the surrounding code
Files:
client/client_test.goctx_test.go
🧠 Learnings (8)
📚 Learning: 2025-10-16T07:19:52.418Z
Learnt from: grivera64
Repo: gofiber/fiber PR: 3807
File: adapter_test.go:118-144
Timestamp: 2025-10-16T07:19:52.418Z
Learning: In the Fiber codebase, the linter does not allow `require` assertions from within HTTP handlers (including net/http-style handlers). Use `t.Fatalf`, `t.Errorf`, or similar `testing.T` methods for error handling inside handler functions instead.
Applied to files:
client/client_test.go
📚 Learning: 2024-12-13T08:14:22.851Z
Learnt from: efectn
Repo: gofiber/fiber PR: 3162
File: hooks_test.go:228-228
Timestamp: 2024-12-13T08:14:22.851Z
Learning: In Go test files, prefer using the `require` methods from the `testify` package for assertions instead of manual comparisons and calls to `t.Fatal` or `t.Fatalf`.
Applied to files:
client/client_test.go
📚 Learning: 2024-11-29T12:37:27.581Z
Learnt from: efectn
Repo: gofiber/fiber PR: 3162
File: app_test.go:893-895
Timestamp: 2024-11-29T12:37:27.581Z
Learning: In the `Test_App_ShutdownWithContext` function in `app_test.go`, the `clientDone` channel is used to synchronize the client's request completion before proceeding, eliminating the need for additional `time.Sleep` calls.
Applied to files:
client/client_test.goctx_test.go
📚 Learning: 2024-10-08T19:06:06.583Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/config.go:122-122
Timestamp: 2024-10-08T19:06:06.583Z
Learning: In `DefaultErrorHandler(c *fiber.Ctx, err error)`, since `c` is a pointer to an interface, we need to dereference `*c` when calling interface methods like `SendStatus`.
Applied to files:
client/client_test.go
📚 Learning: 2024-11-10T23:44:13.704Z
Learnt from: gaby
Repo: gofiber/fiber PR: 3193
File: middleware/adaptor/adaptor.go:111-111
Timestamp: 2024-11-10T23:44:13.704Z
Learning: In the `middleware/adaptor/adaptor.go` file of the Fiber framework, when updating context handling, replacing `c.Context()` with `c.RequestCtx()` is appropriate to access the `fasthttp.RequestCtx`.
Applied to files:
client/client_test.go
📚 Learning: 2024-11-15T07:56:21.623Z
Learnt from: ReneWerner87
Repo: gofiber/fiber PR: 3161
File: app.go:923-932
Timestamp: 2024-11-15T07:56:21.623Z
Learning: In the Fiber framework, breaking changes are acceptable when moving from version 2 to version 3, including modifications to method signatures such as in the `Test` method in `app.go`.
Applied to files:
ctx_test.go
📚 Learning: 2024-11-08T04:10:42.990Z
Learnt from: gaby
Repo: gofiber/fiber PR: 3193
File: middleware/cache/cache_test.go:897-897
Timestamp: 2024-11-08T04:10:42.990Z
Learning: In the Fiber framework, `Context()` is being renamed to `RequestCtx()`, and `UserContext()` to `Context()` to improve clarity and align with Go's context conventions.
Applied to files:
ctx_test.go
📚 Learning: 2024-10-12T10:01:44.206Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/middleware_test.go:190-191
Timestamp: 2024-10-12T10:01:44.206Z
Learning: When testing session `IdleTimeout` expiration, it's acceptable to use `time.Sleep` to simulate the passage of time in tests.
Applied to files:
ctx_test.go
🧬 Code graph analysis (2)
client/client_test.go (1)
client/client.go (3)
New(752-757)NewWithHostClient(768-773)NewWithLBClient(776-781)
ctx_test.go (1)
app.go (1)
TestConfig(1115-1126)
🔇 Additional comments (1)
client/client_test.go (1)
2349-2401: Streaming configuration tests look correct and cover key pathsThese subtests validate default value, enable/disable toggling, and propagation to standard, host, and LB client transports using the new
StreamResponseBodyAPI. The assertions are straightforward and align with the intended behavior.
Abhirup-99
force-pushed
the
add-response
branch
from
9514e05 to
92c59e6
Compare
Abhirup-99
force-pushed
the
add-response
branch
from
92c59e6 to
f2a19c7
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
client/response.go (1)
186-209: Add explicit BodyStream cleanup inResponse.Close()to prevent connection leaksWhen
StreamResponseBodyis enabled, fasthttp'sResponse.Reset()does not automatically close an outstandingBodyStreamreader. If the stream is not fully consumed beforeClose()is called, the underlying connection will not be released back to the pool, causing connection leaks.Add
r.RawResponse.CloseBodyStream()inClose()before callingReleaseResponse():func (r *Response) Close() { if r.request != nil { tmp := r.request r.request = nil ReleaseRequest(tmp) } r.RawResponse.CloseBodyStream() ReleaseResponse(r) }
♻️ Duplicate comments (4)
client/response.go (2)
92-106: Consider tighteningBodyStream()docs to be explicit about lifetime/consumption
E.g., “consume beforeResponse.Close(); streaming bodies may makeBody()empty.” (This has come up before, but worth keeping crisp since it’s public API.)
155-177: SimplifySave()by always copying fromBodyStream()
BodyStream()already returns the rightio.Readerfor both streaming and non-streaming cases, so theIsStreaming()branching is redundant.client/response_test.go (2)
1062-1075:bodyStreamis acquired twice - potential double consumption.
bodyStreamis declared inside theif/elseblocks at lines 1065 and 1070, then again at line 1075. The second call toresp.BodyStream()at line 1075 may return a different reader or an already-consumed stream depending on implementation. Acquire the stream once before conditional checks.This was flagged in a past review. Ensure
bodyStreamis acquired only once:rawStream := resp.RawResponse.BodyStream() + bodyStream := resp.BodyStream() if rawStream != nil { require.True(t, resp.IsStreaming()) - bodyStream := resp.BodyStream() require.NotNil(t, bodyStream) require.Same(t, rawStream, bodyStream, "BodyStream() should return the exact same stream object when RawResponse.BodyStream() is not nil") } else { require.False(t, resp.IsStreaming()) - bodyStream := resp.BodyStream() require.NotNil(t, bodyStream) _, ok := bodyStream.(*bytes.Reader) require.True(t, ok, "When RawResponse.BodyStream() is nil, BodyStream() should return a *bytes.Reader") } - bodyStream := resp.BodyStream() data, err := io.ReadAll(bodyStream)
739-751: Potential flaky test:normalBodymay reference pooled buffer afterresp1.Close().In "compare with regular body",
normalBodyholds a reference toresp1's internal buffer. Ifresp1.Close()is called before comparison (or if there's any reordering), the buffer could be reused by the pool, causing intermittent failures. Copy the bytes before proceeding.client1 := New().SetDial(server.dial()) resp1, err := client1.Get("http://example.com/stream") require.NoError(t, err) defer resp1.Close() - normalBody := resp1.Body() + normalBody := append([]byte(nil), resp1.Body()...) client2 := New().SetDial(server.dial()).SetStreamResponseBody(true)
🧹 Nitpick comments (1)
docs/client/rest.md (1)
409-423: Documentation covers the new streaming API but could benefit from additional detail.The signatures and basic descriptions are present. Consider enhancing with:
- Default value (
false)- Note about per-request override via
Request.SetStreamResponseBody(bool)- Caution about not mixing
Body()andBodyStream()on the same responseThese details would help users avoid common pitfalls.
## StreamResponseBody -Returns whether response body streaming is enabled. When enabled, the response body is not fully loaded into memory and can be read as a stream using `BodyStream()`. This is useful for handling large responses or server-sent events. +Returns whether response body streaming is enabled for the client. + +When enabled, the response body is not fully buffered in memory and should be consumed via `Response.BodyStream()`. This is useful for large downloads and server-sent events. + +:::note +- Default: `false` +- Per-request override: use `Request.SetStreamResponseBody(bool)` +- When streaming, prefer `Response.BodyStream()` over `Response.Body()` for large payloads +::: ```go title="Signature" func (c *Client) StreamResponseBody() bool
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
client/client.go(1 hunks)client/client_test.go(1 hunks)client/core.go(2 hunks)client/core_test.go(1 hunks)client/response.go(3 hunks)client/response_test.go(2 hunks)client/transport.go(4 hunks)client/transport_test.go(1 hunks)ctx_test.go(2 hunks)docs/client/rest.md(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- client/client.go
- client/core_test.go
🧰 Additional context used
📓 Path-based instructions (4)
**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
Prefer
github.com/gofiber/utils/v2helpers (for example,utils.Trim) when performing common operations such as string manipulation, whenever it is practical and appropriate for the surrounding code
Files:
client/core.goclient/response.goctx_test.goclient/response_test.goclient/transport_test.goclient/client_test.goclient/transport.go
**/*_test.go
📄 CodeRabbit inference engine (AGENTS.md)
When adding Go tests, always invoke
t.Parallel()at the start of each test and subtest to maximize concurrency
Files:
ctx_test.goclient/response_test.goclient/transport_test.goclient/client_test.go
docs/**
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Review and update the contents of the
docsfolder if necessary when modifying code
Files:
docs/client/rest.md
**/*.md
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Run
make markdownto lint all Markdown files when modifying code
Files:
docs/client/rest.md
🧠 Learnings (13)
📚 Learning: 2024-11-10T23:44:13.704Z
Learnt from: gaby
Repo: gofiber/fiber PR: 3193
File: middleware/adaptor/adaptor.go:111-111
Timestamp: 2024-11-10T23:44:13.704Z
Learning: In the `middleware/adaptor/adaptor.go` file of the Fiber framework, when updating context handling, replacing `c.Context()` with `c.RequestCtx()` is appropriate to access the `fasthttp.RequestCtx`.
Applied to files:
client/core.goclient/client_test.go
📚 Learning: 2025-12-07T15:07:23.885Z
Learnt from: CR
Repo: gofiber/fiber PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-07T15:07:23.885Z
Learning: Applies to **/*.go : Prefer `github.com/gofiber/utils/v2` helpers (for example, `utils.Trim`) when performing common operations such as string manipulation, whenever it is practical and appropriate for the surrounding code
Applied to files:
client/response.go
📚 Learning: 2025-10-16T07:19:52.418Z
Learnt from: grivera64
Repo: gofiber/fiber PR: 3807
File: adapter_test.go:118-144
Timestamp: 2025-10-16T07:19:52.418Z
Learning: In the Fiber codebase, the linter does not allow `require` assertions from within HTTP handlers (including net/http-style handlers). Use `t.Fatalf`, `t.Errorf`, or similar `testing.T` methods for error handling inside handler functions instead.
Applied to files:
client/response.goctx_test.goclient/response_test.goclient/client_test.go
📚 Learning: 2025-09-28T17:39:26.644Z
Learnt from: arturmelanchyk
Repo: gofiber/fiber PR: 3768
File: client/request.go:162-166
Timestamp: 2025-09-28T17:39:26.644Z
Learning: In fasthttp-based code like gofiber, using utils.UnsafeString(key) instead of string(key) is acceptable when values are used immediately within the same call stack and not stored for later use, as it avoids allocations and aligns with fasthttp's zero-allocation design philosophy.
Applied to files:
client/response.go
📚 Learning: 2025-10-16T07:15:26.529Z
Learnt from: grivera64
Repo: gofiber/fiber PR: 3807
File: adapter_test.go:118-144
Timestamp: 2025-10-16T07:15:26.529Z
Learning: In Fiber v3, net/http handlers (http.Handler, http.HandlerFunc, or raw func(http.ResponseWriter, *http.Request)) can be passed directly to routing methods like app.Get(), app.Post(), etc. The framework automatically detects and wraps them internally via toFiberHandler/collectHandlers. The github.com/gofiber/fiber/v3/middleware/adaptor package is legacy and should not be suggested for tests or code using native net/http handler support.
Applied to files:
client/response.go
📚 Learning: 2024-11-15T07:56:21.623Z
Learnt from: ReneWerner87
Repo: gofiber/fiber PR: 3161
File: app.go:923-932
Timestamp: 2024-11-15T07:56:21.623Z
Learning: In the Fiber framework, breaking changes are acceptable when moving from version 2 to version 3, including modifications to method signatures such as in the `Test` method in `app.go`.
Applied to files:
ctx_test.go
📚 Learning: 2024-10-12T10:01:44.206Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/middleware_test.go:190-191
Timestamp: 2024-10-12T10:01:44.206Z
Learning: When testing session `IdleTimeout` expiration, it's acceptable to use `time.Sleep` to simulate the passage of time in tests.
Applied to files:
ctx_test.go
📚 Learning: 2024-11-29T12:37:27.581Z
Learnt from: efectn
Repo: gofiber/fiber PR: 3162
File: app_test.go:893-895
Timestamp: 2024-11-29T12:37:27.581Z
Learning: In the `Test_App_ShutdownWithContext` function in `app_test.go`, the `clientDone` channel is used to synchronize the client's request completion before proceeding, eliminating the need for additional `time.Sleep` calls.
Applied to files:
ctx_test.goclient/response_test.goclient/client_test.go
📚 Learning: 2025-12-07T15:07:23.885Z
Learnt from: CR
Repo: gofiber/fiber PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-07T15:07:23.885Z
Learning: Applies to **/*_test.go : When adding Go tests, always invoke `t.Parallel()` at the start of each test and subtest to maximize concurrency
Applied to files:
ctx_test.go
📚 Learning: 2024-09-25T16:18:34.719Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/config.go:122-122
Timestamp: 2024-09-25T16:18:34.719Z
Learning: In `DefaultErrorHandler(c *fiber.Ctx, err error)`, since `c` is a pointer to an interface, we need to dereference `*c` when calling interface methods like `SendStatus`.
Applied to files:
client/response_test.goclient/client_test.go
📚 Learning: 2024-10-08T19:06:06.583Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/store.go:164-167
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests in this project use testify require.
Applied to files:
client/response_test.go
📚 Learning: 2024-10-08T19:06:06.583Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 2922
File: middleware/cors/utils.go:63-71
Timestamp: 2024-10-08T19:06:06.583Z
Learning: The project uses the testify/assert package for assertions in unit tests.
Applied to files:
client/response_test.go
📚 Learning: 2024-12-13T08:14:22.851Z
Learnt from: efectn
Repo: gofiber/fiber PR: 3162
File: hooks_test.go:228-228
Timestamp: 2024-12-13T08:14:22.851Z
Learning: In Go test files, prefer using the `require` methods from the `testify` package for assertions instead of manual comparisons and calls to `t.Fatal` or `t.Fatalf`.
Applied to files:
client/client_test.go
🧬 Code graph analysis (4)
client/core.go (1)
client/response.go (1)
ReleaseResponse(232-235)
ctx_test.go (1)
app.go (1)
TestConfig(1114-1125)
client/response_test.go (2)
client/client.go (1)
Get(831-833)client/response.go (3)
Response(19-25)AcquireResponse(222-228)ReleaseResponse(232-235)
client/transport_test.go (1)
client/client.go (1)
Client(37-67)
🔇 Additional comments (12)
client/core.go (2)
88-93: Nice: nil-guardedfasthttp.ReleaseResponsematches the new ownership transfer
This avoids releasingrespvafter it’s handed off toresp.RawResponse.
125-133: Good: swapresp.RawResponseto preserve response streaming without forcing a copyOne thing to double-check: whichever path ultimately "owns"
resp.RawResponsemust ensure the underlying response BodyStream is closed onResponse.Close()/Response.Reset()to avoid leaking connections. In fasthttp,(*Response).Reset()does not close an active BodyStream—you must either fully read the stream to EOF or explicitly callCloseBodyStream()beforehand.ctx_test.go (2)
3951-3952: LGTM: per-callTestConfig{Timeout: 5 * time.Second}makes the test less flaky
4020-4022: LGTM: per-callTestConfigaligns with the newApp.Test(req, cfg)APIclient/client_test.go (1)
2349-2409: LGTM! Comprehensive streaming state tests with proper parallel execution.Tests cover all transport types (standard, host, LB) and correctly verify both getter/setter behavior and chainability. All subtests properly invoke
t.Parallel()per coding guidelines.Note: The "with standard client" and "getter with standard client without setter" subtests overlap slightly with "enable streaming" and "default value" respectively, but they add clarity by explicitly naming the transport type being tested.
client/transport_test.go (1)
350-548: Well-structured transport streaming tests with thorough coverage.The tests comprehensively cover:
- Default values, enable/disable toggling for all transport types
- LB transport edge cases: empty clients, nested LBs, mixed client types
- Interface consistency verification across all implementations
All subtests properly invoke
t.Parallel()per coding guidelines.client/response_test.go (2)
546-619: Good streaming Save() tests with proper server-side streaming.Tests correctly use
SendStreamWriterwith explicit flushes to exercise true streaming behavior. The helper types (testClosableBuffer,testErrorWriter) effectively validate thatSave()doesn't close the writer and properly propagates write errors.
806-858: SSE test now usesSendStreamWriterwith flushes - properly exercises streaming.This addresses earlier review feedback. The test correctly streams multiple events with delays and verifies all content is received.
client/transport.go (4)
200-221: LB transport getter/setter asymmetry is acceptable but should be documented.The getter returns the first
HostClient'sStreamResponseBodyvalue, while the setter propagates to all. This assumes all clients should share the same setting, which is reasonable for streaming configuration.However, clients added dynamically via
fasthttp.LBClient.AddClient()won't inherit the streaming setting - they'll keep the defaultfalse. This edge case may surprise users in dynamic load-balancing scenarios.Consider adding a comment clarifying this behavior:
func (l *lbClientTransport) StreamResponseBody() bool { if len(l.client.Clients) == 0 { return false } - // Return the StreamResponseBody setting from the first HostClient + // Return the StreamResponseBody setting from the first HostClient. + // All clients are expected to share the same setting after SetStreamResponseBody. + // Note: Clients added dynamically via LBClient.AddClient() won't inherit this setting. var streamEnabled bool
30-31: Interface extended with streaming control methods.The additions to
httpClientTransportprovide a clean abstraction for streaming configuration across all transport types.
80-86: Standard transport streaming implementation is correct.Direct delegation to
fasthttp.Client.StreamResponseBodyfield.
134-140: Host transport streaming implementation is correct.Direct delegation to
fasthttp.HostClient.StreamResponseBodyfield.
|
@grivera64 @gaby bump |
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
client/core.go (1)
96-115: Retries will fail with non-rewindable request body streams.When
cfg != nil(retry enabled),retry.Retry()will invoke the closure multiple times (lines 103-108), butSetBodyStreamat line 97 is called only once with the original stream. After the first attempt consumes the stream, subsequent retries will send an empty or partial body, causing silent data corruption or unexpected failures.Solutions (in order of preference):
Disable retries when body streaming is present:
if bodyStream := c.req.RawRequest.BodyStream(); bodyStream != nil { reqv.SetBodyStream(bodyStream, c.req.RawRequest.Header.ContentLength()) cfg = nil // Force disable retry for streaming requests }Require
io.ReadSeekerand rewind before each attempt (add seeker detection and seek to 0 in the retry closure).Document the limitation if retries + body streaming is intentionally unsupported and add validation to return an error when both are configured.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
client/core.goclient/transport.go
🚧 Files skipped from review as they are similar to previous changes (1)
- client/transport.go
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
Prefer
github.com/gofiber/utils/v2helpers (for example,utils.Trim) when performing common operations such as string manipulation, whenever it is practical and appropriate for the surrounding code
Files:
client/core.go
🧠 Learnings (1)
📚 Learning: 2024-11-10T23:44:13.704Z
Learnt from: gaby
Repo: gofiber/fiber PR: 3193
File: middleware/adaptor/adaptor.go:111-111
Timestamp: 2024-11-10T23:44:13.704Z
Learning: In the `middleware/adaptor/adaptor.go` file of the Fiber framework, when updating context handling, replacing `c.Context()` with `c.RequestCtx()` is appropriate to access the `fasthttp.RequestCtx`.
Applied to files:
client/core.go
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: lint
- GitHub Check: Compare
- GitHub Check: unit (1.25.x, macos-latest)
- GitHub Check: unit (1.25.x, windows-latest)
- GitHub Check: repeated
🔇 Additional comments (2)
client/core.go (2)
89-93: LGTM: Safe resource cleanup with nil check.The nil check prevents double-free when
respvownership is transferred toresp.RawResponse(line 130). This is a correct defensive pattern.
125-133: LGTM: Correct ownership transfer pattern.The code correctly transfers
respvownership toresp.RawResponsewhile preventing double-free (by settingrespv = nil) and avoiding leaks (by releasingoriginalRaw). The comment clearly explains that this preserves the response body stream for lazy reading.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
|
@gaby The The most recently added commit in this PR reverted the manual copy to this old We could either:
|
|
test failed === Failed
=== FAIL: client Test_Response_Save_Streaming_FilePath (0.10s)
response_test.go:1292:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:1292
Error: Not equal:
expected: "streaming file content"
actual : ""
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-streaming file content
+
Test: Test_Response_Save_Streaming_FilePath
=== FAIL: client Test_Response_Save/streaming_with_io.Writer_error_during_copy (0.10s)
response_test.go:616:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:616
Error: An error is expected but got nil.
Test: Test_Response_Save/streaming_with_io.Writer_error_during_copy
=== FAIL: client Test_Response_Save/streaming_with_io.Writer_without_closing (10.00s)
response_test.go:581:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:581
Error: "" does not contain "SSSSSS"
Test: Test_Response_Save/streaming_with_io.Writer_without_closing
=== FAIL: client Test_Response_Save (0.00s)
=== FAIL: client Test_Response_IsStreaming/streaming_with_large_response (10.10s)
response_test.go:1195:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:1195
Error: "[]" should have 65536 item(s), but has 0
Test: Test_Response_IsStreaming/streaming_with_large_response
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:1078
Error: "[]" should have 8192 item(s), but has 0
Test: Test_Response_BodyStream/stream_object_identity_when_streaming_enabled
=== FAIL: client Test_Response_BodyStream/basic_streaming (0.10s)
response_test.go:678:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:678
Error: Not equal:
expected: "streaming data"
actual : ""
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-streaming data
+
Test: Test_Response_BodyStream/basic_streaming
=== FAIL: client Test_Response_BodyStream/chunked_streaming_with_delays (0.10s)
response_test.go:802:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:802
exit status 1
Error: Not equal:
expected: "chunk1chunk2chunk3"
actual : ""
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-chunk1chunk2chunk3
+
Test: Test_Response_BodyStream/chunked_streaming_with_delays
=== FAIL: client Test_Response_BodyStream/large_response_streaming_validation (10.10s)
response_test.go:1034:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:1034
Error: "[]" should have 10240 item(s), but has 0
Test: Test_Response_BodyStream/large_response_streaming_validation
=== FAIL: client Test_Response_BodyStream/server_sent_events_with_incremental_reads (10.10s)
response_test.go:852:
Error Trace: /home/runner/work/fiber/fiber/client/response_test.go:852
Error: "" does not contain "event 1"
Test: Test_Response_BodyStream/server_sent_events_with_incremental_reads |
|
@ReneWerner87 I believe this is due to using https://github.com/valyala/fasthttp/blob/7b5cb77b95e2200cab14572519bd7dfdcc55fdeb/http.go#L909-L923 This method does not copy body streams to the diff --git a/client/core.go b/client/core.go
index 38333099..09ed2a5c 100644
--- a/client/core.go
+++ b/client/core.go
@@ -122,16 +122,16 @@ func (c *core) execFunc() (*Response, error) {
resp := AcquireResponse()
resp.setClient(c.client)
resp.setRequest(c.req)
- // Copy the fasthttp response into the Fiber response's RawResponse field
- // instead of taking ownership of the pooled fasthttp.Response (respv). This
- // allows respv to be safely released back to the fasthttp pool by the
- // deferred cleanup above, avoiding resource leaks.
- targetRaw := resp.RawResponse
- if targetRaw == nil {
- targetRaw = fasthttp.AcquireResponse()
- }
- respv.CopyTo(targetRaw)
- resp.RawResponse = targetRaw
+
+ // Swap the fasthttp response with the Fiber response's RawResponse field.
+ // This is required, as (*fasthttp.Response).CopyTo() explicitly does not
+ // copy body streams.
+ //
+ // See: https://github.com/valyala/fasthttp/blob/master/http.go#L909-L923
+ //
+ // The defer statement above ensures that the original RawResponse
+ // (now stored in respv) will be properly released.
+ resp.RawResponse, respv = respv, resp.RawResponse
respChan <- resp
}()Should I push a commit with this change to the PR? |
|
@grivera64 Yes, thanks! 💪 |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
ctx_test.go (2)
3989-3996: Closeresp.BodyinTest_Ctx_SaveFileafterapp.TestYou now capture
respbut never close the body, which is inconsistent with the rest of this file and can leak resources in large parallel suites.Proposed fix
resp, err := app.Test(req, TestConfig{Timeout: 5 * time.Second}) require.NoError(t, err, "app.Test(req)") +t.Cleanup(func() { require.NoError(t, resp.Body.Close()) }) require.Equal(t, StatusOK, resp.StatusCode, "Status code")
4058-4065: Closeresp.BodyinTest_Ctx_SaveFileToStorageafterapp.TestSame issue as above:
resp.Bodyshould be closed.Proposed fix
resp, err := app.Test(req, TestConfig{Timeout: 5 * time.Second}) require.NoError(t, err, "app.Test(req)") +t.Cleanup(func() { require.NoError(t, resp.Body.Close()) }) require.Equal(t, StatusOK, resp.StatusCode, "Status code")client/core.go (1)
95-115: Guard retries when request body is a non-replayable stream, or ensure stream replayability.The retry loop in
execFuncreuses the samereqvobject with its body stream across multiple attempts. Ifc.req.RawRequest.BodyStream()returns a non-replayable stream (notio.ReadSeeker), the retry will attempt to read from an already-consumed stream, resulting in empty or corrupted request bodies on retry attempts.Consider adding one of these safeguards when
cfg != nil:
- Disable retries if
bodyStream != niland it's not anio.ReadSeeker- Reset the stream position before each retry attempt using
Seek(0, io.SeekStart)forReadSeekerstreams- Document the limitation clearly for users relying on retries
🤖 Fix all issues with AI agents
In @client/core.go:
- Around line 88-93: The code currently uses respv.CopyTo(targetRaw) which
discards BodyStream and then releases respv, breaking streamed responses;
instead transfer ownership of the acquired response into the wrapper by
assigning resp.RawResponse = respv (preserving BodyStream), then set respv =
oldRaw (the new empty response to be released), and adjust the deferred Release
to only release respv if it's not nil; apply the same swap/ownership pattern for
the other block that handles responses (the block around respv at lines 125–134)
so resp.Close() / Reset() will correctly clean up the preserved RawResponse.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
client/core.goctx_test.godocs/client/rest.md
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/client/rest.md
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
Prefer
github.com/gofiber/utils/v2helpers (for example,utils.Trim) when performing common operations such as string manipulation, whenever it is practical and appropriate for the surrounding code
Files:
client/core.goctx_test.go
**/*_test.go
📄 CodeRabbit inference engine (AGENTS.md)
When adding Go tests, always invoke
t.Parallel()at the start of each test and subtest to maximize concurrency
Files:
ctx_test.go
🧠 Learnings (7)
📚 Learning: 2024-11-10T23:44:13.704Z
Learnt from: gaby
Repo: gofiber/fiber PR: 3193
File: middleware/adaptor/adaptor.go:111-111
Timestamp: 2024-11-10T23:44:13.704Z
Learning: In the `middleware/adaptor/adaptor.go` file of the Fiber framework, when updating context handling, replacing `c.Context()` with `c.RequestCtx()` is appropriate to access the `fasthttp.RequestCtx`.
Applied to files:
client/core.go
📚 Learning: 2024-09-25T15:57:10.221Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/csrf/csrf_test.go:188-193
Timestamp: 2024-09-25T15:57:10.221Z
Learning: In the Fiber framework tests, using `ctx.Response.Header.Cookie` may not be suitable for parsing cookies from the response header, as it requires a `*Cookie` and fills it rather than returning a string value; thus, manual parsing of the `Set-Cookie` header may be necessary.
Applied to files:
client/core.go
📚 Learning: 2025-10-16T07:19:52.418Z
Learnt from: grivera64
Repo: gofiber/fiber PR: 3807
File: adapter_test.go:118-144
Timestamp: 2025-10-16T07:19:52.418Z
Learning: In the Fiber codebase, the linter does not allow `require` assertions from within HTTP handlers (including net/http-style handlers). Use `t.Fatalf`, `t.Errorf`, or similar `testing.T` methods for error handling inside handler functions instead.
Applied to files:
client/core.goctx_test.go
📚 Learning: 2024-11-15T07:56:21.623Z
Learnt from: ReneWerner87
Repo: gofiber/fiber PR: 3161
File: app.go:923-932
Timestamp: 2024-11-15T07:56:21.623Z
Learning: In the Fiber framework, breaking changes are acceptable when moving from version 2 to version 3, including modifications to method signatures such as in the `Test` method in `app.go`.
Applied to files:
ctx_test.go
📚 Learning: 2024-11-29T12:37:27.581Z
Learnt from: efectn
Repo: gofiber/fiber PR: 3162
File: app_test.go:893-895
Timestamp: 2024-11-29T12:37:27.581Z
Learning: In the `Test_App_ShutdownWithContext` function in `app_test.go`, the `clientDone` channel is used to synchronize the client's request completion before proceeding, eliminating the need for additional `time.Sleep` calls.
Applied to files:
ctx_test.go
📚 Learning: 2025-12-07T15:07:23.885Z
Learnt from: CR
Repo: gofiber/fiber PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-07T15:07:23.885Z
Learning: Applies to **/*_test.go : When adding Go tests, always invoke `t.Parallel()` at the start of each test and subtest to maximize concurrency
Applied to files:
ctx_test.go
📚 Learning: 2024-10-12T10:01:44.206Z
Learnt from: sixcolors
Repo: gofiber/fiber PR: 3016
File: middleware/session/middleware_test.go:190-191
Timestamp: 2024-10-12T10:01:44.206Z
Learning: When testing session `IdleTimeout` expiration, it's acceptable to use `time.Sleep` to simulate the passage of time in tests.
Applied to files:
ctx_test.go
🧬 Code graph analysis (2)
client/core.go (2)
client/response.go (2)
ReleaseResponse(232-235)AcquireResponse(222-228)client/request.go (1)
Header(708-710)
ctx_test.go (1)
app.go (1)
TestConfig(1114-1125)
🔇 Additional comments (1)
client/core.go (1)
88-93: Good: nil-guardedfasthttp.ReleaseResponse(respv)enables safe ownership transfer later.
This makes it possible to setrespv = nil(or swap it) when the response needs to outlive the goroutine.
| respv := fasthttp.AcquireResponse() | ||
| defer fasthttp.ReleaseResponse(respv) | ||
| defer func() { | ||
| if respv != nil { | ||
| fasthttp.ReleaseResponse(respv) | ||
| } | ||
| }() |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's check the file and understand the context
cat -n client/core.go | sed -n '85,135p'Repository: gofiber/fiber
Length of output: 2051
🌐 Web query:
fasthttp Response CopyTo BodyStream — does CopyTo copy the body stream
💡 Result:
No — Response.CopyTo does not copy the body stream. It copies response contents except the body stream; if resp.IsBodyStream() is true you must transfer the stream yourself (e.g., read and SetBody / SetBodyStream on the destination). [1][2]
Sources: [1] pkg.go.dev/github.com/valyala/fasthttp — Response.CopyTo docs. [2] pkg.go.dev/github.com/go-fasthttp/fasthttp — CopyTo doc.
🏁 Script executed:
# Check Response.Close and Response.Reset implementations
cat -n client/response.go | head -150Repository: gofiber/fiber
Length of output: 5518
🏁 Script executed:
# Search for RawResponse assignments in client directory
rg -n 'RawResponse\s*=' client/Repository: gofiber/fiber
Length of output: 106
🏁 Script executed:
# Also check the second location mentioned (lines 125-134)
cat -n client/core.go | sed -n '122,140p'Repository: gofiber/fiber
Length of output: 787
🏁 Script executed:
# Check for Response.Close and Response.Reset function signatures
rg -n 'func \(.*\*Response\) (Close|Reset)' client/response.go -A 5Repository: gofiber/fiber
Length of output: 329
🏁 Script executed:
# Get full Close() and Reset() implementations
cat -n client/response.go | sed -n '187,215p'Repository: gofiber/fiber
Length of output: 867
🏁 Script executed:
# Search for where RawResponse might be released
rg -n 'ReleaseResponse|fasthttp.Release' client/response.go -A 2Repository: gofiber/fiber
Length of output: 524
🏁 Script executed:
# Check if AcquireResponse/ReleaseResponse pattern is used elsewhere
rg -n 'AcquireResponse|ReleaseResponse' client/Repository: gofiber/fiber
Length of output: 1786
🏁 Script executed:
# Check the full response.go file to understand complete lifecycle
wc -l client/response.goRepository: gofiber/fiber
Length of output: 79
Critical: respv.CopyTo(targetRaw) loses BodyStream, breaking response streaming.
fasthttp.Response.CopyTo does not copy body streams. When the server returns a streamed response, the BodyStream remains on respv, which is immediately released by the deferred cleanup (line 91–93). This causes resp.BodyStream() to return an empty buffered body instead of the actual stream.
Transfer ownership of respv into resp.RawResponse by swapping instead: set resp.RawResponse = respv, then set respv = oldRaw. This preserves the body stream in the response, which is properly reset when resp.Close() is called (via Reset() at line 197).
Proposed fix
respv := fasthttp.AcquireResponse()
defer func() {
if respv != nil {
fasthttp.ReleaseResponse(respv)
}
}()
@@
resp := AcquireResponse()
resp.setClient(c.client)
resp.setRequest(c.req)
-// Copy the fasthttp response into the Fiber response's RawResponse field
-// instead of taking ownership of the pooled fasthttp.Response (respv). This
-// allows respv to be safely released back to the fasthttp pool by the
-// deferred cleanup above, avoiding resource leaks.
-targetRaw := resp.RawResponse
-if targetRaw == nil {
- targetRaw = fasthttp.AcquireResponse()
-}
-respv.CopyTo(targetRaw)
-resp.RawResponse = targetRaw
+// Transfer ownership of respv into resp.RawResponse (preserves BodyStream).
+// Swap so the deferred release returns the old RawResponse (if any) to the pool.
+oldRaw := resp.RawResponse
+resp.RawResponse = respv
+respv = oldRaw
respChan <- respAlso applies to: lines 125–134
🤖 Prompt for AI Agents
In @client/core.go around lines 88 - 93, The code currently uses
respv.CopyTo(targetRaw) which discards BodyStream and then releases respv,
breaking streamed responses; instead transfer ownership of the acquired response
into the wrapper by assigning resp.RawResponse = respv (preserving BodyStream),
then set respv = oldRaw (the new empty response to be released), and adjust the
deferred Release to only release respv if it's not nil; apply the same
swap/ownership pattern for the other block that handles responses (the block
around respv at lines 125–134) so resp.Close() / Reset() will correctly clean up
the preserved RawResponse.
@gaby I am having a permissions issue pushing my local commit to the branch (I think because contributors aren't able to push to another contributor's PRs). I made a commit under my fork: https://github.com/grivera64/fiber.git in the branch git pull https://github.com/grivera64/fiber.git fix/Abhirup-99-add-responseIf that doesn't work, I think it is also fine to just apply this diff directly: diff --git a/client/core.go b/client/core.go
index 38333099..050f76e6 100644
--- a/client/core.go
+++ b/client/core.go
@@ -122,16 +122,16 @@ func (c *core) execFunc() (*Response, error) {
resp := AcquireResponse()
resp.setClient(c.client)
resp.setRequest(c.req)
- // Copy the fasthttp response into the Fiber response's RawResponse field
- // instead of taking ownership of the pooled fasthttp.Response (respv). This
- // allows respv to be safely released back to the fasthttp pool by the
- // deferred cleanup above, avoiding resource leaks.
- targetRaw := resp.RawResponse
- if targetRaw == nil {
- targetRaw = fasthttp.AcquireResponse()
- }
- respv.CopyTo(targetRaw)
- resp.RawResponse = targetRaw
+
+ // Swap the fasthttp response with the Fiber response's RawResponse field.
+ // This is required, as (*fasthttp.Response).CopyTo() explicitly does not
+ // copy body streams.
+ //
+ // See: https://github.com/valyala/fasthttp/blob/v1.69.0/http.go#L909-L923
+ //
+ // The defer statement above ensures that the original RawResponse
+ // (now stored in respv) will be properly released.
+ resp.RawResponse, respv = respv, resp.RawResponse
respChan <- resp
}()
After merging, I believe all test cases should pass. |
|
@grivera64 Will check tomorrow, it's a repo repo so it only allows maintainers to push to it. |
Sounds good, thanks! 💪 |
|
@Abhirup-99 can you overtake the change from the diff |
|
I created a copy of the PR with the patch #4014 |
6 participants
This pull request introduces support for streaming HTTP response bodies in the client, allowing responses to be read as streams rather than being fully loaded into memory. This is particularly useful for handling large responses or server-sent events. The changes include new configuration options at both the client and request levels, implementation of the streaming logic, and comprehensive tests to ensure correct behavior.
Streaming response body support:
streamResponseBodyfield to theClientstruct, along withSetStreamResponseBodyandStreamResponseBodymethods to enable or disable response body streaming at the client level (client/client.go). [1] [2]streamResponseBodyfield to theRequeststruct, with correspondingSetStreamResponseBodyandStreamResponseBodymethods to allow per-request configuration that overrides the client setting (client/request.go). [1] [2]client/core.go).BodyStreammethod on theResponsestruct to provide anio.Readerfor streaming the response body, falling back to an in-memory reader if streaming is not enabled (client/response.go).Testing and validation:
client/client_test.go,client/response_test.go). [1] [2]streamResponseBodyoverride (client/request.go).# DescriptionPlease provide a clear and concise description of the changes you've made and the problem they address. Include the purpose of the change, any relevant issues it solves, and the benefits it brings to the project. If this change introduces new features or adjustments, highlight them here.
Related #3425