Enable HTTP debugging for AWS client requests #11015

runderwo · 2025-10-13T17:48:23Z

Previously, there no connection from the HTTP client created by the AWS library to the built-in HTTP debugging callbacks. This change creates that path, so that HTTP traffic created by AWS HTTP clients can be debugged in the usual way (by enabling HTTP debugging at build time and supplying config variables in the output block).

Without this change there is zero visibility into AWS HTTP sessions even when HTTP debugging is enabled, which violates the principle of least surprise to the user, and also as a result it's difficult to get help from AWS support because they tend to blame the client rather than the service until malformed requests are ruled out.

Testing
Before we can approve your change; please submit the following in a comment:

Example configuration file for the change

To enable HTTP debugging output for an AWS service, add one or more of the following to an output section:

      _debug.http.request_headers: true
      _debug.http.request_payload: true
      _debug.http.response_headers: true
      _debug.http.response_payload: true

Debug log output from testing the change

Relevant log snippet (sanitized):

[2025/10/13 16:06:09] [debug] [output:kinesis_firehose:kinesis_firehose.XXX] firehose:PutRecordBatch: events=5, payload=2729 bytes
[2025/10/13 16:06:09] [debug] [output:kinesis_firehose:kinesis_firehose.XXX] Sending log records to delivery stream XXX
[2025/10/13 16:06:09] [debug] [upstream] KA connection #569 to us-east-1.aws.x.y.z:15173 is connected
[2025/10/13 16:06:09] [debug] [http_client] not using http_proxy for header
[2025/10/13 16:06:09] [debug] [http] request headers
POST / HTTP/1.1
Host: us-east-1.aws.x.y.z:1234
Content-Length: 2729
User-Agent: aws-fluent-bit-plugin
Content-Type: application/x-amz-json-1.1
X-Amz-Target: Firehose_20150804.PutRecordBatch
x-amz-date: 20251013T160609Z
Authorization: AWS4-HMAC-SHA256 Credential=XXX
Connection: keep-alive


[2025/10/13 16:06:09] [debug] [http] request payload (2729 bytes)
{"DeliveryStreamName":"atlas-kinesis-splunk","Records":[{"Data":"<snip>"},{"Data":"<snip>"},{"Data":"<snip>"},{"Data":"<snip>"}]}
[2025/10/13 16:06:09] [debug] [http] response headers
HTTP/1.1 200 OK
x-amzn-RequestId: xxx
x-amz-id-2: yyy
Content-Type: application/x-amz-json-1.1
Content-Length: 1259
Date: Mon, 13 Oct 2025 16:06:09 GMT


[2025/10/13 16:06:09] [debug] [http] response payload (1259 bytes)
{"Encrypted":false,"FailedPutCount":0,"RequestResponses":[{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"}]}

Attached Valgrind output that shows no leaks or memory corruption was found

==362092== 
==362092== HEAP SUMMARY:
==362092==     in use at exit: 13,874 bytes in 49 blocks
==362092==   total heap usage: 247,903 allocs, 247,854 frees, 58,923,676 bytes allocated
==362092== 
==362092== LEAK SUMMARY:
==362092==    definitely lost: 0 bytes in 0 blocks
==362092==    indirectly lost: 0 bytes in 0 blocks
==362092==      possibly lost: 0 bytes in 0 blocks
==362092==    still reachable: 13,874 bytes in 49 blocks
==362092==         suppressed: 0 bytes in 0 blocks
==362092== Reachable blocks (those to which a pointer was found) are not shown.
==362092== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==362092== 
==362092== For lists of detected and suppressed errors, rerun with: -s
==362092== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Documentation

Documentation required for this feature

I can't tell if the HTTP debug callback configuration keys are documented somewhere; I had to find them and guess how to use them by reading code.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

New Features
- Added a callback context to the AWS client to enable richer HTTP request handling and debug hooks.
Refactor
- Integrated the AWS client callback into the HTTP request flow and output plugin debug setup.
Bug Fixes
- Fail-fast behavior when callback setup during AWS client initialization fails.
Chores
- Ensure proper cleanup of the callback resource during client teardown.

coderabbitai · 2025-10-13T17:50:58Z

Walkthrough

Added a public http_cb_ctx pointer to struct flb_aws_client. Creation now allocates this callback and returns NULL on allocation failure. Requests propagate the callback to the HTTP client. The callback is destroyed on client teardown. Firehose plugin calls HTTP client debug setup during init.

Changes

Cohort / File(s)	Summary
Public AWS client struct update `include/fluent-bit/flb_aws_util.h`	Added field `struct flb_callback *http_cb_ctx;` to `struct flb_aws_client` (placed after `debug_only`).
AWS client lifecycle and request wiring `src/aws/flb_aws_util.c`	`flb_aws_client_create()` initializes `client->http_cb_ctx` via `flb_callback_create("aws client")` and returns NULL on failure; `flb_aws_client_destroy()` calls `flb_callback_destroy()` on `http_cb_ctx`; `request_do()` assigns `c->cb_ctx = aws_client->http_cb_ctx` before using the HTTP client.
Firehose plugin: HTTP debug setup `plugins/out_kinesis_firehose/firehose.c`	Added `#include` for `flb_http_client_debug.h`; during `cb_firehose_init` calls `flb_http_client_debug_setup(ctx->firehose_client->http_cb_ctx, &ins->properties)` and logs an error / aborts init on negative return.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant App as Caller
  participant AWS as flb_aws_client_create()
  participant CB as flb_callback_create()
  participant CL as flb_aws_client

  App->>AWS: create()
  AWS->>CB: flb_callback_create("aws client")
  alt Callback created
    CB-->>AWS: flb_callback*
    AWS-->>App: flb_aws_client* (with http_cb_ctx)
  else Creation fails
    CB-->>AWS: NULL
    AWS-->>App: NULL (early return)
  end

sequenceDiagram
  autonumber
  participant CL as flb_aws_client
  participant RQ as request_do()
  participant HC as HTTP Client

  CL->>RQ: make request
  RQ->>RQ: c->cb_ctx = CL.http_cb_ctx
  RQ->>HC: init/send with cb_ctx
  HC-->>RQ: response / error
  RQ-->>CL: result

sequenceDiagram
  autonumber
  participant App as Caller
  participant D as flb_aws_client_destroy()
  participant CL as flb_aws_client

  App->>D: destroy(CL)
  D->>CL: flb_callback_destroy(CL->http_cb_ctx)
  D-->>App: done

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

I twitch my whiskers, find a hook,
A tiny callback joins the book.
It hops into requests with cheer,
If creation fails, we stop here.
I tidy up and bound away—hooray! 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The PR title "Enable HTTP debugging for AWS client requests" directly and accurately reflects the main objective of the changeset. The modifications add HTTP callback context support to the AWS client library and integrate HTTP debugging setup in the Kinesis Firehose plugin, enabling users to capture and debug AWS service HTTP traffic through Fluent Bit's built-in HTTP debugging mechanism. The title is concise, specific, and clearly conveys the primary change without vague terminology or unnecessary details. A teammate scanning the repository history would immediately understand that this change introduces HTTP debugging capabilities for AWS requests.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2b16b24 and 7e4d2c2.

📒 Files selected for processing (2)

include/fluent-bit/flb_aws_util.h (1 hunks)
src/aws/flb_aws_util.c (3 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

src/aws/flb_aws_util.c (1)

include/fluent-bit/flb_mem.h (1)

flb_free (126-128)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)

GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
GitHub Check: pr-compile-centos-7
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-without-cxx (3.31.6)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
GitHub Check: PR - fuzzing test

🔇 Additional comments (2)

include/fluent-bit/flb_aws_util.h (1)

105-107: LGTM!

The new callback field is appropriately placed in the struct with a clear comment describing its purpose.

src/aws/flb_aws_util.c (1)

395-396: LGTM!

The callback is correctly wired into the HTTP client context for debugging purposes.

src/aws/flb_aws_util.c

edsiper · 2025-10-13T20:02:35Z

why is this change needed ?
what's the end-user outcome ?
fix DCO (sign-off)

Signed-off-by: Ryan Underwood <[email protected]>

It's conditional on a build flag, so it's no-op by default. Signed-off-by: Ryan Underwood <[email protected]>

coderabbitai

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7ac03a2 and 62f98ed.

📒 Files selected for processing (1)

plugins/out_kinesis_firehose/firehose.c (2 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)

GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-without-cxx (3.31.6)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-centos-7
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
GitHub Check: PR - fuzzing test

plugins/out_kinesis_firehose/firehose.c

Signed-off-by: Ryan Underwood <[email protected]>

cosmo0920

I added a nitpick comment.
The logic of PR is reasonable so we only need to follow contributor guideline.

src/aws/flb_aws_util.c

runderwo · 2025-10-21T18:35:44Z

I added a nitpick comment. The logic of PR is reasonable so we only need to follow contributor guideline.

Resolved that since the client name was considered acceptable.

Signed-off-by: Ryan Underwood <[email protected]>

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 909b105 and 4daa282.

📒 Files selected for processing (1)

src/aws/flb_aws_util.c (3 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

src/aws/flb_aws_util.c (1)

include/fluent-bit/flb_mem.h (1)

flb_free (126-128)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)

GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
GitHub Check: pr-compile-without-cxx (3.31.6)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
GitHub Check: pr-compile-centos-7
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
GitHub Check: PR - fuzzing test

🔇 Additional comments (2)

src/aws/flb_aws_util.c (2)

271-276: LGTM! Proper error handling for callback creation.

The callback creation with error handling is correct. On failure, the code properly calls flb_errno(), frees the allocated client, and returns NULL, preventing memory leaks.

396-397: LGTM! Correct propagation of callback context.

The callback context is properly propagated to the HTTP client, enabling HTTP debugging for AWS requests as intended by this PR.

src/aws/flb_aws_util.c

runderwo · 2025-10-21T18:55:40Z

@cosmo0920 I'm unable to re-request review through the UI for some reason, so please re-review.

edsiper · 2025-11-06T22:02:32Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2025-11-06T22:06:11Z

src/aws/flb_aws_util.c

        if (aws_client->extra_user_agent) {
            flb_sds_destroy(aws_client->extra_user_agent);
        }
+        flb_callback_destroy(aws_client->http_cb_ctx);
        flb_free(aws_client);


Guard destroying AWS client HTTP callback context

When tearing down an AWS client the code now calls flb_callback_destroy(aws_client->http_cb_ctx) unconditionally. Several call sites (e.g. the credential provider unit tests’ custom client generators) allocate struct flb_aws_client via flb_calloc and never populate http_cb_ctx, yet still invoke flb_aws_client_destroy. Because flb_callback_destroy dereferences its argument, passing a NULL pointer will segfault during test runs or in any external code that injects a lightweight mock client. A null check before destroying the callback context avoids the crash while preserving existing behaviour.

Useful? React with 👍 / 👎.

@runderwo We need to wrap up with existence checking due to there's a possibility to be absent for registering aws_client->http_cb_ctx callbacks. This is because this type of register couldn't be caught up on dry_run or unit testing. So, we need to use a guard like a pessimistic way.

cosmo0920 · 2025-11-07T08:13:38Z

plugins/out_kinesis_firehose/firehose.c

    ctx->firehose_client->static_headers_len = 1;
+    if (flb_http_client_debug_setup(ctx->firehose_client->http_cb_ctx, &ins->properties) < 0) {


flb_http_client_debug_setup is not always existing in Fluent Bit core.
This is because only if enabled for -DFLB_HTTP_CLIENT_DEBUG=ON on configure.
So, we need to wrap up with

#ifdef FLB_HAVE_HTTP_CLIENT_DEBUG if (flb_http_client_debug_setup(ctx->firehose_client->http_cb_ctx, &ins->properties) < 0) { flb_plg_error(ctx->ins, "AWS HTTP client debug initialization error"); goto error; } #endif

here.

runderwo requested review from PettitWesley, celalettin1286, edsiper, fujimotos, koleini, niedbalski and patrick-stephens as code owners October 13, 2025 17:48

github-actions bot added the docs-required label Oct 13, 2025

runderwo temporarily deployed to pr October 13, 2025 17:48 — with GitHub Actions Inactive

runderwo temporarily deployed to pr October 13, 2025 17:52 — with GitHub Actions Inactive

coderabbitai bot reviewed Oct 13, 2025

View reviewed changes

src/aws/flb_aws_util.c Outdated Show resolved Hide resolved

src/aws/flb_aws_util.c Outdated Show resolved Hide resolved

src/aws/flb_aws_util.c Outdated Show resolved Hide resolved

runderwo temporarily deployed to pr October 13, 2025 18:12 — with GitHub Actions Inactive

Enable HTTP debugging for AWS client requests

3863eea

Signed-off-by: Ryan Underwood <[email protected]>

runderwo force-pushed the enable-aws-http-debug branch from 7e4d2c2 to 3863eea Compare October 14, 2025 09:13

runderwo temporarily deployed to pr October 14, 2025 09:13 — with GitHub Actions Inactive

runderwo temporarily deployed to pr October 14, 2025 09:19 — with GitHub Actions Inactive

Apply code quality changes

7ac03a2

Signed-off-by: Ryan Underwood <[email protected]>

runderwo temporarily deployed to pr October 14, 2025 09:40 — with GitHub Actions Inactive

Enable HTTP debug in Firehose client for illustration

62f98ed

It's conditional on a build flag, so it's no-op by default. Signed-off-by: Ryan Underwood <[email protected]>

runderwo had a problem deploying to pr October 14, 2025 11:18 — with GitHub Actions Failure

coderabbitai bot reviewed Oct 14, 2025

View reviewed changes

plugins/out_kinesis_firehose/firehose.c Outdated Show resolved Hide resolved

plugins/out_kinesis_firehose/firehose.c Show resolved Hide resolved

Code quality fixes

e27feac

Signed-off-by: Ryan Underwood <[email protected]>

runderwo had a problem deploying to pr October 14, 2025 11:25 — with GitHub Actions Failure

Code quality fixes

909b105

Signed-off-by: Ryan Underwood <[email protected]>

runderwo had a problem deploying to pr October 14, 2025 11:39 — with GitHub Actions Failure

cosmo0920 requested changes Oct 20, 2025

View reviewed changes

src/aws/flb_aws_util.c Outdated Show resolved Hide resolved

runderwo had a problem deploying to pr October 21, 2025 18:35 — with GitHub Actions Failure

Remove fixme

4daa282

Signed-off-by: Ryan Underwood <[email protected]>

runderwo force-pushed the enable-aws-http-debug branch from 3246c32 to 4daa282 Compare October 21, 2025 18:36

runderwo had a problem deploying to pr October 21, 2025 18:36 — with GitHub Actions Failure

runderwo requested a review from cosmo0920 October 21, 2025 18:38

coderabbitai bot reviewed Oct 21, 2025

View reviewed changes

src/aws/flb_aws_util.c Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Nov 6, 2025

View reviewed changes

cosmo0920 requested changes Nov 7, 2025

View reviewed changes

		ctx->firehose_client->static_headers_len = 1;
		if (flb_http_client_debug_setup(ctx->firehose_client->http_cb_ctx, &ins->properties) < 0) {

Enable HTTP debugging for AWS client requests #11015

Are you sure you want to change the base?

Enable HTTP debugging for AWS client requests #11015

Conversation

runderwo commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edsiper commented Oct 13, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cosmo0920 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

runderwo commented Oct 21, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

runderwo commented Oct 21, 2025

Uh oh!

edsiper commented Nov 6, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

cosmo0920 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

cosmo0920 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

runderwo commented Oct 13, 2025 •

edited

Loading

coderabbitai bot commented Oct 13, 2025 •

edited

Loading