kafka: fix cmake cross compile error #9600

ThomasDevoogdt · 2024-11-16T18:52:32Z

KAFKA_INCLUDEDIR is not set if FLB_PREFER_SYSTEM_LIB_KAFKA is not used, when cross-compiling, it just translates to -I/librdkafka, which is not allowed. Fix this by only including KAFKA_INCLUDEDIR if really set.

x86_64-linux-gcc: ERROR: unsafe header/library path used in cross-compilation: '-I/librdkafka'

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

[N/A] Example configuration file for the change
[N/A] Debug log output from testing the change

[N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

[N/A] Run local packaging test showing all targets (including any new ones) build.
[N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

[N/A] Documentation required for this feature

Backporting

[N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

Bug Fixes
- Improved build stability for Kafka input and output plugins by avoiding invalid header include paths when Kafka headers aren’t configured, reducing build errors in misconfigured environments.
Chores
- Hardened build configuration for Kafka plugins to conditionally apply include paths only when appropriate, keeping behavior unchanged for correctly configured setups.

ThomasDevoogdt · 2024-12-15T14:16:55Z

@edsiper It has been a while. Any chance that this gets approved? I try to reduce my own patch list.

ThomasDevoogdt · 2025-01-01T19:22:57Z

@edsiper Happy New Year! Can you have a look at this PR please?

ThomasDevoogdt · 2025-01-10T06:57:42Z

@edsiper @leonardo-albertovich @fujimotos @koleini This PR is open for a while. Is it possible to have a look?

ThomasDevoogdt · 2025-03-24T21:18:33Z

@edsiper @patrick-stephens @leonardo-albertovich ping

patrick-stephens · 2025-03-26T13:38:20Z

Can we rebase to see if it resolves the unit test failures?

ThomasDevoogdt · 2025-03-26T14:08:32Z

Can we rebase to see if it resolves the unit test failures?

I rebased it.

ThomasDevoogdt · 2025-03-26T15:28:55Z

Can we rebase to see if it resolves the unit test failures?

@patrick-stephens rebased it, and all checks are passing.

ThomasDevoogdt · 2025-03-26T16:11:32Z

@patrick-stephens what is the next step to get this merged? I also have other open PRs which are approved, but not merged yet. Can you check my other PRs also? https://github.com/fluent/fluent-bit/pulls/ThomasDevoogdt

patrick-stephens · 2025-03-27T10:34:48Z

@edsiper will be merging when he is happy to put it in a release - be aware we're finalising the 4.0 release now.

ThomasDevoogdt · 2025-04-01T20:19:18Z

@edsiper will be merging when he is happy to put it in a release - be aware we're finalising the 4.0 release now.

@patrick-stephens @edsiper I see that 4.0.0 has landed, this is the ideal moment to consider this PR.

ThomasDevoogdt · 2025-04-30T11:32:37Z

@patrick-stephens can you also check this one, while at it?

patrick-stephens · 2025-04-30T11:37:50Z

Looks fine to me, just want to check @edsiper is happy with the commit title

ThomasDevoogdt · 2025-08-04T11:59:16Z

@patrick-stephens @edsiper It's again a bit silent around this PR. Everything is approved and ready to be merged, on what do we exactly wait? Can you just merge this one?

ThomasDevoogdt · 2025-08-07T21:25:00Z

@cosmo0920 Can you put this PR back on another more milestone, I have the feeling that next means forgotten. Or just merge this one.

coderabbitai · 2025-08-31T11:42:33Z

Walkthrough

Applied conditional guards around librdkafka include directories in Kafka input and output plugin CMake configurations, executing target_include_directories only when KAFKA_INCLUDEDIR is defined. Other build settings remain unchanged.

Changes

Cohort / File(s)	Change Summary
Kafka CMake include guards `plugins/in_kafka/CMakeLists.txt`, `plugins/out_kafka/CMakeLists.txt`	Wrap `target_include_directories(... ${KAFKA_INCLUDEDIR}/librdkafka)` with `if(DEFINED KAFKA_INCLUDEDIR)` to avoid invalid include paths when the variable is unset; no other configuration changes.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

I nibbled CMake with careful delight,
Wrapped paths in guards to keep builds light.
No stray “/librdkafka” to roam,
Only defined trails lead us home.
Ears up, I hop—green checks in sight! 🐇✨

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

Visit our Status Page to check the current availability of CodeRabbit.
Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

MCP integration is disabled by default for public repositories
Jira integration is disabled by default for public repositories
Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 1132279 and 3a1a13e.

📒 Files selected for processing (2)

plugins/in_kafka/CMakeLists.txt (1 hunks)
plugins/out_kafka/CMakeLists.txt (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (62)

GitHub Check: PR - container builds / Windows container images (2025)
GitHub Check: PR - container builds / Windows container images (2022)
GitHub Check: PR - packages build Linux / debian/buster.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / ubuntu/24.04.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / debian/bookworm package build and stage to S3
GitHub Check: PR - packages build Linux / debian/bullseye package build and stage to S3
GitHub Check: PR - packages build Linux / almalinux/9.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / ubuntu/22.04 package build and stage to S3
GitHub Check: PR - packages build Linux / raspbian/bookworm package build and stage to S3
GitHub Check: PR - packages build Linux / ubuntu/24.04 package build and stage to S3
GitHub Check: PR - packages build Linux / ubuntu/22.04.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / debian/bullseye.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / almalinux/9 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/8.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / debian/buster package build and stage to S3
GitHub Check: PR - packages build Linux / debian/bookworm.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / rockylinux/8 package build and stage to S3
GitHub Check: PR - packages build Linux / almalinux/8.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / almalinux/8 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/9 package build and stage to S3
GitHub Check: PR - packages build Linux / rockylinux/8.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / amazonlinux/2023.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / amazonlinux/2023 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/8 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/9.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / rockylinux/9 package build and stage to S3
GitHub Check: PR - packages build Linux / amazonlinux/2 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/7 package build and stage to S3
GitHub Check: PR - packages build Linux / amazonlinux/2.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / rockylinux/9.arm64v8 package build and stage to S3
GitHub Check: PR - packages build Linux / centos/7.arm64v8 package build and stage to S3
GitHub Check: PR - container builds / amd64/production container image build
GitHub Check: PR - container builds / arm/v7/production container image build
GitHub Check: PR - container builds / arm/v7/debug container image build
GitHub Check: PR - container builds / arm64/production container image build
GitHub Check: PR - container builds / arm64/debug container image build
GitHub Check: PR - container builds / amd64/debug container image build
GitHub Check: PR - packages build MacOS / call-build-macos-package (Intel macOS runner, macos-14-large, 3.31.6)
GitHub Check: PR - packages build MacOS / call-build-macos-package (Apple Silicon macOS runner, macos-14, 3.31.6)
GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)

plugins/in_kafka/CMakeLists.txt

plugins/out_kafka/CMakeLists.txt

ThomasDevoogdt · 2025-08-31T12:33:08Z

@cosmo0920 This PR starts to age, can you pick this up? I would like to have it merged.

patrick-stephens · 2025-09-12T11:11:15Z

It looks like we're hitting some older compiler failures:

/tmp/fluent-bit/include/fluent-bit/flb_simd.h:60:18: error: missing binary operator before token "("
 #if __has_include(<arm_neon.h>)

cosmo0920 · 2025-09-12T14:44:56Z

We need to rebase off the current master.

KAFKA_INCLUDEDIR is not set if FLB_PREFER_SYSTEM_LIB_KAFKA is not used, when cross-compiling, it just translates to -I/librdkafka, which is not allowed. Fix this by only including KAFKA_INCLUDEDIR if really set. x86_64-linux-gcc: ERROR: unsafe header/library path used in cross-compilation: '-I/librdkafka' Signed-off-by: Thomas Devoogdt <[email protected]>

ThomasDevoogdt · 2025-09-12T14:54:16Z

We need to rebase off the current master.

Done. Please also merge it now, to avoid having to rebase it over and over again.

patrick-stephens · 2025-09-12T16:29:23Z

@edsiper I know it's your birthday but are we ok to merge? :)

ThomasDevoogdt · 2025-09-19T13:49:42Z

@patrick-stephens @edsiper Upstream buildroot ships now with 4.0.9 (buildroot/buildroot@39afca7) 🎉 🎉. It would be nice to get this merged, to further reduce the custom patch set in buildroot. Perhaps in 4.0.11?

ThomasDevoogdt · 2025-10-01T18:44:37Z

@edsiper can you merge this one?

ThomasDevoogdt · 2025-10-25T20:16:50Z

@patrick-stephens do you mind to just merge this PR? Otherwise it will just be open forever.

cosmo0920 · 2025-10-27T07:32:16Z

@edsiper Couldn't we include this change in Fluent Bit 4.2?
This change was postponed due to CI instabilities of 4.0 and 4.1 releasing processes.
But all of the CI results are green and there is no side effect within this patch, imo.

patrick-stephens · 2025-11-05T13:09:09Z

I'll merge as seems low risk and we can always revert

ThomasDevoogdt · 2025-11-05T18:56:39Z

Thanks!

This is a major release that introduces new features, including one highlighted in the release notes: "The v4.2 release introduces a powerful new Direct Routing capability that allows inputs to specify routes directly to outputs, bypassing the traditional routing mechanism." But it also brings some security fixes. Not all of them are relevant, but some piece of lecture can be found here [1]. It fixes the following CVEs: CVE-2025-12977 CVE-2025-12978 CVE-2025-12972 CVE-2025-12970 CVE-2025-12969 News: - https://fluentbit.io/announcements/v4.1.0/ - https://fluentbit.io/announcements/v4.1.1/ - https://fluentbit.io/announcements/v4.2.0/ The patch 0001-plugins-kafka-fix-cmake-cross-compile-error.patch can be dropped as it has been merged upstream [2]. [1] https://www.theregister.com/2025/11/24/fluent_bit_cves/ [2] fluent/fluent-bit#9600 Signed-off-by: Thomas Devoogdt <[email protected]> Signed-off-by: Thomas Petazzoni <[email protected]>

This is a major release that introduces new features, including one highlighted in the release notes: "The v4.2 release introduces a powerful new Direct Routing capability that allows inputs to specify routes directly to outputs, bypassing the traditional routing mechanism." But it also brings some security fixes. Not all of them are relevant, but some piece of lecture can be found here [1]. It fixes the following CVEs: CVE-2025-12977 CVE-2025-12978 CVE-2025-12972 CVE-2025-12970 CVE-2025-12969 News: - https://fluentbit.io/announcements/v4.1.0/ - https://fluentbit.io/announcements/v4.1.1/ - https://fluentbit.io/announcements/v4.2.0/ The patch 0001-plugins-kafka-fix-cmake-cross-compile-error.patch can be dropped as it has been merged upstream [2]. [1] https://www.theregister.com/2025/11/24/fluent_bit_cves/ [2] fluent/fluent-bit#9600 Signed-off-by: Thomas Devoogdt <[email protected]> Signed-off-by: Thomas Petazzoni <[email protected]> (cherry picked from commit 7a037d0) Signed-off-by: Thomas Perale <[email protected]>

ThomasDevoogdt requested review from edsiper, fujimotos, koleini and leonardo-albertovich as code owners November 16, 2024 18:52

github-actions bot added the docs-required label Nov 16, 2024

ThomasDevoogdt changed the title ~~fix kafka cross compile error~~ kafka: fix cross compile error Nov 16, 2024

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from b398800 to c96d616 Compare November 17, 2024 08:01

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from c96d616 to abc06b7 Compare December 15, 2024 14:16

ThomasDevoogdt mentioned this pull request Dec 18, 2024

cprofiles: can't compile due to incompatible-pointer-types #9601

Closed

ThomasDevoogdt mentioned this pull request Jan 9, 2025

build: use the system provided libmsgpack and libsqlite3 if found #9572

Merged

ThomasDevoogdt changed the title ~~kafka: fix cross compile error~~ lib: kafka: fix cross compile error Jan 10, 2025

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from abc06b7 to 5c51513 Compare January 10, 2025 07:14

patrick-stephens added the ok-package-test Run PR packaging tests label Jan 13, 2025

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from 5c51513 to 18017d7 Compare March 26, 2025 14:08

patrick-stephens approved these changes Mar 26, 2025

View reviewed changes

ThomasDevoogdt mentioned this pull request Apr 9, 2025

Fluent Bit v4.0.1 in Buildroot?? - Need support! #10190

Closed

patrick-stephens approved these changes Apr 30, 2025

View reviewed changes

ThomasDevoogdt mentioned this pull request May 5, 2025

build: only compile simdutf if there is CXX support #10277

Merged

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from fb46e10 to dcecf8f Compare July 19, 2025 13:35

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from dcecf8f to 3a1a13e Compare August 31, 2025 11:42

coderabbitai bot reviewed Aug 31, 2025

View reviewed changes

plugins/in_kafka/CMakeLists.txt Show resolved Hide resolved

plugins/out_kafka/CMakeLists.txt Show resolved Hide resolved

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from 3a1a13e to f042b7e Compare September 10, 2025 08:24

ThomasDevoogdt force-pushed the bugfix/cross-compile-kafka branch from f042b7e to 2ea0c01 Compare September 12, 2025 14:53

patrick-stephens approved these changes Sep 12, 2025

View reviewed changes

patrick-stephens approved these changes Sep 19, 2025

View reviewed changes

patrick-stephens changed the title ~~plugins: kafka: fix cmake cross compile error~~ kafka: fix cmake cross compile error Sep 19, 2025

patrick-stephens assigned edsiper Sep 19, 2025

cosmo0920 approved these changes Oct 27, 2025

View reviewed changes

patrick-stephens merged commit 7eab11a into fluent:master Nov 5, 2025
116 of 117 checks passed

ThomasDevoogdt deleted the bugfix/cross-compile-kafka branch November 5, 2025 18:56

BrewTestBot mentioned this pull request Nov 12, 2025

fluent-bit 4.2.0 Homebrew/homebrew-core#254082

Merged

kafka: fix cmake cross compile error #9600

kafka: fix cmake cross compile error #9600

Conversation

ThomasDevoogdt commented Nov 16, 2024 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

ThomasDevoogdt commented Dec 15, 2024

Uh oh!

ThomasDevoogdt commented Jan 1, 2025

Uh oh!

ThomasDevoogdt commented Jan 10, 2025

Uh oh!

ThomasDevoogdt commented Mar 24, 2025

Uh oh!

patrick-stephens commented Mar 26, 2025

Uh oh!

ThomasDevoogdt commented Mar 26, 2025

Uh oh!

ThomasDevoogdt commented Mar 26, 2025

Uh oh!

ThomasDevoogdt commented Mar 26, 2025

Uh oh!

patrick-stephens commented Mar 27, 2025

Uh oh!

ThomasDevoogdt commented Apr 1, 2025

Uh oh!

ThomasDevoogdt commented Apr 30, 2025

Uh oh!

patrick-stephens commented Apr 30, 2025

Uh oh!

ThomasDevoogdt commented Aug 4, 2025

Uh oh!

ThomasDevoogdt commented Aug 7, 2025

Uh oh!

coderabbitai bot commented Aug 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Status, Documentation and Community

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ThomasDevoogdt commented Aug 31, 2025

Uh oh!

patrick-stephens commented Sep 12, 2025

Uh oh!

cosmo0920 commented Sep 12, 2025

Uh oh!

ThomasDevoogdt commented Sep 12, 2025

Uh oh!

patrick-stephens commented Sep 12, 2025

Uh oh!

ThomasDevoogdt commented Sep 19, 2025

Uh oh!

ThomasDevoogdt commented Oct 1, 2025

Uh oh!

ThomasDevoogdt commented Oct 25, 2025

Uh oh!

cosmo0920 commented Oct 27, 2025

Uh oh!

patrick-stephens commented Nov 5, 2025

Uh oh!

Uh oh!

ThomasDevoogdt commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

ThomasDevoogdt commented Nov 16, 2024 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Aug 31, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)