Skip to content

s390x binaries included in librdkafka.redist NuGet package and static builds package#5365

Open
pranav shah (prashah-confluent) wants to merge 15 commits intomasterfrom
dev_prashah_s390x_glibc
Open

s390x binaries included in librdkafka.redist NuGet package and static builds package#5365
pranav shah (prashah-confluent) wants to merge 15 commits intomasterfrom
dev_prashah_s390x_glibc

Conversation

@prashah-confluent
Copy link
Copy Markdown
Member

No description provided.

@confluent-cla-assistant
Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_prashah_s390x_glibc branch 4 times, most recently from afe3d09 to 095bcb8 Compare March 21, 2026 07:30
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_prashah_s390x_glibc branch 3 times, most recently from 9b1844d to c066649 Compare March 22, 2026 03:19
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_prashah_s390x_glibc branch from 8270eb8 to 4edace1 Compare March 22, 2026 14:04
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_prashah_s390x_glibc branch from d736613 to 15165b8 Compare April 9, 2026 09:36
@emasab Emanuele Sabellico (emasab) marked this pull request as ready for review April 9, 2026 09:37
@emasab Emanuele Sabellico (emasab) requested a review from a team as a code owner April 9, 2026 09:37
Copilot AI review requested due to automatic review settings April 9, 2026 09:37
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bumps librdkafka to v2.14.1 and extends the release/packaging pipeline to produce and bundle prebuilt Linux s390x (glibc/centos8 manylinux) artifacts.

Changes:

  • Bump project/version constants to 2.14.1 (vcpkg + public headers) and add a v2.14.1 changelog entry.
  • Add --platform support to the release-artifact docker build script and install an extra CentOS Perl dependency.
  • Add Semaphore CI jobs and NuGet/static package mappings for linux-s390x (glibc/centos8) artifacts.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
vcpkg.json Bumps vcpkg manifest version to 2.14.1.
src/rdkafka.h Updates RD_KAFKA_VERSION macro for 2.14.1.
src-cpp/rdkafkacpp.h Updates C++ header RD_KAFKA_VERSION macro for 2.14.1.
packaging/tools/build-release-artifacts.sh Adds docker --platform option support and tweaks CentOS deps.
packaging/nuget/staticpackage.py Adds static bundle mappings for glibc linux s390x.
packaging/nuget/nugetpackage.py Adds NuGet runtime mapping for glibc linux s390x.
CHANGELOG.md Adds v2.14.1 entry noting linux-s390x binaries.
.semaphore/semaphore.yml Adds linux-s390x docker build block and wires it into packaging dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

fi

docker_platform=""
if [ "$1" = "--platform" ]; then
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--platform is parsed as --platform=$2 and then shift 2 without validating that a value was actually provided. If someone calls the script with --platform but no argument, this will fail with a generic shift error rather than a clear usage message. Add an explicit check (e.g., ensure $# -ge 2 and $2 is non-empty) and print usage/exit when the argument is missing.

Suggested change
if [ "$1" = "--platform" ]; then
if [ "$1" = "--platform" ]; then
if [ $# -lt 2 ] || [ -z "$2" ]; then
echo "Usage: $0 [--disable-gssapi] [--platform <docker-platform>] <manylinux-docker-image> <output-path.tgz>"
exit 1
fi

Copilot uses AI. Check for mistakes.
Comment on lines +94 to +101
# Linux glibc centos8 s390x without GSSAPI (no external deps)
Mapping({'arch': 's390x',
'plat': 'linux',
'dist': 'centos8',
'lnk': 'all'},
'librdkafka.tgz',
'./usr/local/lib/librdkafka.so.1',
'runtimes/linux-s390x/native/librdkafka.so'),
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a linux/s390x runtime mapping, but packaging/nuget/packaging.py's magic_patterns currently has ELF checks for linux x64 and arm64 only. That means the s390x .so won't be validated for architecture during packaging, reducing defense-in-depth against accidentally bundling the wrong binary. Consider adding a corresponding magic_patterns entry for (linux, s390x, .so).

Copilot uses AI. Check for mistakes.
prologue:
commands:
- '[[ -z $DOCKERHUB_APIKEY ]] || docker login --username $DOCKERHUB_USER --password $DOCKERHUB_APIKEY'
- docker run --privileged --rm tonistiigi/binfmt --install s390x
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow installs QEMU binfmt using tonistiigi/binfmt without a tag or digest, which makes the build non-reproducible and risks sudden CI breakage if the upstream image changes. Pin this image to a specific version tag or digest (similar to how the manylinux images are pinned).

Suggested change
- docker run --privileged --rm tonistiigi/binfmt --install s390x
- docker run --privileged --rm tonistiigi/binfmt:qemu-v8.1.5 --install s390x

Copilot uses AI. Check for mistakes.
@airlock-confluentinc airlock-confluentinc bot force-pushed the dev_prashah_s390x_glibc branch from f5c7769 to 66085fb Compare April 9, 2026 09:54
@emasab Emanuele Sabellico (emasab) changed the title Dev prashah s390x glibc s390x binaries included in librdkafka.redist NuGet package and static builds package Apr 9, 2026
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants