Feature/tlsauth #301

ThomasVerhoeven1998 · 2025-04-08T09:01:38Z

Fixes streamnative/pulsar-resources-operator#108

Motivation

As described in the linked issue, we want to support TLS authentication in the Pulsar Resources Operator, which is already supported by Pulsar itself. This improves security and flexibility for Pulsar deployments that require encrypted and authenticated connections.

Modifications

This PR includes the following changes:

Allow configuration of only the secure URL properties (adminServiceSecureURL and brokerServiceSecureURL). Previously, it was mandatory to configure adminServiceURL.
When using secure URLs (TLS), add support for:
- Hostname verification
- Disabling certificate verification via allowInsecureConnection
Add support for TLS authentication using client certificates.
Extend the Helm chart to support volumes and volumeMounts, so users can mount TLS certificates into the operator.

Verifying this change

Make sure that the change passes the CI checks.

This change is not yet covered by tests.

End-to-end (e2e) tests would be most appropriate here. However, I currently don’t have access to a full GitHub Actions setup with a TLS-enabled Pulsar broker. I suggest we add a new e2e test that:

Spins up a Pulsar broker with TLS enabled.
Loads required certificates.
Verifies resource creation, retrieval, and deletion using TLS authentication.

I’d be happy to collaborate on this if another contributor with access to the test infrastructure can help set up the environment.

Documentation

doc – This PR includes changes that will require corresponding documentation updates, such as usage examples for the new TLS fields and Helm chart enhancements.

…treamnative#187) fix: update the PulsarTopic doc Signed-off-by: ericsyh <[email protected]>

* improve log * improve * fix lint * comment * Update pkg/connection/reconciler.go Co-authored-by: Max Xu <[email protected]> * fix go build --------- Co-authored-by: Max Xu <[email protected]>

Signed-off-by: ericsyh <[email protected]>

* update pulsar-client-go lib Signed-off-by: ericsyh <[email protected]> * update dockerfile Signed-off-by: ericsyh <[email protected]> * fix the go mod Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

Signed-off-by: ericsyh <[email protected]>

* update resource operator chart Signed-off-by: ericsyh <[email protected]> * fix lint Signed-off-by: ericsyh <[email protected]> * update Signed-off-by: ericsyh <[email protected]> * update k8s test Signed-off-by: ericsyh <[email protected]> * update appversion Signed-off-by: ericsyh <[email protected]> * update the chart version Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

* fix: add olm required labels Signed-off-by: ericsyh <[email protected]> * update gitignore Signed-off-by: ericsyh <[email protected]> * update csv Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

Signed-off-by: ericsyh <[email protected]>

…treamnative#194) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matteo Merli <[email protected]>

…e#193) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0)

* add crd for pulsar package & function & connector * add controllers * add tests * Update api/v1alpha1/zz_generated.deepcopy.go Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix ci * fix lint * fix build * fix build * fix test * fix lint * fix golint * fix install * add to charts * fix role * fix controller * fix controller * fix package * remove package test * fix ci * fix * use functions worker * revert * fix admin cli * fix * fix charts * fix license * fix client version * fix * skip function package * fix reconcile * fix ci * fix * fix pulsar * fix rep * fix url * fix tab * fix * cleanup proxy * fix container name * fix builtin * fix connectors * use pulsar-all * docker hub login * fix pulsarctl * fix narExtractionDirectory * timeout * fix name * add docs * fix license * fix chart role * fix * fix license and lint * fix lint --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Neng Lu <[email protected]>

* chore: upgrade chart and change the image registry Signed-off-by: ericsyh <[email protected]> * update to use the 0.4.12 Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

…treamnative#216) fix the reconcile order

* fix: olm release CI bug Signed-off-by: ericsyh <[email protected]> * add a test job Signed-off-by: ericsyh <[email protected]> * add dependency Signed-off-by: ericsyh <[email protected]> * remove test job Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

streamnative#219) * fix function reconcile uses incorrect status to check if update or not * fix ci to cleanup * fix style * fix * fix cleanup

* mark the configs optional for function * fix CryptoConfig * revert chart Changes

* release helm chart 0.5.1 # Conflicts: # charts/pulsar-resources-operator/Chart.yaml # charts/pulsar-resources-operator/tests/deployment_test.yaml * fix lint # Conflicts: # charts/pulsar-resources-operator/tests/deployment_test.yaml * bump to 0.5.2 * fix header

* remove the kube-rbac-proxy Signed-off-by: ericsyh <[email protected]> * update the test Signed-off-by: ericsyh <[email protected]> * fix ci Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

fix makefile gen path and noused dep

…ive#287) * allow using file:// to manage packages * add cloud storage support * fix ci * go mod tidy * fix mod * fix ci * fix ci * fix path parser * fix package * sync charts * fix lint * fix ci * fix lint

use go1.22.12

Update Chart.yaml and README.md to reflect the new pre-release version, preparing for the next iteration of the Pulsar Resources Operator chart.

fix preflight

# Conflicts: # api/v1alpha1/common.go # config/crd/bases/resource.streamnative.io_pulsarconnections.yaml # docs/pulsar_connection.md # pkg/admin/interface.go

github-actions · 2025-04-08T09:01:52Z

@ThomasVerhoeven1998:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

github-actions · 2025-04-08T09:37:19Z

@ThomasVerhoeven1998:Thanks for providing doc info!

lhotari · 2025-04-08T13:38:35Z

@freeznet Please review

freeznet

The changes lgtm, but could you please add the changes to helm charts as well, to address the Extend the Helm chart to support volumes and volumeMounts, so users can mount TLS certificates into the operator. part?

freeznet

i noticed the helm charts part are addressed in #184, I will approve this PR first, and later will spin a new PR to add e2e tests.

ThomasVerhoeven1998 · 2025-04-14T12:12:47Z

@freeznet do you want me to merge this PR in the #184 PR now? Or open a separate PR with both changes?

freeznet · 2025-04-15T08:54:45Z

@ThomasVerhoeven1998 could you please rebase the master so we could make the e2e ready to use? i have created #306 to fix the ci failure from forked repo's PR.

ThomasVerhoeven1998 · 2025-04-15T11:00:52Z

@ThomasVerhoeven1998 could you please rebase the master so we could make the e2e ready to use? i have created #306 to fix the ci failure from forked repo's PR.

Rebase done

lhotari

LGTM, good work @ThomasVerhoeven1998

ThomasVerhoeven1998 · 2025-05-23T11:20:52Z

@freeznet @lhotari Is this already released in the latest version? Cause we really want to use it

tomjo and others added 30 commits February 15, 2024 15:08

feat: host key verification

a0c85a0

feat: TLS authentication

12f5de5

fix: update the PulsarTopic doc to remove the partition limitation. (s…

088f79e

…treamnative#187) fix: update the PulsarTopic doc Signed-off-by: ericsyh <[email protected]>

fix client id mismatch (streamnative#188)

9b7f384

improve log output (streamnative#191)

b343eb2

* improve log * improve * fix lint * comment * Update pkg/connection/reconciler.go Co-authored-by: Max Xu <[email protected]> * fix go build --------- Co-authored-by: Max Xu <[email protected]>

chore: bump chart to 0.4.8 (streamnative#192)

9c712ad

Signed-off-by: ericsyh <[email protected]>

update the dockerfile (streamnative#199)

73f42ad

Signed-off-by: ericsyh <[email protected]>

fix openshift-preflight (streamnative#200)

e7cd59e

Signed-off-by: ericsyh <[email protected]>

change to use the ubi-minimal (streamnative#202)

c4ecd19

Signed-off-by: ericsyh <[email protected]>

doc: update the pulsarconnection doc and examples (streamnative#203)

ed54689

Signed-off-by: ericsyh <[email protected]>

feat: support release image to DockerHub (streamnative#205)

428afe0

Signed-off-by: ericsyh <[email protected]>

Upgrade all Alpine packages at build time (streamnative#208)

7d1e576

chore: upgrade workflow to go 1.21 (streamnative#209)

868b6d2

Signed-off-by: ericsyh <[email protected]>

Fixed APK upgrade operation in dockerfile (streamnative#213)

4b4d1bc

Upgraded golint to 1.55.2 (streamnative#212)

dc1d5e6

Upgrade protobuf to 1.34.2 (streamnative#214)

9198616

[sec] Update x/crypto to fix CVE-2023-48795 (streamnative#207)

fc807f8

chore(deps): bump golang.org/x/net from 0.17.0 to 0.23.0 (streamnativ…

faea9bd

…e#193) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0)

fix: fix the reconcile order for packages and functions / connectors (s…

46a21e4

…treamnative#216) fix the reconcile order

fix function reconcile uses incorrect status to check if update or not (

c29183d

streamnative#219) * fix function reconcile uses incorrect status to check if update or not * fix ci to cleanup * fix style * fix * fix cleanup

mark the configs optional for function (streamnative#221)

5bde161

* mark the configs optional for function * fix CryptoConfig * revert chart Changes

remove the kube-rbac-proxy (streamnative#224)

d917aaf

* remove the kube-rbac-proxy Signed-off-by: ericsyh <[email protected]> * update the test Signed-off-by: ericsyh <[email protected]> * fix ci Signed-off-by: ericsyh <[email protected]> --------- Signed-off-by: ericsyh <[email protected]>

freeznet and others added 8 commits April 8, 2025 10:00

fix makefile gen path and cleanup (streamnative#292)

ca0e373

fix makefile gen path and noused dep

allow using file:// and cloud storage to manage packages (streamnat…

20ebd42

…ive#287) * allow using file:// to manage packages * add cloud storage support * fix ci * go mod tidy * fix mod * fix ci * fix ci * fix path parser * fix package * sync charts * fix lint * fix ci * fix lint

[cves] use go1.22.12 (streamnative#293)

4b6c017

use go1.22.12

bump chart version to v0.9.1-rc.3 (streamnative#296)

1cfe7a0

Update Chart.yaml and README.md to reflect the new pre-release version, preparing for the next iteration of the Pulsar Resources Operator chart.

fix preflight when release (streamnative#299)

261eab4

fix preflight

release v0.9.1 helm chart (streamnative#298)

0f0941d

feat: TLS authentication

3474dd9

Merge remote-tracking branch 'upstream/main' into feature/tlsauth

9691ac1

# Conflicts: # api/v1alpha1/common.go # config/crd/bases/resource.streamnative.io_pulsarconnections.yaml # docs/pulsar_connection.md # pkg/admin/interface.go

ThomasVerhoeven1998 requested review from a team as code owners April 8, 2025 09:01

github-actions bot assigned ThomasVerhoeven1998 Apr 8, 2025

github-actions bot added doc-info-missing This pr needs to mark a document option in description and removed doc-info-missing This pr needs to mark a document option in description labels Apr 8, 2025

github-actions bot added the doc This pr contains a document label Apr 8, 2025

remove import

e83ee06

lhotari mentioned this pull request Apr 8, 2025

feat: TLS authentication #143

Closed

4 tasks

Merge branch 'main' into feature/tlsauth

b953c50

freeznet reviewed Apr 11, 2025

View reviewed changes

freeznet approved these changes Apr 11, 2025

View reviewed changes

Merge branch 'main' into feature/tlsauth

f6d9a90

Merge branch 'main' into feature/tlsauth

d0a227c

lhotari approved these changes Apr 16, 2025

View reviewed changes

lhotari merged commit a506176 into streamnative:main Apr 16, 2025
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/tlsauth #301

Feature/tlsauth #301

Uh oh!

ThomasVerhoeven1998 commented Apr 8, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

lhotari commented Apr 8, 2025

Uh oh!

freeznet left a comment

Uh oh!

freeznet left a comment •

edited

Loading

Uh oh!

ThomasVerhoeven1998 commented Apr 14, 2025

Uh oh!

freeznet commented Apr 15, 2025

Uh oh!

ThomasVerhoeven1998 commented Apr 15, 2025

Uh oh!

lhotari left a comment

Uh oh!

Uh oh!

ThomasVerhoeven1998 commented May 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Feature/tlsauth #301

Feature/tlsauth #301

Uh oh!

Conversation

ThomasVerhoeven1998 commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verifying this change

Documentation

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

lhotari commented Apr 8, 2025

Uh oh!

freeznet left a comment

Choose a reason for hiding this comment

Uh oh!

freeznet left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ThomasVerhoeven1998 commented Apr 14, 2025

Uh oh!

freeznet commented Apr 15, 2025

Uh oh!

ThomasVerhoeven1998 commented Apr 15, 2025

Uh oh!

lhotari left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ThomasVerhoeven1998 commented May 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ThomasVerhoeven1998 commented Apr 8, 2025 •

edited

Loading

freeznet left a comment •

edited

Loading