Dbz 9419 add connection validator pulsar#295
Conversation
|
Hi @pxcamus, thanks for your contribution. Please prefix the commit message(s) with the debezium/dbz#xxx GitHub issue key. |
|
Hi @pxcamus thanks for the PR. Could you please prefix the commit messages with |
3d17fb9 to
1ef50b0
Compare
| }, | ||
| { | ||
| "type": "KAFKA", | ||
| "type": "APACHE_PULSAR", |
...n/java/io/debezium/platform/environment/connection/destination/pulsar/PulsarAuthHandler.java
Show resolved
Hide resolved
.../java/io/debezium/platform/environment/connection/destination/PulsarConnectionValidator.java
Show resolved
Hide resolved
| // private PulsarConnectionValidator validator; | ||
| // | ||
| // @BeforeEach | ||
| // void setUp() { | ||
| // validator = new PulsarConnectionValidator(DEFAULT_30_SECONDS_TIMEOUT); | ||
| // } |
There was a problem hiding this comment.
If this isn't needed, let's remove it.
| // ConnectionValidationResult result = validator.validate(connection); | ||
|
|
||
| // assertTrue(result.valid(), "Connection validation should succeed"); |
There was a problem hiding this comment.
Same, remove if its no longer needed.
There was a problem hiding this comment.
I finally got around to fix the tests. I will create a PulsarConnectionValidatorTest.java very soon as well, and hoping later today to add more tests for the different auth handlers.
|
@pxcamus Could you please rebase with latest main? |
528da49 to
53812e8
Compare
Done! |
...java/io/debezium/platform/environment/connection/destination/pulsar/PulsarAdminProvider.java
Outdated
Show resolved
Hide resolved
|
Hi @pxcamus, thanks for your contribution. Please prefix the commit message(s) with the debezium/dbz#xxx GitHub issue key. |
11667a4 to
f58ffe3
Compare
|
@mfvitale If you could please take a look at the approach I used for testing JWT auth. I am going to add IT tests with expired token, etc... then switch to OAuth2. Thanks! |
One small question please @mfvitale : for integration tests, should I spinnup an actual OAuth2 provider, such as Keycloak or Hydra? Or mock one to return hardcoded tokens? If using an actual provider, any preference? Thanks! |
Honestly for now I'll go with a mocked one. |
@mfvitale Should the OAuth2 connection validation include the retrieval of the token? I.e. do I request the client credentials and request a token first? My concern is that the OAuth2 provider could be un-reachable from the platform conductor, right? The other option is to just collect the token through the stage UI and validate the connection. I have a working OAuth2 provider implementation for the integration tests, so can use the actual flow. |
Naros
left a comment
There was a problem hiding this comment.
From my PoV, once the following is addressed then LGTM.
| } | ||
|
|
||
| public PulsarAuthHandler getAuthHandler(String authType) { | ||
| if (authType == null || authType.trim().isEmpty()) { |
There was a problem hiding this comment.
| if (authType == null || authType.trim().isEmpty()) { | |
| if (Strings.isNullorEmpty(authType)) { |
This is a good point. We should not only think about the validation phase but also what happens when a connection is then validated and used in the DS instance deployed by the platform. @pxcamus So for now since there is no support for loading credentials file. |
@mfvitale unless I missed something, it can be loaded as a base64 encoded string, right?
But to clarify, do I then wrap up the PR by passing the token for the validation? |
Where you get this infos? |
|
Well then it should work but is not the best from the security point, since this base64 string will be then passed in the CRD definition. Let's add the support for the validation for now. |
266eff9 to
b75c041
Compare
|
@mfvitale I am done AFAIK. One question (hopefully not too stupid): when rebasing the latest
But I cannot resolve this dependency. How do I build it locally? Thanks! |
b00b1ea to
0db866b
Compare
@mfvitale I am struggling so much to run my branch to update the ITs 😅 When trying to build the project:
But then when trying to build |
|
There are now a few cross-repo dependencies, and it's probably something we should try to document clearly, particularly for first-time contributors. The So build
|
Hi @Naros , I was able to resolve everything last night, except the Informix connector: [INFO] DOCKER> [debezium/informix-test-database:15]: Created docker-build.tar in 26 milliseconds |
|
@pxcamus for the meantime can you try |
That worked, thanks! |
@Naros actually a few more issues but on macOS ARM. the
I tried everything I could think of: setting up a default profile in
I downloaded the image before running the Maven command:
Then tried passing in various way a flag: I think next I will have to modify the POM files to pass Good news is that I am saving to buy a Linux workstation 😅 |
|
@pxcamus snapshots should now be published on maven central so you could just try to run test on your branch and dependencies should be resolved. |
Still seeing this @mfvitale :
|
Hi @pxcamus,
Ah for sure you have to enable Rosetta emulation in your docker engine |
|
Thanks @vsantonastaso , that's what I thought I would have to do, but @mfvitale helped me out, and I was able to build. |
Signed-off-by: Philippe Camus <pxcamus@pm.me>
…min` integration and improved error handling. Update dependencies for Pulsar and Debezium. Signed-off-by: Philippe Camus <pxcamus@pm.me>
… validation using Testcontainers Signed-off-by: Philippe Camus <pxcamus@pm.me>
…emes through `PulsarAuthHandler` abstraction Signed-off-by: Philippe Camus <pxcamus@pm.me>
… `JwtAuthHandler` implementation and enhance connection validation logic Signed-off-by: Philippe Camus <pxcamus@pm.me>
…inProvider` interfaces to document authentication configuration and builder abstraction Signed-off-by: Philippe Camus <pxcamus@pm.me>
…s and minor code formatting adjustments Signed-off-by: Philippe Camus <pxcamus@pm.me>
…tion in interface Signed-off-by: Philippe Camus <pxcamus@pm.me>
…rFactory` and comprehensive unit tests for Pulsar authentication handlers. Signed-off-by: Philippe Camus <pxcamus@pm.me>
…tion with JWT authentication and enhance test coverage. Signed-off-by: Philippe Camus <pxcamus@pm.me>
… scenarios for expired JWT tokens and inject missing `authScheme` configuration. Add constant for JWT token secret key. Signed-off-by: Philippe Camus <pxcamus@pm.me>
…uth2AuthHandler` and extend connection validation tests with integration support Signed-off-by: Philippe Camus <pxcamus@pm.me>
…ort order inconsistencies, and include missing license headers in Pulsar-related classes and tests Signed-off-by: Philippe Camus <pxcamus@pm.me Signed-off-by: Philippe Camus <pxcamus@pm.me>
…ence correct Apache Software License version Signed-off-by: Philippe Camus <pxcamus@pm.me>
…Test` to reflect authority component validation in service URI. Signed-off-by: Philippe Camus <pxcamus@pm.me>
…ion tests to provide more specific failure reasons Signed-off-by: Philippe Camus <pxcamus@pm.me>
458c73f to
2e25131
Compare


WIP PR to add Apache Pulsar connection validation.