Kafka 3.9.2 when upgrading from Strimzi 0.45.2

[jeroenmoons-klarrio]

Triggered by the 0.45.2 release notes section "Upgrade to Strimzi 0.46-0.51" indicating that upgrading Strimzi would mean downgrading Kafka from 3.9.2 to 3.9.1:

Is there no way to stay on Kafka 3.9.2 when moving on from Strimzi 0.45.2? Are there any plans to patch newer releases (0.47.0 in particular) to add 3.9.2 support? I am working on an upgrade path that has 3.9.2 as an in-between milestone. Upgrading directly to Kafka 4 binaries is deemed too risky. The aim would be to land on Strimzi 0.47.0 with Kafka 3.9.x first, and Kafka 3.9.2 would be much preferred to 3.9.1 at that point due to the security fixes it includes.

Anybody with knowledge of the Strimzi project who knows what I can and can not expect to be supported within this and a couple of months' time? What could I do today to have Strimzi 0.47.0 running Kafka 3.9.2 binaries?

Thanks for any insights!

[scholzj]

There are no plans to support Kafka 3.9.2 in any other Strimzi release. The expectation is that you upgrade to Kafka 4.x from 0.45.2.

But please keep in mind that the steps in the release do not downgrade the Kafka version. The annotation change just fools the newer operator version to upgrade from 3.9.2 to some other supported Kafka version (ideally to 4.x and not back to 3.9.0/3.9.1). But it will not do any actual downgrade on its own.

@im-konge @ppatierno @katheris Perhaps we should try to make it more clear in the release notes somehow?

If you would want to run Strimzi 0.47 with Kafka 3.9.2, you would need to cherry-pick the changes and make your custom private 0.47 build..

[scholzj]

As in if you took the 0.47.0-kafka-3.9.1 image, overwrote the Kafka 3.9.1 with 3.9.2 binaries and used it? It would probably work. It would just be a bit confusing as it would say that you are running 3.9.1 while you were really running 3.9.2. But as long as you keep it in mind, it should be fine as there were no breaking changes between 3.9.1 and 3.9.2 as far as I know.

(Obviously, you should test it first somewhere before going to production ... but I guess you understand that 😉 )

[jeroenmoons-klarrio]

Yes that is what I was thinking of. I figured that would be the case, the confusion is indeed something to be very aware of. This would in any case be a stepping stone to a latest situation where we have Strimzi's latest and greatest + Kafka 4.2.

We are moving from koperator, Kafka 3.7 to Strimzi, this is a complicated operation already involving a ZooKeeper to KRaft migration + upstepping from Strimzi 0.45.1 to 0.47.0 to take advantage of the custom tls auth feature in there which we need. Going to Kafka 4.x as well as part of that journey seems to add too much extra risk in what is a risky operation already.

We would like to keep the Kafka 3.9 to 4.x as a separate exercise.

Forking Strimzi 0.47.0 might be a route as well, I see the commit adding Kafka 3.9.2 support does nothing more than changing 3.9.1 to 3.9.2 in lots of places + adding a few items to lists to add that version.

This makes our options clear I think, thanks again for your input!

[scholzj]

Yeah, that sounds pretty complicated. It might make sense to not jump too much in too short time.

[im-konge]

Thanks for the reply Jakub, I wanted to write similar thing.

Perhaps we should try to make it more clear in the release notes somehow?

Yeah we should write there some note that it will not downgrade the Kafka version, it will just fool the operator, so the upgrade is possible.

Regarding other patch releases - we had a discussion around this, it would mean having wide range of patch releases which would be a lot of effort (releasing patch release for every minor release of Strimzi that is not supported anymore). That's why we came up with the solution of having that one command that will unblock users to upgrade to 4.x. But I understand why it is risky for you. I don't see any other possible solution than building your own image (as described in the thread).

[jeroenmoons-klarrio]

Thanks both! We'll weigh our options. At some point we're planning to blog about the entire journey, will share on one of the discussion boards when we've got that done.