Advanced failure strategies for kafka message consumption

[mrickly]

It would be nice to be able to use other failure strategies than those currently offered by quarkus kafka: https://quarkus.io/guides/kafka#error-handling).
For instance: In case of a failure, the consumption is paused on the affected (alternatively all) partitions. The consumption resumes based on the result of a configurable check (for instance verifying that some system is available) that is performed periodically (with a configurable period). Message consumptions continues on the partitions that are not paused.

Are there plans to add such strategies, or to enable applications to define their own failure strategies?

[quarkus-bot[bot]]

/cc @alesj, @cescoffier, @ozangunalp

[ozangunalp]

It may be pretty tricky to implement but there is a recent SPI for commit and failure strategies: https://smallrye.io/smallrye-reactive-messaging/3.21.0/kafka/receiving-kafka-records/#custom-commit-and-failure-strategies. I think the latest version of quarkus docs refers to that.

[mrickly]

Thank you @ozangunalp, fast as usual ;-) I was not aware of that. It sounds promising. I will look at it in details.