Skip to content

Conversation

@thomas-bousquet
Copy link
Contributor

@thomas-bousquet thomas-bousquet commented Sep 3, 2025

Motivation

When the retry mechanism is enabled, the RLQ producer first sends messages to the RLQ topic using the producer name if passed in the producer options inside the DLQPolicy.
However, with DLQPolicy.ProducerOptions.Name set, when the delivery count is reached and the consumer Nack() the messages, the DLQ producer sending messages to the DLQ uses RLQ producer name.

This is an issue, especially for message deduplication process where each producers must have a globally unique name for a given topic.

Modifications

This PR adds the DeadLetterTopicProducerName field to the DLQPolicy in order to either allow using a custom producer name or let the client create the producer name with the existing naming pattern.

Verifying this change

  • Added integration test TestWithoutDeadLetterTopicDeadLetterTopicProducerName
  • Added integration test TestWithDeadLetterTopicDeadLetterTopicProducerName

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API: no
  • The schema: no
  • The default values of configurations: no
  • The wire protocol: no

Documentation

  • Does this pull request introduce a new feature? no
  • The DeadLetterTopicProducerName field added is documented where it's defined in the DLQPolicy struct

@thomas-bousquet thomas-bousquet changed the title fix: add DLQPolicy.DeadLetterTopicProducerName so that it can be used… fix: add DLQPolicy.DeadLetterTopicProducerName Sep 3, 2025
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes an issue where the DLQ (Dead Letter Queue) producer incorrectly uses the RLQ (Retry Letter Queue) producer name when sending messages to the DLQ topic. The fix adds a new DeadLetterTopicProducerName field to the DLQPolicy struct to allow specifying a custom producer name for the DLQ or letting the system generate one using the existing naming pattern.

  • Adds DeadLetterTopicProducerName field to DLQPolicy struct for configuring DLQ producer names
  • Updates DLQ router logic to use the new field instead of incorrectly using RLQ producer name
  • Adds comprehensive integration tests to verify both custom and auto-generated DLQ producer names

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
pulsar/consumer.go Adds the new DeadLetterTopicProducerName field to the DLQPolicy struct with documentation
pulsar/dlq_router.go Updates producer name logic to check and use the new DeadLetterTopicProducerName field
pulsar/consumer_test.go Adds two integration tests to verify the new functionality works correctly

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@thomas-bousquet
Copy link
Contributor Author

Looks like CI failed with a timeout in the 1.23 run. Would it be possible to rerun it since it passed with 1.24?

@crossoverJie crossoverJie added this to the v0.17.0 milestone Sep 4, 2025
@crossoverJie crossoverJie merged commit 6aa1fb6 into apache:master Sep 4, 2025
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants