feat: add WithSelfNotification option to filter self-published messages

[laciferin2024]

Summary

Adds a WithSelfNotification(enabled bool) subscribe option that allows subscribers to opt out of receiving messages published by the local node. This addresses a common use case where applications need to avoid processing their own published messages.

Closes #537

Changes

• Added WithSelfNotification(enabled bool) as a new SubOpt for Topic.Subscribe()
• When set to false, messages published by the local host are filtered out in notifySubs before delivery
• Default behavior is unchanged — subscriptions still receive all messages, including self-published ones
• Added test TestWithSelfNotification covering:
  • Self-published messages are skipped on filtered subscriptions
  • Self-published messages still arrive on default subscriptions
  • Messages from remote peers still arrive on filtered subscriptions

Design

The filtering happens at the notifySubs level rather than in Next(), so self-published messages never enter the subscription channel buffer. This avoids wasting buffer space and prevents unnecessary drops when the subscriber is slow.

Each subscription stores the local host's peer ID at creation time and compares it against msg.ReceivedFrom during delivery.

[MarcoPolo]

Did you need this feature? Why was skipping the message on the application side not good enough?

[laciferin2024]

Did you need this feature? Why was skipping the message on the application side not good enough?

The issue with filtering on the application side (i.e. in the handler after Next()) is that the message has already been delivered into the subscription's channel buffer. For slow subscribers this means the buffer fills up with self-published messages that the app will just discard, which can cause legitimate remote messages to be dropped. Filtering at notifySubs prevents self-published messages from ever entering the buffer, so the buffer capacity is fully available for messages the subscriber actually wants. That was the main reason to do it at this layer rather than leaving it to the caller.

[MarcoPolo]

Realistically a single equality check by the handler is not going to slow the subscriber enough to cause it to drop messages. Did you see this issue in production?

[laciferin2024]

Realistically a single equality check by the handler is not going to slow the subscriber enough to cause it to drop messages. Did you see this issue in production?

With tight buffer sizes yes, this filter auto drops self messages.

Just to give u my story:

1. Processing in heart beat intervals with 'N' goroutines . A large fraction of those goroutines are consumed by self messages that are dropped.

[MarcoPolo]

Let's change this to be an option that sets a message subscription filter function that can be passed in as a SubOpt. Then the application can filter out messages where ReceivedFrom == self, and this can happen in notifySubs

[vyzo]

precedent from IP Multicast here too, this is a sockopt in POSIX.

[laciferin2024]

Let's change this to be an option that sets a message subscription filter function that can be passed in as a SubOpt. Then the application can filter out messages where ReceivedFrom == self, and this can happen in notifySubs

Done