Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints #799

danielmarbach · 2025-08-26T09:28:01Z

Fixes #767

This PR introduces a separation of the outbox records with a partition key that defaults to the endpoint name. The chosen approach is to use a structured ID that encodes the partition key into the document’s _id. This ensures that deduplication happens per endpoint and opens the door for future scenarios where the outbox collection may need to be sharded.

By making the partition key an explicit part of the identity { pk, mid }, we avoid the need for additional unique indexes (reducing storage and operational overhead) while still enforcing uniqueness at the correct granularity. The structured ID keeps reads and writes efficient by targeting _id, allows direct queries and indexes on subfields (e.g. _id.pk or _id.mid), and preserves flexibility in choosing shard keys later on. Compared to concatenated string IDs or separate compound indexes, this model is both more transparent and more naturally aligned with MongoDB’s compound key design.

Api Changes

Marked the old TimeToKeepOutboxDeduplicationData extension method as obsolete and redirected usage to the new MongoOutboxSettingsExtensions.TimeToKeepOutboxDeduplicationData to align with the settings root they belong to
Added new extension methods to MongoOutboxSettingsExtensions for disabling read fallback and configuring deduplication data retention under the outbox settings

Implementation tradeoffs

Replaced the $or filter used for lookups and updates with an explicit two-step flow: first attempt the structured _id, and only if no match is found (and fallback is enabled) attempt the legacy _id. The $or version was slightly more efficient in terms of roundtrips, but it left a small ambiguity: if both legacy and structured documents existed for the same message during migration, MongoDB could match either one, making it non-deterministic which record was updated. The explicit fallback removes that ambiguity and guarantees we always prefer the structured record, at the cost of an extra roundtrip only in the rare case where the legacy record is still present. This provides clearer semantics and safer migration behavior without meaningful performance impact.

danielmarbach · 2025-08-26T17:33:51Z

src/NServiceBus.Storage.MongoDB.AcceptanceTests/When_subscribers_handles_the_same_event.cs

+using NUnit.Framework;
+
+// TODO Can we also move this test to Core?
+public class When_subscribers_handles_the_same_event : NServiceBusAcceptanceTest


@andreasohlund I took over the test you wrote. I think this should be moved into Core ideally. Thoughts?

danielmarbach · 2025-08-26T17:35:41Z

src/NServiceBus.Storage.MongoDB.TransactionalSession/MongoTransactionalSessionExtensions.cs

        settings.Set(transactionalSessionOptions);
        settings.EnableFeatureByDefault<MongoTransactionalSession>();

+        if (!string.IsNullOrEmpty(transactionalSessionOptions.ProcessorEndpoint))


@andreasohlund @cquirosj @poornimanayar FYI I'm adopting a similar pattern to the other persisters, which requires InternalsVisibleTo but that is standard practice with the transactional session integration. Just pinging you here in case you can think of anything I might have missed.

danielmarbach · 2025-08-26T17:36:38Z

src/NServiceBus.Storage.MongoDB/Configuration/MongoSettingsExtensions.cs

    /// <summary>
    /// Configures the amount of time to keep outbox deduplication data.
    /// </summary>
+    [ObsoleteMetadata(


Note to myself: When releasing, these need to be moved down

danielmarbach · 2025-08-26T17:37:43Z

src/NServiceBus.Storage.MongoDB/Outbox/OutboxPersister.cs

 class OutboxPersister : IOutboxStorage
 {
-    public OutboxPersister(IMongoClient client, string databaseName, MongoDatabaseSettings databaseSettings, Func<Type, string> collectionNamingConvention, MongoCollectionSettings collectionSettings)
+    public OutboxPersister(IMongoClient client, string partitionKey, bool readFallbackEnabled, string databaseName, MongoDatabaseSettings databaseSettings, Func<Type, string> collectionNamingConvention, MongoCollectionSettings collectionSettings)


The constructor gets bigger and bigger. At some point it might make sense to clean this up a bit, but I wanted to stay focused on the task at hand for now.

src/NServiceBus.Storage.MongoDB/Outbox/OutboxPersister.cs

danielmarbach · 2025-08-26T17:45:35Z

src/NServiceBus.Storage.MongoDB/Outbox/OutboxStorage.cs

        {
            UsesDefaultClassMap = usesDefaultClassMap,
-            TimeToKeepDeduplicationData = context.Settings.GetTimeToKeepOutboxDeduplicationData(),
+#pragma warning disable IDE0037


This keeps the left side consistent and allows us to rename the internal property.

danielmarbach · 2025-08-27T10:12:15Z

src/NServiceBus.Storage.MongoDB/Outbox/OutboxPersister.cs


+        var outboxRecordId = new OutboxRecordId { MessageId = messageId, PartitionKey = partitionKey };
+
+        var equalityPredicateWithFallback = CreateOutboxRecordFilterPredicate(outboxRecordId);


When the fallback is enabled, it is theoretically possible under a GUID collision (highly unlikely) or during a mixed migration that an entry is stored under the same message ID once in the legacy format and once in the new format. The only way to end up with both a legacy (string _id) and a structured (document _id) record for the same message is during a mixed-version migration. In that rare case, SetAsDispatched might update the legacy record instead of the structured one because the UpdateOneAsync with the OR is non-deterministic. In a regular outbox interaction, we always do a Get first to retrieve the deduplication record. If that is the case, the SingleOrDefault would throw, which safeguards us from doing the wrong thing.

In the transactional session cases, we currently do not support setting the "message id" on the outbox record so that part is not affected at this stage (even though we are not doing a Get there).

The alternative would be to get tighter control over the flow and do explicit roundtrips with fallbacks. This would remove the roundtrip optimization but might lead to more correct flows under any circumstances. I'm leaning towards this approach for now.

See b07e368

src/NServiceBus.Storage.MongoDB/Outbox/OutboxPersister.cs

afprtclr · 2025-08-28T10:05:15Z

src/NServiceBus.Storage.MongoDB/Outbox/OutboxPersistenceConfiguration.cs

+
+    public bool ReadFallbackEnabled { get; set; } = true;
+
+    public string PartitionKey { get; set; } = null!; // will be set by defaults


Not sure if we should call it ParitionKey. It's leaking internal implementation. Not even sure if this should be exposed at all.

I agree the name should be questioned. Would you prefer to name it what it represents currently which is EndpointName?

Regarding the "not sure it should be exposed at all". The current thinking of mine was that it is not exposed to the user but exposed as a configuration option so that there is a single path on which the configuration options flow into the persister. This helps bridge the settings over to a simple configuration class and pass that into the persister where needed. At one point I was thinking we could even streamline all the configuration that is currently passed into the outbox persister including the database settings with defaults and only pass in this configuration object as sort of a "creation option" into the type.

This would allow addressing this comment too https://github.com/Particular/NServiceBus.Storage.MongoDB/pull/799/files/16bbb427918ec8675550884a307eb28be54af7df#r2301681637 but I was hesitant to jump the gun.

Are you thinking along the lines that this class should only represent the properties that are exposed to the users and all the "internal stuff" should not be added here to clearly separate those configuration paths?

Are you thinking along the lines that this class should only represent the properties that are exposed to the users and all the "internal stuff" should not be added here to clearly separate those configuration paths?

Yep, that was my thought.

Got it. I think that is a valid angle. Will change that in the next update

Been thinking about this a bit more and concluded it is not worth changing. The OutboxPersistenceConfiguration PartitionKey currently is either by default the endpoint name or in the case of the transactional session the remote processor endpoint name. This is what we currently use and therefore it presents a configuration option (even when only used "internally". Given that the OutboxRecordId also encodes the partition code as "pk" (partition key) and not "ep" (endpoint name) we seem to be using consistently the same "language".

I'd be more than happy to rediscuss the entire "verbiage" here in case we are no longer convinced partition key is the right term in general.

…lient is used

… the persistence tests have their own infrastructureˆ

…default approach

internalautomation bot assigned danielmarbach Aug 26, 2025

danielmarbach commented Aug 26, 2025

View reviewed changes

danielmarbach commented Aug 27, 2025

View reviewed changes

danielmarbach changed the title ~~Outbox record seperation~~ Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints Aug 27, 2025

danielmarbach changed the title ~~Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints~~ Seperate outbox records by endpoint name to enable properly sharing the same collection between endpoints Aug 27, 2025

danielmarbach changed the title ~~Seperate outbox records by endpoint name to enable properly sharing the same collection between endpoints~~ Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints Aug 27, 2025

danielmarbach force-pushed the outbox-record-seperation branch from de9fbe7 to 16bbb42 Compare August 27, 2025 10:32

danielmarbach requested a review from afprtclr August 27, 2025 10:35

danielmarbach marked this pull request as ready for review August 27, 2025 10:35

afprtclr reviewed Aug 28, 2025

View reviewed changes

danielmarbach added 20 commits September 1, 2025 09:05

Add an acceptance test

8348554

Simple straight forward implementation to make the test pass

9603a74

Use an equality predicate with fallback

3137ab7

Upconvert old legacy entries

1f447cb

Remove todo for now

eb9e8c0

Move time to keep duplication data to outbox settings

6445a6f

Making sure the outbox class mappings are always present before the c…

e7d5675

…lient is used

Proper namespaces and wire up new configuration

52a4036

Remove unneeded Mongo prefix

1aa7a3a

Remove unneeded check

d93a971

Introduce read fallback and run persistence tests without

f917653

Fix default class map indicator

3e1e6b6

Forgot to approve the rename

1cfe791

Bring back the default of seven days for TTL

6aee372

Make sure the processor endpoint is taken into account

78ccdac

Disable outbox installation when processor endpoint is set

5baeae5

Suppress null warning

9afcacb

Make outbox settings extensions still fluent since others are too

00d6270

Inline the hierarchy since we really only have these outbox tests and…

b98b545

… the persistence tests have their own infrastructureˆ

Some minor formattingˆ

392684c

danielmarbach added 6 commits September 1, 2025 09:05

Additional test

34b3e64

Use explicit fallback sacrificing network latency for a more safe by …

88a5956

…default approach

Remove TODO for now since we can deal with this elsewhere

4b1a306

Extract filtering

ec1258f

Comment

efaef8c

Comment

379c950

danielmarbach force-pushed the outbox-record-seperation branch from a5814ad to 379c950 Compare September 1, 2025 07:06

afprtclr approved these changes Sep 1, 2025

View reviewed changes

danielmarbach merged commit 72d1e47 into master Sep 1, 2025
4 checks passed

danielmarbach deleted the outbox-record-seperation branch September 1, 2025 14:15

danielmarbach added this to the 6.0.0 milestone Sep 1, 2025

danielmarbach mentioned this pull request Sep 4, 2025

Introduce EnableReadFallback and obsolete DisableReadFallback #804

Open

danielmarbach added the Improvement label Sep 4, 2025

dependabot bot mentioned this pull request Sep 8, 2025

Bump the nservice group with 1 update futugyou/resource-topology#187

Merged


		var outboxRecordId = new OutboxRecordId { MessageId = messageId, PartitionKey = partitionKey };

		var equalityPredicateWithFallback = CreateOutboxRecordFilterPredicate(outboxRecordId);


		public bool ReadFallbackEnabled { get; set; } = true;

		public string PartitionKey { get; set; } = null!; // will be set by defaults

Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints #799

Separate outbox records by endpoint name to enable properly sharing the same collection between endpoints #799

Uh oh!

Conversation

danielmarbach commented Aug 26, 2025 • edited by PhilBastian Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Api Changes

Implementation tradeoffs

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielmarbach Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

danielmarbach commented Aug 26, 2025 •

edited by PhilBastian

Loading

danielmarbach Aug 27, 2025 •

edited

Loading