Skip to content

Commit 2f7d17d

Browse files
bibryammsfussellmarcduiker
authored
Added 'How the Outbox Works' section and expand requirements with int… (#4875)
* Added 'How the Outbox Works' section and expand requirements with internal topic details Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md Co-authored-by: Mark Fussell <[email protected]> Signed-off-by: Bilgin Ibryam <[email protected]> * Update howto-outbox.md Applied suggestions from the review manually. Signed-off-by: Bilgin Ibryam <[email protected]> * Updated the diagram with Dapr-styled version Signed-off-by: Bilgin Ibryam <[email protected]> --------- Signed-off-by: Bilgin Ibryam <[email protected]> Co-authored-by: Mark Fussell <[email protected]> Co-authored-by: Marc Duiker <[email protected]>
1 parent 95f4442 commit 2f7d17d

File tree

2 files changed

+60
-7
lines changed

2 files changed

+60
-7
lines changed

daprdocs/content/en/developing-applications/building-blocks/state-management/howto-outbox.md

Lines changed: 60 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,23 +16,72 @@ For example, you can use the outbox pattern to:
1616

1717
With Dapr's outbox support, you can notify subscribers when an application's state is created or updated when calling Dapr's [transactions API]({{% ref "state_api.md#state-transactions" %}}).
1818

19-
The diagram below is an overview of how the outbox feature works:
19+
The diagram below is an overview of how the outbox feature works at a high level:
2020

2121
1) Service A saves/updates state to the state store using a transaction.
2222
2) A message is written to the broker under the same transaction. When the message is successfully delivered to the message broker, the transaction completes, ensuring the state and message are transacted together.
2323
3) The message broker delivers the message topic to any subscribers - in this case, Service B.
2424

25-
<img src="/images/state-management-outbox.png" width=800 alt="Diagram showing the steps of the outbox pattern">
25+
<img src="/images/state-management-outbox.png" width=800 alt="Diagram showing the overview of outbox pattern">
2626

27+
## How outbox works under the hood
28+
29+
Dapr outbox processes requests in two flows: the user request flow and the background message flow. Together, they guarantee that state and events stay consistent.
30+
31+
<img src="/images/state-management-outbox-steps.png" width=800 alt="Diagram showing the steps of the outbox pattern">
32+
33+
This is the sequence of interactions:
34+
35+
1. An application calls the Dapr State Management API to write state transactionally using the transactional methods.
36+
This is the entry point where business data, such as an order or profile update, is submitted for persistence.
37+
38+
2. Dapr publishes an intent message with a unique transaction ID to an internal outbox topic.
39+
This durable record ensures the event intent exists before any database commit happens.
40+
41+
3. The state and a transaction marker are written atomically in the same state store.
42+
Both the business data and the marker are committed in the same transaction, preventing partial writes.
43+
44+
4. The application receives a success response after the transaction commits.
45+
At this point, the application can continue, knowing state is saved and the event intent is guaranteed.
46+
47+
5. A background subscriber reads the intent message.
48+
When outbox is enabled, Dapr starts consumers that process the internal outbox topic.
49+
50+
6. The subscriber verifies the transaction marker in the state store.
51+
This check confirms that the database commit was successful before publishing externally.
52+
53+
7. Verified business event is published to the external pub/sub topic.
54+
The event is sent to the configured broker (Kafka, RabbitMQ, etc.) where other services can consume it.
55+
56+
8. The marker is cleaned up (deleted) from the state store.
57+
This prevents unbounded growth in the database once the event has been successfully delivered.
58+
59+
9. Message is acknowledged and removed from internal topic
60+
If publishing or cleanup fails, Dapr retries, ensuring reliable at-least-once delivery.
61+
2762
## Requirements
2863

29-
The outbox feature can be used with using any [transactional state store]({{% ref supported-state-stores %}}) supported by Dapr. All [pub/sub brokers]({{% ref supported-pubsub %}}) are supported with the outbox feature.
64+
1. The outbox feature requires a [transactional state store]({{% ref supported-state-stores %}}) supported by Dapr.
65+
[Learn more about the transactional methods you can use.]({{% ref "howto-get-save-state.md#perform-state-transactions" %}})
3066

31-
[Learn more about the transactional methods you can use.]({{% ref "howto-get-save-state.md#perform-state-transactions" %}})
67+
2. Any [pub/sub broker]({{% ref supported-pubsub %}}) supported by Dapr can be used with the outbox feature.
3268

33-
{{% alert title="Note" color="primary" %}}
34-
Message brokers that work with the competing consumer pattern (for example, [Apache Kafka]({{% ref setup-apache-kafka%}})) are encouraged to reduce the chances of duplicate events.
35-
{{% /alert %}}
69+
{{% alert title="Note" color="primary" %}}
70+
Message brokers that support the competing consumer pattern (for example, [Apache Kafka]({{% ref setup-apache-kafka%}})) are recommended to reduce the chance of duplicate events.
71+
{{% /alert %}}
72+
73+
3. Internal outbox topic
74+
When outbox is enabled, Dapr creates an internal topic using the following naming convention: `{namespace}{appID}{topic}outbox`, where:
75+
76+
- `namespace`: the Dapr application namespace (if configured)
77+
- `appID`: the Dapr application identifier
78+
- `topic`: the value specified in the `outboxPublishTopic` metadata
79+
80+
This way each outbox topic is uniquely identified per application and external topic, preventing routing conflicts in multi-tenant environments.
81+
82+
{{% alert title="Note" color="primary" %}}
83+
Ensure that the topic is created in advance, or Dapr has sufficient permissions to create the topic at startup time.
84+
{{% /alert %}}
3685

3786
## Enable the outbox pattern
3887

@@ -682,3 +731,7 @@ The `data` CloudEvent field is reserved for Dapr's use only, and is non-customiz
682731
Watch [this video for an overview of the outbox pattern](https://youtu.be/rTovKpG0rhY?t=1338):
683732

684733
{{< youtube id=rTovKpG0rhY start=1338 >}}
734+
735+
## Next Steps
736+
737+
[How Dapr Outbox Eliminates Dual Writes in Distributed Applications](https://www.diagrid.io/blog/how-dapr-outbox-eliminates-dual-writes-in-distributed-applications)
136 KB
Loading

0 commit comments

Comments
 (0)