You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/event-hubs/log-compaction.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,14 +8,14 @@ ms.custom: ignite-2022
8
8
9
9
# Log Compaction
10
10
11
-
Log compaction is a way of retaining data in Event Hubs using event key based retention. By default, each event hub/Kafka topic is created with time-based retention or deletes* cleanup policy, where events are purged upon the expiration of the retention time. Rather using coarser-grained time based retention, you can use event key-based retention mechanism where Event Hubs retrains the last known value for each event key of an event hub or a Kafka topic.
11
+
Log compaction is a way of retaining data in Event Hubs using event key based retention. By default, each event hub/Kafka topic is created with time-based retention or *delete* cleanup policy, where events are purged upon the expiration of the retention time. Rather using coarser-grained time based retention, you can use event key-based retention mechanism where Event Hubs retrains the last known value for each event key of an event hub or a Kafka topic.
12
12
13
13
> [!NOTE]
14
14
> Log compaction feature is available only in **premium** and **dedicated** tiers.
15
15
16
16
As shown below, an event log (of an event hub partition) may have multiple events with the same key. If you're using a compacted event hub, then Event Hubs service will take care of purging old events and only keeping the latest events of a given event key.
17
17
18
-
:::image type="content" source="./media/event-hubs-log-compaction/log-compaction.png" alt-text="Image showing capturing of Event Hubs data into Azure Storage or Azure Data Lake Storage." lightbox="./media/event-hubs-resource-governance-overview/app-groups.png":::
18
+
:::image type="content" source="./media/event-hubs-log-compaction/log-compaction.png" alt-text="Image showing how a topic get compacted." lightbox="./media/event-hubs-resource-governance-overview/app-groups.png":::
19
19
20
20
### Compaction key
21
21
The partition key that you set with each event is used as the compaction key.
@@ -27,7 +27,7 @@ Client application can mark existing events of an event hub to be deleted during
27
27
28
28
You can enable log compaction at each event hub/Kafka topic level. You can ingest events to a compacted article from any support protocol. Azure Event Hubs service runs a compaction job for each compacted event hub. Compaction job cleans each event hub partition log by only retaining the latest event of a given event key.
29
29
30
-
:::image type="content" source="./media/event-hubs-log-compaction/how-compaction-work.png" alt-text="Image showing capturing of Event Hubs data into Azure Storage or Azure Data Lake Storage." lightbox="./media/event-hubs-log-compaction/how-compaction-work.png":::
30
+
:::image type="content" source="./media/event-hubs-log-compaction/how-compaction-work.png" alt-text="Image showing how log compaction works." lightbox="./media/event-hubs-log-compaction/how-compaction-work.png":::
31
31
32
32
At a given time the event log of a compacted event hub can have a *cleaned* portion and *dirty* portion. The clean portion contains the events that are compacted by the compaction job while the dirty portion comprises the events that are yet to be compacted.
33
33
@@ -46,4 +46,4 @@ Log compaction can be useful in scenarios where you stream the same set of updat
46
46
47
47
48
48
## Next steps
49
-
For instructions on how to use log compaction in Event Hubs, see [Use log compaction](user-log-compaction.md)
49
+
For instructions on how to use log compaction in Event Hubs, see [Use log compaction](./use-log-compaction.md)
Copy file name to clipboardExpand all lines: articles/event-hubs/use-log-compaction.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ ms.date: 10/7/2022
7
7
---
8
8
9
9
# Use log compaction
10
-
This article shows you how to create log compaction feature in Event Hubs. To understand the details of log compaction, see [Log Compaction](log-compaction.md).
10
+
This article shows you how to use log compaction feature in Event Hubs. To understand the details of log compaction, see [Log Compaction](log-compaction.md).
11
11
12
12
In this article you'll, follow these key steps:
13
13
- Create a compacted event hub/Kafka topic.
@@ -26,9 +26,9 @@ You can create a compacted event hub using the Azure portal by following these s
26
26
1. Navigate to your Event Hubs namespace.
27
27
1. On the Event Hubs Namespace page, select Event Hubs in the left menu.
28
28
1. At the top of the window, select + Event Hubs.
29
-
:::image type="content" source="./media/event-hubs-quickstart-portal/create-event-hub4.png" alt-text="Screenshot of the Application Groups page in the Azure portal.":::
29
+
:::image type="content" source="./media/event-hubs-quickstart-portal/create-event-hub4.png" alt-text="Screenshot of event hub creation UI.":::
30
30
1. Type a *name* for your event hub, and specify the *partition count*. Since we're creating a compacted event hub, select *compaction policy* as *compaction* and provide the desired value for *tombstone retention time*.
31
-
:::image type="content" source="./media/event-hubs-log-compaction/enabling-compaction.png" alt-text="Screenshot of the Application Groups page in the Azure portal.":::
31
+
:::image type="content" source="./media/event-hubs-log-compaction/enabling-compaction.png" alt-text="Screenshot of the event hubs creation UI with compation related attributes.":::
32
32
1. Select *create* and create the compacted event hub.
0 commit comments