You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/how-to-change-data-capture-resource.md
+22-22Lines changed: 22 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,46 +31,46 @@ In this tutorial, you follow these steps:
31
31
32
32
1. Navigate to the **Author** blade in your data factory. You will see a new top-level artifact below **Pipelines** called **Change Data Capture (preview)**.
33
33
34
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-61.png" alt-text="Updated screenshot of new top level artifact shown under Factory resources panel." lightbox="media/adf-cdc/change-data-capture-resource-61.png":::
34
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-61.png" alt-text="Screenshot of new top level artifact shown under Factory resources panel." lightbox="media/adf-cdc/change-data-capture-resource-61.png":::
35
35
36
36
2. To create a new **Change Data Capture**, hover over **Change Data Capture (preview)** until you see 3 dots appear. Click on the **Change Data Capture (preview) Actions**.
37
37
38
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-62.png" alt-text="Updated screenshot of Change Data Capture (preview) Actions after hovering on the new top-level artifact." lightbox="media/adf-cdc/change-data-capture-resource-62.png":::
38
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-62.png" alt-text="Screenshot of Change Data Capture (preview) Actions after hovering on the new top-level artifact." lightbox="media/adf-cdc/change-data-capture-resource-62.png":::
39
39
40
40
3. Select **New CDC (preview)**. This will open a flyout to begin the guided process.
41
41
42
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-63.png" alt-text="Updated screenshot of a list of Change Data Capture actions." lightbox="media/adf-cdc/change-data-capture-resource-63.png":::
42
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-63.png" alt-text="Screenshot of a list of Change Data Capture actions." lightbox="media/adf-cdc/change-data-capture-resource-63.png":::
43
43
44
44
4. You will then be prompted to name your CDC resource. By default, the name will be set to “adfcdc” and continue to increment up by 1. You can replace this default name with your own.
45
45
46
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-64.png" alt-text="Updated screenshot of the text box to update the name of the resource.":::
46
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-64.png" alt-text="Screenshot of the text box to update the name of the resource.":::
47
47
48
48
5. Use the drop-down selection list to choose your data source. For this tutorial, we will use **DelimitedText**.
49
49
50
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-65.png" alt-text="Updated screenshot of the guided process flyout with source options in a drop-down selection menu.":::
50
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-65.png" alt-text="Screenshot of the guided process flyout with source options in a drop-down selection menu.":::
51
51
52
52
6. You will then be prompted to select a linked service. Create a new linked service or select an existing one.
53
53
54
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-66.png" alt-text="Updated screenshot of the selection box to choose or create a new linked service.":::
54
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-66.png" alt-text="Screenshot of the selection box to choose or create a new linked service.":::
55
55
56
56
7. Use the **Browse** button to select your source data folder.
57
57
58
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-67.png" alt-text="Updated screenshot of a folder icon to browse for a folder path.":::
58
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-67.png" alt-text="Screenshot of a folder icon to browse for a folder path.":::
59
59
60
60
8. Once you’ve selected a folder path, click **Continue** to set your data target.
61
61
62
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-68.png" alt-text="Updated screenshot of the continue button in the guided process to proceed to select data targets.":::
62
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-68.png" alt-text="Screenshot of the continue button in the guided process to proceed to select data targets.":::
63
63
64
64
> [!NOTE]
65
65
> You can choose to add multiple source folders with the **+** button. The other sources must also use the same linked service that you’ve already selected.
66
66
67
67
9. Then, select a **Target type** using the drop-down selection. For this tutorial, we will select **Azure SQL Database**.
68
68
69
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-69.png" alt-text="Updated screenshot of a drop-down selection menu of all data target types.":::
69
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-69.png" alt-text="Screenshot of a drop-down selection menu of all data target types.":::
70
70
71
71
10. You will then be prompted to select a linked service. Create a new linked service or select an existing one.
72
72
73
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-70.png" alt-text="Updated screenshot of the selection box to choose or create a new linked service to your data target.":::
73
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-70.png" alt-text="Screenshot of the selection box to choose or create a new linked service to your data target.":::
74
74
75
75
11. Create new **Target table(s)** or select an existing **Target table(s)**. Under **Existing entities** use the checkbox to select an existing Target table(s) or Under **New entities** select **Edit new tables** to create new Target table(s). The **Preview** button will allow you to view your table data.
76
76
@@ -83,18 +83,18 @@ In this tutorial, you follow these steps:
83
83
84
84
12. Click **Continue** when you have finalized your selection(s).
85
85
86
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-73.png" alt-text="Updated screenshot of the continue button in the guided process to proceed to the next step.":::
86
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-73.png" alt-text="Screenshot of the continue button in the guided process to proceed to the next step.":::
87
87
88
88
> [!NOTE]
89
89
> You can choose multiple target tables from your Azure SQL DB. Use the check boxes to select all targets.
90
90
91
91
13. You will automatically land in a new change data capture tab, where you can configure your new resource.
92
92
93
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-74.png" alt-text="Updated screenshot of the change data capture studio." lightbox="media/adf-cdc/change-data-capture-resource-74.png":::
93
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-74.png" alt-text="Screenshot of the change data capture studio." lightbox="media/adf-cdc/change-data-capture-resource-74.png":::
94
94
95
95
14. A new mapping will automatically be created for you. You can update the **Source** and **Target** selections for your mapping by using the drop-down selection lists.
96
96
97
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-75.png" alt-text="Updated screenshot of the source to target mapping in the change data capture studio." lightbox="media/adf-cdc/change-data-capture-resource-75.png":::
97
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-75.png" alt-text="Screenshot of the source to target mapping in the change data capture studio." lightbox="media/adf-cdc/change-data-capture-resource-75.png":::
98
98
99
99
15. Once you’ve selected your tables, you should see that their columns are auto mapped by default with the **Auto map** toggle on. Auto map automatically maps the columns by name in the sink, picks up new column changes when source schema evolves and flows this to the supported sink types. If you would want to retain Auto map and not change any column mappings, proceed to **Step 19** directly.
100
100
@@ -125,27 +125,27 @@ In this tutorial, you follow these steps:
125
125
126
126
20. Once your mapping complete, set your CDC latency using the **Set Latency** button.
127
127
128
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-82.png" alt-text="Updated screenshot of the set frequency button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-82.png":::
128
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-82.png" alt-text="Screenshot of the set frequency button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-82.png":::
129
129
130
130
21. Select the latency of your CDC and click **Apply** to make the changes. By default, it will be set to **15 minutes**. For this tutorial, we will select the **Real-time** latency. Real-time latency will continuously keep picking up changes in your source data in a less than 1 minute interval.
131
131
132
132
For other latencies, say if you select 15 minutes, every 15 minutes, your change data capture will process your source data and pick up any changed data since the last processed time.
133
133
134
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-83.png" alt-text="Updated screenshot of the set frequency selection menu.":::
134
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-83.png" alt-text="Screenshot of the set frequency selection menu.":::
135
135
136
136
> [!NOTE]
137
137
> Support for **streaming data integration** (EventHub & Kafka data sources) is coming soon. When available the latency will be set to Real-time by default.
138
138
139
139
22. Once everything has been finalized, click the **Publish All** to publish your changes.
140
140
141
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-84.png" alt-text="Updated screenshot of the publish button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-84.png":::
141
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-84.png" alt-text="Screenshot of the publish button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-84.png":::
142
142
143
143
> [!NOTE]
144
144
> If you do not publish your changes, you will not be able to start your CDC resource. The start button will be greyed out.
145
145
146
146
23. Click **Start** to start running your **Change Data Capture**.
147
147
148
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-85.png" alt-text="Updated screenshot of the start button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-85.png":::
148
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-85.png" alt-text="Screenshot of the start button at the top of the canvas." lightbox="media/adf-cdc/change-data-capture-resource-85.png":::
149
149
150
150
151
151
## Monitor your Change data capture
@@ -154,26 +154,26 @@ For other latencies, say if you select 15 minutes, every 15 minutes, your change
154
154
155
155
:::image type="content" source="media/adf-cdc/change-data-capture-resource-86.png" alt-text="Screenshot of the monitoring blade.":::
156
156
157
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-87.png" alt-text="Updated screenshot of the monitoring button at the top of the CDC canvas.":::
157
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-87.png" alt-text="Screenshot of the monitoring button at the top of the CDC canvas.":::
158
158
159
159
2. Select **Change Data Capture (preview)** to view your CDC resources.
160
160
161
161
:::image type="content" source="media/adf-cdc/change-data-capture-resource-88.png" alt-text="Screenshot of the Change Data Capture monitoring section.":::
162
162
163
163
3. Here you can see the **Source**, **Target**, **Status**, and **Last processed** time of your change data capture.
164
164
165
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-89.png" alt-text="Updated screenshot of an overview of the change data capture monitoring page.":::
165
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-89.png" alt-text="Screenshot of an overview of the change data capture monitoring page.":::
166
166
167
167
4. Click the name of your CDC to see more details. You can see how many changes (insert/update/delete) were read and written and other diagnostic information.
168
168
169
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-90.png" alt-text="Updated screenshot of the detailed monitoring of a selected change data capture.":::
169
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-90.png" alt-text="Screenshot of the detailed monitoring of a selected change data capture.":::
170
170
171
171
> [!NOTE]
172
172
> If you have multiple mappings set up in your Change data capture, each mapping will show as a different color. Click on the bar to see specific details for each mapping or use the Diagnostics at the bottom of the screen.
173
173
174
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-91.png" alt-text="Updated screenshot of the detailed monitoring page of a change data capture with multiple sources to target mappings.":::
174
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-91.png" alt-text="Screenshot of the detailed monitoring page of a change data capture with multiple sources to target mappings.":::
175
175
176
-
:::image type="content" source="media/adf-cdc/change-data-capture-resource-92.png" alt-text="Updated screenshot of a detailed breakdown of each mapping in the change data capture artifact.":::
176
+
:::image type="content" source="media/adf-cdc/change-data-capture-resource-92.png" alt-text="Screenshot of a detailed breakdown of each mapping in the change data capture artifact.":::
0 commit comments