You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/concepts-data-flow-schema-drift.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ ms.service: data-factory
9
9
ms.subservice: data-flows
10
10
ms.topic: conceptual
11
11
ms.custom: synapse
12
-
ms.date: 09/09/2021
12
+
ms.date: 09/19/2022
13
13
---
14
14
15
15
# Schema drift in mapping data flow
@@ -38,15 +38,15 @@ Columns coming into your data flow from your source definition are defined as "d
38
38
39
39
In a source transformation, schema drift is defined as reading columns that aren't defined in your dataset schema. To enable schema drift, check **Allow schema drift** in your source transformation.
When schema drift is enabled, all incoming fields are read from your source during execution and passed through the entire flow to the Sink. By default, all newly detected columns, known as *drifted columns*, arrive as a string data type. If you wish for your data flow to automatically infer data types of drifted columns, check **Infer drifted column types** in your source settings.
44
44
45
45
## Schema drift in sink
46
46
47
47
In a sink transformation, schema drift is when you write additional columns on top of what is defined in the sink data schema. To enable schema drift, check **Allow schema drift** in your sink transformation.
If schema drift is enabled, make sure the **Auto-mapping** slider in the Mapping tab is turned on. With this slider on, all incoming columns are written to your destination. Otherwise you must use rule-based mapping to write drifted columns.
52
52
@@ -70,7 +70,7 @@ To explicitly reference drifted columns, you can quickly generate mappings for t
70
70
71
71
In the generated Derived Column transformation, each drifted column is mapped to its detected name and data type. In the above data preview, the column 'movieId' is detected as an integer. After **Map Drifted** is clicked, movieId is defined in the Derived Column as `toInteger(byName('movieId'))` and included in schema views in downstream transformations.
72
72
73
-
:::image type="content" source="media/data-flow/mapdrifted2.png" alt-text="Screenshot shows the Derived Column's Settings tab.":::
73
+
:::image type="content" source="media/data-flow/map-drifted-2.png" alt-text="Screenshot shows the Derived Column's Settings tab.":::
74
74
75
75
## Next steps
76
76
In the [Data Flow Expression Language](data-transformation-functions.md), you'll find additional facilities for column patterns and schema drift including "byName" and "byPosition".
0 commit comments