You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Use `$$` syntax to reference the input name of a matched column. Using the above image as an example, say a user wants to match on all string columns whose names are shorter than six characters. If one incoming column was named `test`, the expression `$$ + '_short'` will rename the column `test_short`. If thats the only mapping that exists, all columns that don't meet the condition will be dropped from the outputted data.
51
+
Use `$$` syntax to reference the input name of a matched column. Using the above image as an example, say a user wants to match on all string columns whose names are shorter than six characters. If one incoming column was named `test`, the expression `$$ + '_short'` will rename the column `test_short`. If that's the only mapping that exists, all columns that don't meet the condition will be dropped from the outputted data.
52
52
53
53
Patterns match both drifted and defined columns. To see which defined columns are mapped by a rule, click the eyeglasses icon next to the rule. Verify your output using data preview.
54
54
55
55
### Regex mapping
56
56
57
-
If you click the downward chevron icon, you can specify a regexmapping condition. A regexmapping condition matches all column names that match the specified regex condition. This can be used in combination with standard rule-based mappings.
57
+
If you click the downward chevron icon, you can specify a regex-mapping condition. A regex-mapping condition matches all column names that match the specified regex condition. This can be used in combination with standard rule-based mappings.
The above example matches on regex pattern `(r)` or any column name that contains a lower case r. Similar to standard rule-based mapping, all matched columns are altered by the condition on the right using `$$` syntax.
62
62
63
63
### Rule-based hierarchies
64
64
65
-
If your defined projection has a hierarchy, you can use rule-based mapping to map the hierarchies sub-columns. Specify a matching condition and the complex column whose sub-columns you wish to map. Every matched sub-column will be outputted using the 'Name as' rule specified on the right.
65
+
If your defined projection has a hierarchy, you can use rule-based mapping to map the hierarchies subcolumns. Specify a matching condition and the complex column whose subcolumns you wish to map. Every matched subcolumn will be outputted using the 'Name as' rule specified on the right.
The above example matches on all sub-columns of complex column `a`. `a` contains two sub-columns`b` and `c`. The output schema will include two columns `b` and `c` as the 'Name as' condition is `$$`.
69
+
The above example matches on all subcolumns of complex column `a`. `a` contains two subcolumns`b` and `c`. The output schema will include two columns `b` and `c` as the 'Name as' condition is `$$`.
Copy file name to clipboardExpand all lines: articles/data-factory/data-flow-select.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,22 +13,22 @@ ms.date: 03/18/2020
13
13
14
14
Use the select transformation to rename, drop, or reorder columns. This transformation doesn't alter row data, but chooses which columns are propagated downstream. This process is called
15
15
16
-
In a select transformation, users can specify fixed mappings, use patterns to do rule-based mapping or enable auto mapping. Fixed and rule-based mappings can both be used within the same select transformation. If a column doesn't match one of the defined mappings, it will be dropped.
16
+
In a select transformation, users can specify fixed mappings, use patterns to do rule-based mapping, or enable auto mapping. Fixed and rule-based mappings can both be used within the same select transformation. If a column doesn't match one of the defined mappings, it will be dropped.
17
17
18
18
## Fixed mapping
19
19
20
-
If there are less than 50 columns defined in your projection, all defined columns will have a fixed mapping by default. A fixed mapping takes a defined, incoming column and maps it an exact name.
20
+
If there are fewer than 50 columns defined in your projection, all defined columns will have a fixed mapping by default. A fixed mapping takes a defined, incoming column and maps it an exact name.
> You can't map or rename a drifted column using a fixed mapping
26
26
27
-
### Mapping hierarchal columns
27
+
### Mapping hierarchical columns
28
28
29
-
Fixed mappings can be used to map a sub-column of a hierarchal column to a toplevel column. If you have a defined hierarchy, use the column dropdown to select a sub-column. The select transformation will create a new column with the value and data type of the sub-column.
29
+
Fixed mappings can be used to map a subcolumn of a hierarchical column to a top-level column. If you have a defined hierarchy, use the column dropdown to select a subcolumn. The select transformation will create a new column with the value and data type of the subcolumn.
Use `$$` syntax to reference the input name of a matched column. Using the above image as an example, say a user wants to match on all string columns whose names are shorter than six characters. If one incoming column was named `test`, the expression `$$ + '_short'` will rename the column `test_short`. If thats the only mapping that exists, all columns that don't meet the condition will be dropped from the outputted data.
45
+
Use `$$` syntax to reference the input name of a matched column. Using the above image as an example, say a user wants to match on all string columns whose names are shorter than six characters. If one incoming column was named `test`, the expression `$$ + '_short'` will rename the column `test_short`. If that's the only mapping that exists, all columns that don't meet the condition will be dropped from the outputted data.
46
46
47
47
Patterns match both drifted and defined columns. To see which defined columns are mapped by a rule, click the eyeglasses icon next to the rule. Verify your output using data preview.
48
48
49
49
### Regex mapping
50
50
51
-
If you click the downward chevron icon, you can specify a regexmapping condition. A regexmapping condition matches all column names that match the specified regex condition. This can be used in combination with standard rule-based mappings.
51
+
If you click the downward chevron icon, you can specify a regex-mapping condition. A regex-mapping condition matches all column names that match the specified regex condition. This can be used in combination with standard rule-based mappings.
@@ -58,23 +58,23 @@ If you have multiple regex matches in your column name, you can refer to specifi
58
58
59
59
### Rule-based hierarchies
60
60
61
-
If your defined projection has a hierarchy, you can use rule-based mapping to map the hierarchies sub-columns. Specify a matching condition and the complex column whose sub-columns you wish to map. Every matched sub-column will be outputted using the 'Name as' rule specified on the right.
61
+
If your defined projection has a hierarchy, you can use rule-based mapping to map the hierarchies subcolumns. Specify a matching condition and the complex column whose subcolumns you wish to map. Every matched subcolumn will be outputted using the 'Name as' rule specified on the right.
The above example matches on all sub-columns of complex column `a`. `a` contains two sub-columns`b` and `c`. The output schema will include two columns `b` and `c` as the 'Name as' condition is `$$`.
65
+
The above example matches on all subcolumns of complex column `a`. `a` contains two subcolumns`b` and `c`. The output schema will include two columns `b` and `c` as the 'Name as' condition is `$$`.
66
66
67
67
### Parameterization
68
68
69
-
You can parameterize field mapping using rule-based mapping. Use the keyword ```name``` to match incoming column names against a parameter. For example, if you have a data flow parameter ```mycolumn```, you can create a rule that matches any column name that is equal to ```mycolumn```. You can rename the matched column to a hard-coded string such as 'business key' and reference it explicitly. In this example, the matching condition is ```name == $mycolumn``` and the name condition is 'business key'.
69
+
You can parameterize column names using rule-based mapping. Use the keyword ```name``` to match incoming column names against a parameter. For example, if you have a data flow parameter ```mycolumn```, you can create a rule that matches any column name that is equal to ```mycolumn```. You can rename the matched column to a hard-coded string such as 'business key' and reference it explicitly. In this example, the matching condition is ```name == $mycolumn``` and the name condition is 'business key'.
70
70
71
71
## Auto mapping
72
72
73
-
When adding a select transformation, **Auto mapping** can be enabled by switching the Auto mapping slider. With auto mapping, the select transformation maps all incoming columns, excluding duplicates, with the same name as their input. This will include drifted columns which means the output data may contain columns not defined in your schema. For more information on drifted columns, see [schema drift](concepts-data-flow-schema-drift.md).
73
+
When adding a select transformation, **Auto mapping** can be enabled by switching the Auto mapping slider. With auto mapping, the select transformation maps all incoming columns, excluding duplicates, with the same name as their input. This will include drifted columns, which means the output data may contain columns not defined in your schema. For more information on drifted columns, see [schema drift](concepts-data-flow-schema-drift.md).
With auto mapping on, the select transformation will honor the skip duplicate settings that are enabled and provide a new alias for the existing columns. Aliasing is useful when doing multiple joins or lookups on the same stream and in self-join scenarios.
77
+
With auto mapping on, the select transformation will honor the skip duplicate settings and provide a new alias for the existing columns. Aliasing is useful when doing multiple joins or lookups on the same stream and in self-join scenarios.
78
78
79
79
## Duplicate columns
80
80
@@ -84,7 +84,7 @@ By default, the select transformation drops duplicate columns in both the input
84
84
85
85
## Ordering of columns
86
86
87
-
The order of mappings determines the order of the output columns. If an input column is mapped multiple times, only the first mapping will be honored. For any duplicate column dropping, the first match will kept.
87
+
The order of mappings determines the order of the output columns. If an input column is mapped multiple times, only the first mapping will be honored. For any duplicate column dropping, the first match will be kept.
88
88
89
89
## Data flow script
90
90
@@ -93,7 +93,7 @@ The order of mappings determines the order of the output columns. If an input co
0 commit comments