Skip to content

Commit b985e1e

Browse files
committed
updating parameters doc
1 parent a5fb6cf commit b985e1e

File tree

7 files changed

+40
-10
lines changed

7 files changed

+40
-10
lines changed
Binary file not shown.
257 KB
Loading
112 KB
Loading
24.7 KB
Loading
56.2 KB
Loading
51.1 KB
Loading

articles/data-factory/parameters-data-flow.md

Lines changed: 40 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,14 @@ ms.author: makromer
66
ms.reviewer: daperlov
77
ms.service: data-factory
88
ms.topic: conceptual
9-
ms.date: 01/07/2020
9+
ms.date: 05/01/2020
1010
---
1111

1212
# Parameterizing mapping data flows
1313

1414
[!INCLUDE[appliesto-adf-asa-md](includes/appliesto-adf-asa-md.md)]
1515

16-
Mapping data flows in Azure Data Factory support the use of parameters. You can define parameters inside of your data flow definition, which you can then use throughout your expressions. The parameter values can be set by the calling pipeline via the Execute Data Flow activity. You have three options for setting the values in the data flow activity expressions:
16+
Mapping data flows in Azure Data Factory support the use of parameters. Define parameters inside of your data flow definition and use them throughout your expressions. The parameter values are set by the calling pipeline via the Execute Data Flow activity. You have three options for setting the values in the data flow activity expressions:
1717

1818
* Use the pipeline control flow expression language to set a dynamic value
1919
* Use the data flow expression language to set a dynamic value
@@ -37,19 +37,44 @@ You can quickly add additional parameters by selecting **New parameter** and spe
3737

3838
![Data flow parameter expression](media/data-flow/new-parameter-expression.png "Data flow parameter expression")
3939

40-
### Passing in a column name as a parameter
40+
## Assign parameter values from a pipeline
4141

42-
A common pattern is to pass in a column name as a parameter value. To reference the column associated with the parameter, use the `byName()` function. Remember to cast the column to its appropriate type with a casting function such as `toString()`.
42+
Once you've created a data flow with parameters, you can execute it from a pipeline with the Execute Data Flow Activity. After you add the activity to your pipeline canvas, you will be presented with the available data flow parameters in the activity's **Parameters** tab.
4343

44-
For example, if you wanted to map a string column based upon a parameter `columnName`, you can add a derived column transformation equal to `toString(byName($columnName))`.
44+
When assigning parameter values, you can use either the [pipeline expression language](control-flow-expression-language-functions.md) or the [data flow expression language](data-flow-expression-functions.md) based on spark types. Each mapping data flow can have any combination of pipeline and data flow expression parameters.
4545

46-
![Passing in a column name as a parameter](media/data-flow/parameterize-column-name.png "Passing in a column name as a paramete")
46+
![Setting a Data Flow parameter](media/data-flow/parameter-assign.png "Setting a Data Flow parameter")
4747

48-
## Assign parameter values from a pipeline
48+
### Pipeline expression parameters
4949

50-
Once you've created your data flow with parameters, you can execute it from a pipeline with the Execute Data Flow Activity. After you add the activity to your pipeline canvas, you will be presented with the available data flow parameters in the activity's **Parameters** tab.
50+
Pipeline expression parameters allow you to reference system variables, functions, pipeline parameters, and variables similar to other pipeline activities. When you click **Pipeline expression**, a side-nav will open allowing you to enter an expression using the expression builder.
5151

52-
![Setting a Data Flow parameter](media/data-flow/parameter-assign.png "Setting a Data Flow parameter")
52+
![Setting a Data Flow parameter](media/data-flow/parameter-pipeline.png "Setting a Data Flow parameter")
53+
54+
When referenced, pipeline parameters are evaluated and then their value is used in the data flow expression language. The pipeline expression type doesn't need to match the data flow parameter type.
55+
56+
#### Example
57+
58+
Say you have an integer parameter `intParam` that is referencing a pipeline parameter of type String, `@pipeline.parameters.pipelineParam`.
59+
60+
![Setting a Data Flow parameter](media/data-flow/parameter-pipeline2.png "Setting a Data Flow parameter")
61+
62+
`@pipeline.parameters.pipelineParam` is assigned a value of `abs(1)` at runtime.
63+
64+
![Setting a Data Flow parameter](media/data-flow/parameter-pipeline4.png "Setting a Data Flow parameter")
65+
66+
When `$intParam` is referenced in an expression such as a derived column, it will evaluate `abs(1)` return `1`.
67+
68+
![Setting a Data Flow parameter](media/data-flow/parameter-pipeline4.png "Setting a Data Flow parameter")
69+
70+
### Data flow expression parameters
71+
72+
Select **Data flow expression** will open up the data flow expression builder. You will be able to reference functions, other parameters and any defined schema column throughout your data flow. This expression will be evaluated as is when referenced.
73+
74+
> [!Note]
75+
> If you pass in an invalid expression or reference a schema column that doesn't exist in that transformation, the parameter will evaluate to null.
76+
77+
### String parameters
5378

5479
If your parameter data type is string, when you click the text box to set parameter values, you can choose to enter either a pipeline or a data flow expression. If you choose pipeline expression, you will be presented with the pipeline expression panel. Make sure to include pipeline functions inside string interpolation syntax using `'@{<expression>}'`, for example:
5580

@@ -60,11 +85,16 @@ If your parameter is not of type string, you will always be presented with the D
6085
* ```toInteger(Role)```
6186
* ```'this is my static literal string'```
6287

63-
Each mapping data flow can have any combination of pipeline and data flow expression parameters.
6488

6589
![Data flow parameters sample](media/data-flow/parameter-example.png "Data flow parameters sample")
6690

91+
### Passing in a column name as a parameter
92+
93+
A common pattern is to pass in a column name as a parameter value. To reference the column associated with the parameter, use the `byName()` function. Remember to cast the column to its appropriate type with a casting function such as `toString()`.
6794

95+
For example, if you wanted to map a string column based upon a parameter `columnName`, you can add a derived column transformation equal to `toString(byName($columnName))`.
96+
97+
![Passing in a column name as a parameter](media/data-flow/parameterize-column-name.png "Passing in a column name as a paramete")
6898

6999
## Next steps
70100
* [Execute data flow activity](control-flow-execute-data-flow-activity.md)

0 commit comments

Comments
 (0)