Merge pull request #227709 from chez-charlie/release-branch-1

Release branch 1

Author: American-Dipper

articles/data-factory/TOC.yml

@@ -761,6 +761,8 @@ items:
     - name: Set Variable activity
       href: control-flow-set-variable-activity.md
       displayName: variables, control flow
+    - name: Set Pipeline Return Value
+      href: tutorial-pipeline-return-value.md
     - name: Switch activity
       href: control-flow-switch-activity.md
       displayName: control flow

articles/data-factory/control-flow-set-variable-activity.md

@@ -27,31 +27,24 @@ To use a Set Variable activity in a pipeline, complete the following steps:
 
 2. Search for _Set Variable_ in the pipeline Activities pane, and drag a Set Variable activity to the pipeline canvas.
 
-3. Select the Set Variable activity on the canvas if it isn't already selected, and then click the **Settings** tab to edit its details.
+3. Select the Set Variable activity on the canvas if it isn't already selected, and then select the **Settings** tab to edit its details.
 
-4. Select the variable for the Name property.
+4. Select **Pipeline variable** for your **Variable type**.
 
-5. Enter an expression to set the value for the variables. This expression can be a literal string expression, or any combination of dynamic [expressions, functions](control-flow-expression-language-functions.md), [system variables](control-flow-system-variables.md), or [outputs from other activities](how-to-expression-language-functions.md#examples-of-using-parameters-in-expressions).
+5. Select the variable for the Name property.
+
+6. Enter an expression to set the value for the variables. This expression can be a literal string expression, or any combination of dynamic [expressions, functions](control-flow-expression-language-functions.md), [system variables](control-flow-system-variables.md), or [outputs from other activities](how-to-expression-language-functions.md#examples-of-using-parameters-in-expressions).
 
 :::image type="content" source="media/control-flow-set-variable-activity/set-variable-activity.png" alt-text="Screenshot of the UI for a Set variable activity.":::
 
 ## Setting a pipeline return value with UI
 
-The Set Variable activity now allows you to set a pipeline return value (preview). The pipeline return value is a system variable that allows you to customize a value that can be consumed by a parent pipeline and used downstream in your pipeline. 
-
-To set a pipeline return value, complete the following steps:
-
-1. Search for _Set Variable_ in the pipeline Activities pane, and drag a Set Variable activity to the pipeline canvas.
-
-2. Select the Set Variable activity on the canvas if it isn't already selected, and then click the **Settings** tab to edit its details.
-
-3. Select **Pipeline return value (preview)** for your **Variable type**.
+We have expanded Set Variable activity to include a special system variable, named _Pipeline Return Value_. This allows communication from the child pipeline to the calling pipeline, in the following scenario. 
 
-4. Enter a **Name** for your variable and select the **Type** from the drop-down menu. 
+You don't need to define the variable, before using it. For more information, see [Pipeline Return Value](tutorial-pipeline-return-value.md)
 
-5. Enter an expression to set the value for the pipeline return value. This expression can be a literal string expression, or any combination of dynamic [expressions, functions](control-flow-expression-language-functions.md), [system variables](control-flow-system-variables.md), or [outputs from other activities](how-to-expression-language-functions.md#examples-of-using-parameters-in-expressions).
 
-:::image type="content" source="media/control-flow-set-variable-activity/set-pipeline-return-value.png" alt-text="Screenshot of the UI for setting a pipeline return value.":::
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-00-paradigm.png" alt-text="Screenshot with ExecutePipeline Activity.":::
 
 ## Type properties
 
@@ -66,7 +59,7 @@ value | String literal or expression object value that the variable is assigned
 
 ## Incrementing a variable
 
-A common scenario involving variables is to use a variable as an iterator within an **Until** or **ForEach** activity. In a **Set variable** activity, you can't reference the variable being set in the `value` field. To work around this limitation, set a temporary variable and then create a second **Set variable** activity. The second **Set variable** activity sets the value of the iterator to the temporary variable. 
+A common scenario involving variable is to use a variable as an iterator within an **Until** or **ForEach** activity. In a **Set variable** activity, you can't reference the variable being set in the `value` field. To work around this limitation, set a temporary variable and then create a second **Set variable** activity. The second **Set variable** activity sets the value of the iterator to the temporary variable. 
 
 Below is an example of this pattern:
 
@@ -126,7 +119,7 @@ Below is an example of this pattern:
 }
 ```
 
-Variables are currently scoped at the pipeline level. This means that they're not thread safe and can cause unexpected and undesired behavior if they are accessed from within a parallel iteration activity such as a ForEach loop, especially when the value is also being modified within that foreach activity.
+Variables are currently scoped at the pipeline level. This means that they're not thread safe and can cause unexpected and undesired behavior if they're accessed from within a parallel iteration activity such as a ForEach loop, especially when the value is also being modified within that foreach activity.
 
 
 ## Next steps

articles/data-factory/media/pipeline-return-value/pipeline-return-value-00-paradigm.png

@@ -0,0 +1,94 @@
+---
+title: Set Pipeline Return Value
+titleSuffix: Azure Data Factory & Azure Synapse
+description: Learn how to use the Set Variable activity to send information from child pipeline to main pipeline
+ms.service: data-factory
+ms.subservice: orchestration
+ms.custom: synapse
+ms.topic: conceptual
+ms.date: 2/12/2022
+author: chez-charlie
+ms.author: chez
+ms.reviewer: jburchel
+---
+
+# Set Pipeline Return Value in Azure Data Factory and Azure Synapse Analytics
+[!INCLUDE[appliesto-adf-asa-md](includes/appliesto-adf-asa-md.md)]
+
+In the calling pipeline-child pipeline paradigm, you can use the [Set Variable activity](control-flow-set-variable-activity.md) to return values from the child pipeline to the calling pipeline. In the following scenario, we have a child pipeline through [Execute Pipeline Activity](control-flow-execute-pipeline-activity.md). And we want to __retrieve information from the child pipeline__, to then be used in the calling pipeline.
+
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-00-paradigm.png" alt-text="Screenshot with ExecutePipeline Activity.":::
+
+Introduce pipeline return value, a dictionary of key value pairs, that allows communications between child pipelines and parent pipeline.
+
+## Prerequisite - Calling a Child Pipeline
+
+The prerequisite of the use case, is that you've an [Execute Pipeline Activity](control-flow-execute-pipeline-activity.md), calling a child pipeline. It's important that we enabled _Wait on Completion_ for the activity
+
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-01-execute-pipeline-setting.png" alt-text="Screenshot setting ExecutePipeline Activity to wait for completion.":::
+
+
+## Configure Pipeline Return Value in Child Pipeline
+
+We've expanded the [Set Variable activity](control-flow-set-variable-activity.md) to include system variables _Pipeline Return Value_. You don't need to define them at pipeline level (as opposed to any other variables you use in the pipeline).
+
+1. Search for _Set Variable_ in the pipeline Activities pane, and drag a Set Variable activity to the pipeline canvas.
+1. Select the Set Variable activity on the canvas if it isn't already selected, and then its **Variables** tab, to edit its details.
+1. Choose _Pipeline return value_ for variable type.
+1. Select _New_ to add a new key value pair.
+1. You can add reasonable number of key value pairs, bounded by size limit of returning json.
+
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-02-child-pipeline.png" alt-text="Screenshot shows the ui for pipeline return value.":::
+
+There are a few options for value types, including
+
+Type Name | Description
+-------- | ----------- 
+String | The most straight forward of all. It expects a string value.
+Expression | It allows you to reference output from previous activities.
+Array | It expects an array of _string values_. Press "enter" key to separate values in the array
+Boolean | True or False
+Null | Signal place holder status; the value is constant _null_
+Int | It expects a numerical value of integer type
+Float | It expects a numerical value of float type
+Object | __Warning__ complicated use cases only. It allows you to embed a list of key value pairs type for the value
+
+Value of object type is defined as follows:
+
+``` json
+[{"key": "myKey1", "value": {"type": "String", "content": "hello world"}}, 
+ {"key": "myKey2", "value": {"type": "String", "content": "hi"}}
+]
+```
+
+## Retrieving Value in Calling Pipeline
+
+The pipeline return value of the child pipeline becomes the activity output of the Execute Pipeline Activity. You can retrieve the information with _@activity('Execute Pipeline1').output.pipelineReturnValue.keyName_. The use case is limitless. For instance, you may use
+* An _int_ value from child pipeline to define the wait period for a [wait activity](control-flow-wait-activity.md)
+* A _sting_ value to define the URL for the [Web activity](control-flow-web-activity.md)
+* An _expression_ value payload for a [script activity](transform-data-using-script.md) for logging purposes.
+
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-03-calling-pipeline.png" alt-text="Screenshot shows the calling pipeline.":::
+
+There are two noticeable call outs in referencing the pipeline return values. 
+
+1.  With _Object_ type, you may further expand into the nested json object, such as _@activity('Execute Pipeline1').output.pipelineReturnValue.keyName.nextLevelKey_
+2.  With _Array_ type, you can specify the index in the list, with _@activity('Execute Pipeline1').output.pipelineReturnValue.keyName[0]_. The number is zero indexed, meaning that it starts with 0.
+
+> [!NOTE]
+> Please make sure that the _keyName_ you are referencing exists in your child pipeline. ADF expression builder can _not_ confirm the referential check for you.
+> Pipeline will fail if the key referenced is missing in the payload
+
+## Special Considerations
+
+You may have multiple Set Pipeline Return value activities in a pipeline. However, please ensure that only one gets to run in a pipeline.
+
+:::image type="content" source="media/pipeline-return-value/pipeline-return-value-04-multiple.png" alt-text="Screenshot with Pipeline Return Value and Branching.":::
+
+To avoid missing key situation in the calling pipeline, described above, we encourage you to have the same list of keys for all branches in child pipeline. Consider using _null_ types for keys that don't have values, in a specific branch.
+
+## Next steps
+Learn about another related control flow activity: 
+- [Set Variable Activity](control-flow-set-variable-activity.md)
+- [Append Variable Activity](control-flow-append-variable-activity.md)
+