Merge pull request #89281 from kromerm/dataflow-1

Dataflow Updates to Pivot and Debug

Author: tiburd

articles/data-factory/data-flow-pivot.md

@@ -51,6 +51,12 @@ Use the ADF Data Flow Expression Language to describe the pivoted column transfo
 
 The Pivot transformation will produce new column names that are dynamic based on your incoming data. The Pivot Key produces the values for each new column name. If you do not specify individual values and wish to create dynamic column names for each unique value in your Pivot Key, then the UI will not display the metadata in Inspect and there will be no column propagation to the Sink transformation. If you set values for the Pivot Key, then ADF can determine the new column names and those column names will be available to you in the Inspect and Sink mapping.
 
+### Generate a new model from dynamic columns
+
+Pivot generates new column names dynamically based on row values. You can turn those new columns into metadata that can be referenced later in your data flow. To do this, click on the Data Preview tab. All new columns generated by the your Pivot transformation appear with a "drifted" icon in the table header. Click on the "Map drifted" button to turn those new columns into metadata, making them part of your data flow's model.
+
+![Pivot columns](media/data-flow/newpivot1.png "Map drifted Pivot columns")
+
 ### Landing new columns in Sink
 
 Even with dynamic column names in Pivot, you can still sink your new column names and values into your destination store. Just set "Allow Schema Drift" to on in your Sink settings. You will not see the new dynamic names in your column metadata, but the schema drift option will allow you to land the data.

articles/data-factory/iterative-development-debugging.md

@@ -46,6 +46,10 @@ You can visualize all the debug runs that are in progress for your data factory
 
 ![Sample list of active debug runs](media/iterative-development-debugging/view-debug-runs-image2.png)
 
+If you have active Data Flow debug sessions, those sessions will appear on the bottom part of the active debug window. You can select an active data flow session and stop the respective cluster.
+
+![Sample list of active data flow debug runs](media/data-flow/dfsessions.png)
+
 ## Monitoring debug runs
 
 The test runs initiated with the **Debug** capability are not available in the list on the **Monitor** tab. You can only see runs triggered with **Trigger Now**, **Schedule**, or **Tumbling Window** triggers in the **Monitor** tab. You can see the last test run initiated with the **Debug** capability in the **Output** window of the pipeline canvas.