Skip to content

Commit fb7a828

Browse files
authored
Merge pull request #100540 from djpmsft/docUpdates
Alter row script and new branch update
2 parents 834ac34 + ff7864a commit fb7a828

File tree

7 files changed

+55
-30
lines changed

7 files changed

+55
-30
lines changed

articles/data-factory/data-flow-alter-row.md

Lines changed: 46 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,52 +1,80 @@
11
---
2-
title: Mapping data flow Alter Row Transformation
3-
description: How to update database target using Azure Data Factory mapping data flow Alter Row Transformation
2+
title: Alter row transformation in mapping data flow
3+
description: How to update database target using the alter row transformation in mapping data flow
44
author: kromerm
55
ms.author: makromer
6+
ms.reviewer: daperlov
67
ms.service: data-factory
78
ms.topic: conceptual
89
ms.custom: seo-lt-2019
9-
ms.date: 03/12/2019
10+
ms.date: 01/08/2020
1011
---
1112

12-
# Azure Data Factory Alter Row Transformation
13+
# Alter row transformation in mapping data flow
1314

1415
Use the Alter Row transformation to set insert, delete, update, and upsert policies on rows. You can add one-to-many conditions as expressions. These conditions should be specified in order of priority, as each row will be marked with the policy corresponding to the first-matching expression. Each of those conditions can result in a row (or rows) being inserted, updated, deleted, or upserted. Alter Row can produce both DDL & DML actions against your database.
1516

16-
17-
1817
![Alter row settings](media/data-flow/alter-row1.png "Alter Row Settings")
1918

20-
> [!NOTE]
21-
> Alter Row transformations will only operate on database or CosmosDB sinks in your data flow. The actions that you assign to rows (insert, update, delete, upsert) will not occur during debug sessions. You must add an Execute Data Flow task to a pipeline and use pipeline debug or triggers to enact the alter row policies on your database tables.
19+
Alter Row transformations will only operate on database or CosmosDB sinks in your data flow. The actions that you assign to rows (insert, update, delete, upsert) won't occur during debug sessions. Run an Execute Data Flow activity in a pipeline to enact the alter row policies on your database tables.
2220

23-
## Indicate a default row policy
21+
## Specify a default row policy
2422

25-
Create an Alter Row transformation and specify a row policy with a condition of `true()`. Each row that does not meet any of the previously defined expressions will be marked for the specified row policy. By default, each row that does not meet any conditional expression will be marked for `Insert`.
23+
Create an Alter Row transformation and specify a row policy with a condition of `true()`. Each row that doesn't match any of the previously defined expressions will be marked for the specified row policy. By default, each row that doesn't match any conditional expression will be marked for `Insert`.
2624

27-
![Alter row one policy](media/data-flow/alter-row4.png "Alter row one policy")
25+
![Alter row policy](media/data-flow/alter-row4.png "Alter row policy")
2826

2927
> [!NOTE]
3028
> To mark all rows with one policy, you can create a condition for that policy and specify the condition as `true()`.
3129
32-
## View policies
30+
## View policies in data preview
3331

34-
Turn on Data Flow Debug mode to view the results of your alter row policies in the Data Preview pane. Executing an alter row in Data Flow Debug mode will not produce DDL or DML actions against your target. In order to have those actions occur, execute the data flow inside an Execute Data Flow activity within a pipeline.
32+
Use [debug mode](concepts-data-flow-debug-mode.md) to view the results of your alter row policies in the data preview pane. A data preview of an alter row transformation won't produce DDL or DML actions against your target.
3533

3634
![Alter row policies](media/data-flow/alter-row3.png "Alter Row Policies")
3735

38-
This will allow you to verify and view the state of each row based on your conditions. There are icon represents for each insert, update, delete, and upsert action that will occur in your data flow, indicating which action will take place when you execute the data flow inside a pipeline.
36+
Each alter row policy is represented by an icon that indicates whether an insert, update, upsert, or deleted action will occur. The top header shows how many rows are affected by each policy in the preview.
3937

40-
## Sink settings
38+
## Allow alter row policies in sink
4139

42-
You must have a database sink type for Alter Row to work. In the sink Settings, you should set each action corresponding to your Alter Row conditions to be allowed.
40+
For the alter row policies to work, the data stream must write to a database or Cosmos sink. In the **Settings** tab in your sink, enable which alter row policies are allowed for that sink.
4341

4442
![Alter row sink](media/data-flow/alter-row2.png "Alter Row Sink")
4543

46-
The default behavior in ADF Data Flow with database sinks is to insert rows. If you want to allow updates, upserts, and deletes as well, you must also check these boxes in the sink to allow the actions.
44+
The default behavior is to only allow inserts. To allow updates, upserts, or deletes, check the box in the sink corresponding to that condition. If updates, upserts, or, deletes are enabled, you must specify which key columns in the sink to match on.
4745

4846
> [!NOTE]
49-
> If your inserts, updates, or upserts modify the schema of the target table in the sink, your data flow will fail. In order to modify the target schema in your database, you must choose the "Recreate table" option in the sink. This will drop and recreate your table with the new schema definition.
47+
> If your inserts, updates, or upserts modify the schema of the target table in the sink, the data flow will fail. To modify the target schema in your database, choose **Recreate table** as the table action. This will drop and recreate your table with the new schema definition.
48+
49+
## Data flow script
50+
51+
### Syntax
52+
53+
```
54+
<incomingStream>
55+
alterRow(
56+
insertIf(<condition>?),
57+
updateIf(<condition>?),
58+
deleteIf(<condition>?),
59+
upsertIf(<condition>?),
60+
) ~> <alterRowTransformationName>
61+
```
62+
63+
### Example
64+
65+
The below example is an alter row transformation named `CleanData` that takes an incoming stream `SpecifyUpsertConditions` and creates three alter row conditions. In the previous transformation, a column named `alterRowCondition` is calculated that determines whether or not a row is inserted, updated, or deleted in the database. If the value of the column has a string value that matches the alter row rule, it is assigned that policy.
66+
67+
In the Data Factory UX, this transformation looks like the below image:
68+
69+
![Alter row example](media/data-flow/alter-row4.png "Alter row example")
70+
71+
The data flow script for this transformation is in the snippet below:
72+
73+
```
74+
SpecifyUpsertConditions alterRow(insertIf(alterRowCondition == 'insert'),
75+
updateIf(alterRowCondition == 'update'),
76+
deleteIf(alterRowCondition == 'delete')) ~> AlterRow
77+
```
5078

5179
## Next steps
5280

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,23 @@
11
---
2-
title: Mapping data flow New Branch Transformation
3-
description: Azure Data Factory mapping data flow New Branch Transformation
2+
title: Multiple branches in mapping data flow
3+
description: Replicating data streams in mapping data flow with multiple branches
44
author: kromerm
55
ms.author: makromer
66
ms.reviewer: douglasl
77
ms.service: data-factory
88
ms.topic: conceptual
99
ms.custom: seo-lt-2019; seo-dt-2019
10-
ms.date: 02/12/2019
10+
ms.date: 01/08/2020
1111
---
1212

13-
# Azure Data Factory mapping data flow New Branch Transformation
13+
# Creating a new branch in mapping data flow
1414

15-
![Branch options](media/data-flow/menu.png "menu")
15+
Add a new branch to do multiple sets of operations and transformations against the same data stream. Adding a new branch is useful when you want to use the same source to for multiple sinks or for self-joining data together.
1616

17-
Branching will take the current data stream in your data flow and replicate it to another stream. Use New Branch to perform multiple sets of operations and transformations against the same data stream.
17+
A new branch can be added from the transformation list similar to other transformations. **New Branch** will only be available as an action when there's an existing transformation following the transformation you're attempting to branch.
1818

19-
Example: Your data flow has a Source Transform with a selected set of columns and data type conversions. You then place a Derived Column immediately following that Source. In the Derived Column, you've create a new field that combines first name and last name to make a new "full name" field.
19+
![Adding a new branch](media/data-flow/new-branch2.png "Adding a new branch")
2020

21-
You can treat that new stream with a set of transformations and a sink on one row and use New Branch to create a copy of that stream where you can transform that same data with a different set of transformations. By transforming that copied data in a separate branch, you can subsequently sink that data to a separate location.
21+
In the below example, the data flow is reading taxi trip data. Output aggregated by both day and vendor is required. Instead of creating two separate data flows that read from the same source, a new branch can be added. This way both aggregations can be executed as part of the same data flow.
2222

23-
> [!NOTE]
24-
> "New Branch" will only show as an action on the + Transformation menu when there is a subsequent transformation following the current location where you are attempting to branch. i.e. You will not see a "New Branch" option at the end here until you add another transformation after the Select
25-
26-
![Branch](media/data-flow/branch2.png "Branch 2")
23+
![Adding a new branch](media/data-flow/new-branch.png "Adding a new branch")
93.4 KB
Loading
-68.4 KB
Binary file not shown.
-44.2 KB
Binary file not shown.
142 KB
Loading
87.5 KB
Loading

0 commit comments

Comments
 (0)