Skip to content

Commit 34cf651

Browse files
authored
Merge pull request #108869 from linda33wj/master
Update ADF copy overview
2 parents 77d95b1 + 47e98c2 commit 34cf651

File tree

2 files changed

+58
-1
lines changed

2 files changed

+58
-1
lines changed

articles/data-factory/copy-activity-overview.md

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ ms.reviewer: douglasl
1010
ms.service: data-factory
1111
ms.workload: data-services
1212
ms.topic: conceptual
13-
ms.date: 03/11/2020
13+
ms.date: 03/24/2020
1414
ms.author: jingwang
1515

1616
---
@@ -179,6 +179,63 @@ See [Schema and data type mapping](copy-activity-schema-and-type-mapping.md) for
179179

180180
By default, the Copy activity stops copying data and returns a failure when source data rows are incompatible with sink data rows. To make the copy succeed, you can configure the Copy activity to skip and log the incompatible rows and copy only the compatible data. See [Copy activity fault tolerance](copy-activity-fault-tolerance.md) for details.
181181

182+
## Add additional columns during copy
183+
184+
In addition to copying data from source data store to sink, you can also configure to add additional data columns to copy along to sink. For example:
185+
186+
- When copy from file-based source, store the relative file path as an additional column to trace from which file the data comes from.
187+
- Add a column with ADF expression, to attach ADF system variables like pipeline name/pipeline id, or store other dynamic value from upstream activity's output.
188+
- Add a column with static value to meet your downstream consumption need.
189+
190+
You can find the following configuration on copy activity source tab:
191+
192+
![Add additional columns in copy activity](./media/copy-activity-overview/copy-activity-add-additional-columns.png)
193+
194+
To configure it programmatically, add the `additionalColumns` property in your copy activity source:
195+
196+
| Property | Description | Required |
197+
| --- | --- | --- |
198+
| additionalColumns | Add additional data columns to copy to sink.<br><br>Each object under the `additionalColumns` array represents an extra column. The `name` defines the column name, and the `value` indicates the data value of that column.<br><br>Allowed data values are:<br>- **`$$FILEPATH`** - a reserved variable indicates to store the source files' relative path to the folder path specified in dataset. Apply to file-based source.<br>- **Expression**<br>- **Static value** | No |
199+
200+
**Example:**
201+
202+
```json
203+
"activities":[
204+
{
205+
"name": "CopyWithAdditionalColumns",
206+
"type": "Copy",
207+
"inputs": [...],
208+
"outputs": [...],
209+
"typeProperties": {
210+
"source": {
211+
"type": "<source type>",
212+
"additionalColumns": [
213+
{
214+
"name": "filePath",
215+
"value": "$$FILEPATH"
216+
},
217+
{
218+
"name": "pipelineName",
219+
"value": {
220+
"value": "@pipeline().Pipeline",
221+
"type": "Expression"
222+
}
223+
},
224+
{
225+
"name": "staticValue",
226+
"value": "sampleValue"
227+
}
228+
],
229+
...
230+
},
231+
"sink": {
232+
"type": "<sink type>"
233+
}
234+
}
235+
}
236+
]
237+
```
238+
182239
## Next steps
183240
See the following quickstarts, tutorials, and samples:
184241

29.1 KB
Loading

0 commit comments

Comments
 (0)