You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/copy-activity-data-consistency.md
+7-2Lines changed: 7 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,6 +20,12 @@ ms.author: yexu
20
20
21
21
When you move data from source to destination store, Azure Data Factory copy activity provides data consistency verification to ensure the data is not only successfully copied from source to destination store, but also verified to be consistent between source and destination store. Once inconsistent data have been found during the data movement, you can either abort the copy activity or continue to copy the rest by enabling fault tolerance setting to skip inconsistent data. You can get the skipped object names by enabling session log setting in copy activity.
22
22
23
+
> [!IMPORTANT]
24
+
> This feature is currently in preview due to the limitations below:
25
+
> 1. Data consistency verification is available only for binary files copy between file-based stores with 'PreserveHierarchy' behavior in copy activity.
26
+
> 2. When you get the skipped inconsistent file names by enabling session log setting in copy activity, the completeness of log file can not be 100% guaranteed if copy activity failed.
27
+
> 3. The session log contains inconsistent files names only, where the successfully copied files are not logged.
28
+
23
29
## Supported data stores
24
30
25
31
### Source data stores
@@ -82,7 +88,6 @@ linkedServiceName | The linked service of [Azure Blob Storage](connector-azure-b
82
88
path | The path of the log file. | Specify the path that you want to store the log file. If you do not provide a path, the service creates a container for you. | No
83
89
84
90
>[!NOTE]
85
-
>- Only binary copy between file-based stores with PreserveHierarchy copy behavior supports data consistency verification in copy activity now.
86
91
>- Data consistency is not supported in staging copy scenario.
87
92
>- When copying binary files to Azure Blob Storage or Azure Data Lake Storage Gen2, copy activity does both file size and MD5 checksum verification to ensure the data consistency between source and destination stores. For copying binary files to other storage stores, copy activity does file size verification to ensure the data consistency between source and destination store.
88
93
@@ -113,7 +118,7 @@ After the copy activity run completes, you can get the result of data consistenc
113
118
Value for **VerificationResult**:
114
119
-**Verified**: Your copied data has been verified to be consistent between source and destination store.
115
120
-**NotVerified**: Your copied data has not been verified to be consistent because you have not enabled the validateDataConsistency setting.
116
-
-**Unsupported**: Your copied data has not been verified to be consistent because data consistency verification is not supported in this copy activity run.
121
+
-**Unsupported**: Your copied data has not been verified to be consistent because data consistency verification is not supported for this copy pairs.
117
122
118
123
Value for **InconsistentData**:
119
124
-**Found**: ADF copy activity has found Inconsistent data.
Copy file name to clipboardExpand all lines: articles/data-factory/copy-activity-fault-tolerance.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,7 +27,7 @@ When you copy data from source to destination store, Azure Data Factory copy act
27
27
- You can abort the copy activity when the first failure is encountered.
28
28
- You can continue to copy the rest by enabling fault tolerance to skip the incompatible data. For example, skip the duplicated row in this case. In addition, you can log the skipped data by enabling session log in copy activity.
29
29
30
-
## Fault tolerance for copying binary files (Preview)
30
+
## Fault tolerance for copying binary files
31
31
32
32
### Configuration
33
33
When you copy binary files between storage stores, you can enable fault tolerance as followings:
0 commit comments