You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-*SECRET: Microsoft Entra application service principal key*
183
-
- Minimum RBAC roles required: Storage blob data contributor, Storage blob data contributor, Storage blob data owner, or Storage blob data reader
183
+
- Minimum RBAC roles required: Storage blob data contributor, Storage blob data contributor, Storage blob data owner, or Storage blob data reader
184
184
185
185
- Authenticating with Storage account key
186
186
187
187
-*IDENTITY: A constant with a value of 'Storage Account Key'*
188
-
-*SECRET: Storage account key*
188
+
-*SECRET: Storage account key*
189
189
190
190
- Authenticating with [Managed Identity](/azure/sql-data-warehouse/load-data-from-azure-blob-storage-using-polybase#authenticate-using-managed-identities-to-load-optional) (VNet Service Endpoints)
191
191
192
192
-*IDENTITY: A constant with a value of 'Managed Identity'*
193
-
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra registered [logical server in Azure](/azure/azure-sql/database/logical-servers). When using a dedicated SQL pool (formerly SQL DW) that is not associated with a Synapse Workspace this RBAC role is not required, but the managed identity requires Access Control List (ACL) permissions on the target objects to enable read access to the source files
193
+
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra registered [logical server in Azure](/azure/azure-sql/database/logical-servers). When using a dedicated SQL pool (formerly SQL DW) that is not associated with a Synapse Workspace this RBAC role is not required, but the managed identity requires Access Control List (ACL) permissions on the target objects to enable read access to the source files
194
194
195
195
- Authenticating with a Microsoft Entra user
196
196
197
197
-*CREDENTIAL isn't required*
198
-
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra user
198
+
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra user
199
199
200
200
#### *ERRORFILE = Directory Location*
201
201
@@ -218,23 +218,23 @@ If ERRORFILE has the full path of the storage account defined, then the ERRORFIL
218
218
- Authenticating with Shared Access Signatures (SAS)
219
219
-*IDENTITY: A constant with a value of 'Shared Access Signature'*
220
220
-*SECRET: The*[*shared access signature*](/azure/storage/common/storage-sas-overview)*provides delegated access to resources in your storage account.*
-*SECRET: Microsoft Entra application service principal key*
226
-
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner
226
+
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner
227
227
228
228
> [!NOTE]
229
229
> Use the OAuth 2.0 token endpoint **V1**
230
230
231
231
- Authenticating with [Managed Identity](/azure/sql-data-warehouse/load-data-from-azure-blob-storage-using-polybase#authenticate-using-managed-identities-to-load-optional) (VNet Service Endpoints)
232
232
-*IDENTITY: A constant with a value of 'Managed Identity'*
233
-
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra registered SQL Database server
233
+
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra registered SQL Database server
234
234
235
235
- Authenticating with a Microsoft Entra user
236
236
-*CREDENTIAL isn't required*
237
-
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra user
237
+
- Minimum RBAC roles required: Storage blob data contributor or Storage blob data owner for the Microsoft Entra user
238
238
239
239
Using a storage account key with ERRORFILE_CREDENTIAL is not supported.
240
240
@@ -345,7 +345,7 @@ GRANT ALTER on SCHEMA::HR to [[email protected]];
345
345
346
346
## Remarks
347
347
348
-
The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data and command parameters. Source files or parameters (such as ROW TERMINATOR or FIELD TERMINATOR) that use invalid characters may be interpreted incorrectly by the COPY statement and cause unexpected results such as data corruption, or other failures. Make sure your source files and parameters are UTF-8 or UTF-16 compliant before you invoke the COPY statement.
348
+
The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data and command parameters. Source files or parameters (such as ROW TERMINATOR or FIELD TERMINATOR) that use invalid characters might be interpreted incorrectly by the COPY statement and cause unexpected results such as data corruption, or other failures. Make sure your source files and parameters are UTF-8 or UTF-16 compliant before you invoke the COPY statement.
349
349
350
350
## Examples
351
351
@@ -488,11 +488,11 @@ The COPY command has better performance depending on your workload.
488
488
489
489
- Compressed files can't be split automatically. For best loading performance, consider splitting your input into multiple files when loading compressed CSVs.
490
490
491
-
- Large uncompressed CSV files can be split and loaded in parallel automatically, so there's no need to manually split uncompressed CSV files in most cases. In certain cases where auto file splitting isn't feasible due to data characteristics, manually splitting large CSVs may still benefit performance.
491
+
- Large uncompressed CSV files can be split and loaded in parallel automatically, so there's no need to manually split uncompressed CSV files in most cases. In certain cases where auto file splitting isn't feasible due to data characteristics, manually splitting large CSVs might still benefit performance.
492
492
493
493
### What is the file splitting guidance for the COPY command loading compressed CSV files?
494
494
495
-
Guidance on the number of files is outlined in the following table. Once the recommended number of files are reached, you have better performance the larger the files. The number of files is determined by number of compute nodes multiplied by 60. For example, at 6000DWU we have 12 compute nodes and 12*60 = 720 partitions. For a simple file splitting experience, refer to [How to maximize COPY load throughput with file splits](https://techcommunity.microsoft.com/t5/azure-synapse-analytics/how-to-maximize-copy-load-throughput-with-file-splits/ba-p/1314474).
495
+
Guidance on the number of files is outlined in the following table. Once the recommended number of files are reached, you have better performance the larger the files. The number of files is determined by number of compute nodes multiplied by 60. For example, at 6000DWU we have 12 compute nodes and 12*60 = 720 partitions. For a simple file splitting experience, refer to [How to maximize COPY load throughput with file splits](https://techcommunity.microsoft.com/blog/azuresynapseanalyticsblog/how-to-maximize-copy-load-throughput-with-file-splits/1314474).
496
496
497
497
| DWU | #Files |
498
498
| :---: | :---: |
@@ -523,7 +523,7 @@ There are no limitations on the number or size of files; however, for best perfo
523
523
524
524
### Are there any known issues with the COPY statement?
525
525
526
-
If you have an Azure Synapse workspace that was created prior to December 7, 2020, you may run into a similar error message when authenticating using Managed Identity: `com.microsoft.sqlserver.jdbc.SQLServerException: Managed Service Identity has not been enabled on this server. Please enable Managed Service Identity and try again.`
526
+
If you have an Azure Synapse workspace that was created prior to December 7, 2020, you might run into a similar error message when authenticating using Managed Identity: `com.microsoft.sqlserver.jdbc.SQLServerException: Managed Service Identity has not been enabled on this server. Please enable Managed Service Identity and try again.`
527
527
528
528
Follow these steps to work around this issue by re-registering the workspace's managed identity:
529
529
@@ -534,7 +534,7 @@ Follow these steps to work around this issue by re-registering the workspace's m
*ERRORFILE_CREDENTIAL* only applies to CSV files. On [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], the only supported authentication mechanism is Shared Access Signature (SAS).
708
+
*ERRORFILE_CREDENTIAL* only applies to CSV files. On [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], the only supported authentication mechanism is Shared Access Signature (SAS).
709
709
710
710
- Authenticating with Shared Access Signatures (SAS)
711
711
-*IDENTITY: A constant with a value of 'Shared Access Signature'*
712
712
-*SECRET: The*[*shared access signature*](/azure/storage/common/storage-sas-overview)*provides delegated access to resources in your storage account.*
> If you are using the same storage account for your ERRORFILE and specifying the ERRORFILE path relative to the root of the container, you do not need to specify the ERROR_CREDENTIAL.
@@ -794,20 +794,21 @@ Parser version 1.0 is available for backward compatibility only, and should be u
794
794
- If a value is not provided to a non-nullable column, the COPY command fails.
795
795
- If *MATCH_COLUMN_COUNT* is `ON`:
796
796
- The COPY command checks if the column count on each row in each file from the source matches the column count of the destination table.
797
-
- If there is a column count mismatch, the COPY command fails.
798
-
797
+
- If there is a column count mismatch, the COPY command fails.
798
+
799
799
> [!NOTE]
800
800
> *MATCH_COLUMN_COUNT* works independently from *MAXERRORS*. A column count mismatch causes `COPY INTO` to fail regardless of *MAXERRORS*.
801
801
802
-
## Using COPY INTO with OneLake (Public Preview)
803
-
You can now use `COPY INTO` to load data directly from files stored in **OneLake**, specifically from the **Files folder** of a Lakehouse. This eliminates the need for external staging accounts (such as ADLS Gen2 or Blob Storage) and enables workspace-governed, SaaS-native ingestion using Fabric permissions.
802
+
## Use COPY INTO with OneLake
803
+
804
+
You can now use `COPY INTO` to load data directly from files stored in the Fabric OneLake, specifically from the **Files folder** of a Fabric Lakehouse. This eliminates the need for external staging accounts (such as ADLS Gen2 or Blob Storage) and enables workspace-governed, SaaS-native ingestion using Fabric permissions. This functionality supports:
804
805
805
-
This functionality supports:
806
806
- Reading from `Files` folders in Lakehouses
807
-
- Workspace-to-Warehouse loads within the same tenant
807
+
- Workspace-to-warehouse loads within the same tenant
808
808
- Native identity enforcement using Microsoft Entra ID
809
+
809
810
> [!NOTE]
810
-
> This feature is currently in Public Preview.
811
+
> This feature is currently in [preview](/fabric/fundamentals/preview).
811
812
812
813
## Permissions
813
814
@@ -837,7 +838,7 @@ GO
837
838
838
839
## Remarks
839
840
840
-
The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data and command parameters. Source files or parameters (such as ROW TERMINATOR or FIELD TERMINATOR) that use invalid characters may be interpreted incorrectly by the COPY statement and cause unexpected results such as data corruption, or other failures. Make sure your source files and parameters are UTF-8 or UTF-16 compliant before you invoke the COPY statement.
841
+
The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data and command parameters. Source files or parameters (such as `ROW TERMINATOR` or `FIELD TERMINATOR`) that use invalid characters might be interpreted incorrectly by the COPY statement and cause unexpected results such as data corruption, or other failures. Make sure your source files and parameters are UTF-8 or UTF-16 compliant before you invoke the COPY statement.
841
842
842
843
## Limitations for OneLake as source (Public Preview)
843
844
@@ -849,7 +850,6 @@ The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data a
849
850
850
851
-**Contributor permissions are required on both workspaces.** The executing user must have at least Contributor role on the source Lakehouse workspace and the target Warehouse workspace.
851
852
852
-
853
853
## Examples
854
854
855
855
For more information on using COPY INTO on your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], see [Ingest data into your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] using the COPY statement](/fabric/data-warehouse/ingest-data-copy).
0 commit comments