Skip to content

Commit 5a4dbe9

Browse files
Merge pull request #34842 from WilliamDAssafMSFT/20250729-openrowset-copy-into
20250729 openrowset copy into
2 parents 13d00a2 + c25d02a commit 5a4dbe9

File tree

2 files changed

+41
-17
lines changed

2 files changed

+41
-17
lines changed

docs/t-sql/functions/openrowset-bulk-transact-sql.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ title: "OPENROWSET BULK (Transact-SQL)"
33
description: "OPENROWSET BULK operations perform bulk data manipulation operations on an external data source."
44
author: MikeRayMSFT
55
ms.author: mikeray
6-
ms.reviewer: randolphwest, hudequei, wiassaf, jovanpop
7-
ms.date: 06/25/2025
6+
ms.reviewer: randolphwest, hudequei, wiassaf, jovanpop, fresantos
7+
ms.date: 07/29/2025
88
ms.service: sql
99
ms.subservice: t-sql
1010
ms.topic: reference
@@ -661,11 +661,21 @@ The URI of the data file(s) whose data is to be read and returned as row set. Th
661661

662662
The URI may include the `*` character to match any sequence of characters, allowing `OPENROWSET` to pattern-match against the URI. Additionally, it can end with `/**` to enable recursive traversal through all subfolders.
663663

664-
The URI of the data file(s) whose data is to be read and returned as row set. The URI can reference Azure Data Lake storage or Azure Blob storage. The supported URI formats are:
664+
You can use `OPENROWSET(BULK)` to read data directly from files stored in the Fabric OneLake, specifically from the **Files folder** of a Fabric Lakehouse. This eliminates the need for external staging accounts (such as ADLS Gen2 or Blob Storage) and enables workspace-governed, SaaS-native ingestion using Fabric permissions. This functionality supports:
665+
666+
- Reading from `Files` folders in Lakehouses
667+
- Workspace-to-warehouse loads within the same tenant
668+
- Native identity enforcement using Microsoft Entra ID
669+
670+
> [!NOTE]
671+
> Fabric OneLake storage is in [preview](/fabric/fundamentals/preview). See the [limitations](../statements/copy-into-transact-sql.md#limitations-for-onelake-as-source) that are applicable both to `COPY INTO` and `OPENROWSET(BULK)`.
672+
673+
The supported URI formats are:
665674

666675
- `https://{storage}.blob.core.windows.net/[container}/{file path}`
667676
- `https://{storage}.dfs.core.windows.net/[container}/{file path}`
668677
- `abfss://[container}@{storage}.dfs.core.windows.net/{file path}`
678+
- `https://onelake.dfs.fabric.microsoft.com/<workspaceId>/<lakehouseId>/Files/{file path}` - currently in [preview](/fabric/fundamentals/preview)
669679

670680
For example:
671681

@@ -913,7 +923,7 @@ The supported features are summarized in the table:
913923
|-------------------|-----------|----------------------|
914924
| **File formats** | Parquet, CSV, JSONL (preview) | Delta, Azure Cosmos DB, JSON, relational databases |
915925
| **Authentication**| EntraID/SPN passthrough, public storage | SAS/SAK, SPN, Managed access |
916-
| **Storage** | Azure Blob Storage, Azure Data Lake Storage | Fabric OneLake |
926+
| **Storage** | Azure Blob Storage, Azure Data Lake Storage, Fabric OneLake (preview) | |
917927
| **Options** | Only full/absolute URI in `OPENROWSET` | Relative URI path in `OPENROWSET`, `DATA_SOURCE` |
918928
| **Partitioning** | You can use the `filepath()` function in a query. | |
919929

docs/t-sql/statements/copy-into-transact-sql.md

Lines changed: 27 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Use the COPY statement in Azure Synapse Analytics and Warehouse in
55
author: WilliamDAssafMSFT
66
ms.author: wiassaf
77
ms.reviewer: procha, mikeray, fresantos
8-
ms.date: 07/28/2025
8+
ms.date: 07/29/2025
99
ms.service: sql
1010
ms.subservice: t-sql
1111
ms.topic: reference
@@ -549,7 +549,7 @@ Follow these steps to work around this issue by re-registering the workspace's m
549549

550550
This article explains how to use the COPY statement in [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)] for loading from external storage accounts. The COPY statement provides the most flexibility for high-throughput data ingestion into your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)], and is as strategy to [Ingest data into your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)]](/fabric/data-warehouse/ingest-data).
551551

552-
In [!INCLUDE [fabric](../../includes/fabric.md)], the [COPY (Transact-SQL)](/sql/t-sql/statements/copy-into-transact-sql?view=fabric&preserve-view=true) statement currently supports the PARQUET and CSV file formats. For data sources, only Azure Data Lake Storage Gen2 accounts are supported.
552+
In [!INCLUDE [fabric](../../includes/fabric.md)], the [COPY (Transact-SQL)](/sql/t-sql/statements/copy-into-transact-sql?view=fabric&preserve-view=true) statement currently supports the PARQUET and CSV file formats. For data sources, Azure Data Lake Storage Gen2 accounts, and OneLake sources are supported.
553553

554554
For more information on using COPY INTO on your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], see [Ingest data into your [!INCLUDE [fabricdw](../../includes/fabric-dw.md)] using the COPY statement](/fabric/data-warehouse/ingest-data-copy).
555555

@@ -631,10 +631,11 @@ When a column list isn't specified, COPY maps columns based on the source and ta
631631

632632
#### *External location*
633633

634-
Specifies where the files containing the data is staged. Currently Azure Data Lake Storage (ADLS) Gen2 and Azure Blob Storage are supported:
634+
Specifies where the files containing the data is staged. Currently Azure Data Lake Storage (ADLS) Gen2, Azure Blob Storage, and OneLake (Preview) are supported:
635635

636636
- *External location* for Blob Storage: `https://<account\>.blob.core.windows.net/<container\>/<path\>`
637637
- *External location* for ADLS Gen2: `https://<account\>.dfs.core.windows.net/<container\>/<path\>`
638+
- *External location* for OneLake (Preview): `https://onelake.dfs.fabric.microsoft.com/<workspaceId>/<lakehouseId>/Files/`
638639

639640
Azure Data Lake Storage (ADLS) Gen2 offers better performance than Azure Blob Storage (legacy). Consider using an ADLS Gen2 account whenever possible.
640641

@@ -677,10 +678,13 @@ To access files on Azure Data Lake Storage (ADLS) Gen2 and Azure Blob Storage lo
677678

678679
#### *CREDENTIAL (IDENTITY = '', SECRET = '')*
679680

680-
*CREDENTIAL* specifies the authentication mechanism to access the external storage account. On [!INCLUDE [fabric-dw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], the only supported authentication mechanisms are Shared Access Signature (SAS) and Storage Account Key (SAK). User's EntraID authentication is default, no credential needs to be specified.
681+
*CREDENTIAL* specifies the authentication mechanism to access the external storage account. On [!INCLUDE [fabric-dw](../../includes/fabric-dw.md)] in [!INCLUDE [fabric](../../includes/fabric.md)], the only supported authentication mechanisms are Shared Access Signature (SAS) and Storage Account Key (SAK).
682+
683+
The user's EntraID authentication is default, no credential needs to be specified. COPY INTO using OneLake as source only supports EntraID authentication.
681684

682685
> [!NOTE]
683686
> When using a public storage account, CREDENTIAL does not need to be specified. By default the executing user's Entra ID is used.
687+
684688
- Authenticating with Shared Access Signature (SAS)
685689

686690
- *IDENTITY: A constant with a value of 'Shared Access Signature'*
@@ -803,14 +807,22 @@ Parser version 1.0 is available for backward compatibility only, and should be u
803807
804808
## Use COPY INTO with OneLake
805809

806-
You can now use `COPY INTO` to load data directly from files stored in the Fabric OneLake, specifically from the **Files folder** of a Fabric Lakehouse. This eliminates the need for external staging accounts (such as ADLS Gen2 or Blob Storage) and enables workspace-governed, SaaS-native ingestion using Fabric permissions. This functionality supports:
810+
You can use `COPY INTO` to load data directly from files stored in the Fabric OneLake, specifically from the **Files folder** of a Fabric Lakehouse. This eliminates the need for external staging accounts (such as ADLS Gen2 or Blob Storage) and enables workspace-governed, SaaS-native ingestion using Fabric permissions. This functionality supports:
807811

808812
- Reading from `Files` folders in Lakehouses
809813
- Workspace-to-warehouse loads within the same tenant
810814
- Native identity enforcement using Microsoft Entra ID
811815

812-
> [!NOTE]
813-
> This feature is currently in [preview](/fabric/fundamentals/preview).
816+
Example:
817+
818+
```sql
819+
COPY INTO t1
820+
FROM 'https://onelake.dfs.fabric.microsoft.com/<workspaceId>/<lakehouseId>/Files/*.csv'
821+
WITH (
822+
FILE_TYPE = 'CSV',
823+
FIRSTROW = 2
824+
);
825+
```
814826

815827
## Permissions
816828

@@ -831,18 +843,20 @@ GO
831843
GRANT INSERT to [mike@contoso.com];
832844
GO
833845
```
834-
> [!NOTE]
835-
> When using the *ErrorFile* option, the user must have the minimal permission of Blob Storage Contributor on the Storage Account container.
836846

837-
> [!NOTE]
838-
> When using OneLake as the source (Public Preview), the user must have **Contributor** or higher permissions on both the **source workspace** (where the Lakehouse is located) and the **target workspace** (where the Warehouse resides).
839-
> All access is governed via Microsoft Entra ID and Fabric workspace roles.
847+
When using the *ErrorFile* option, the user must have the minimal permission of Blob Storage Contributor on the Storage Account container.
848+
849+
When using OneLake as the source, the user must have **Contributor** or higher permissions on both the **source workspace** (where the Lakehouse is located) and the **target workspace** (where the Warehouse resides). All access is governed via Microsoft Entra ID and Fabric workspace roles.
840850

841851
## Remarks
842852

843853
The COPY statement accepts only UTF-8 and UTF-16 valid characters for row data and command parameters. Source files or parameters (such as `ROW TERMINATOR` or `FIELD TERMINATOR`) that use invalid characters might be interpreted incorrectly by the COPY statement and cause unexpected results such as data corruption, or other failures. Make sure your source files and parameters are UTF-8 or UTF-16 compliant before you invoke the COPY statement.
844854

845-
## Limitations for OneLake as source (Public Preview)
855+
<a id="limitations-for-onelake-as-source-public-preview"></a>
856+
857+
## Limitations for OneLake as source
858+
859+
Fabric OneLake storage as a source for both `COPY INTO` and `OPENROWSET(BULK)` is a [preview feature](/fabric/fundamentals/preview).
846860

847861
- **Only Microsoft Entra ID authentication is supported.** Other authentication methods, such as SAS tokens, shared keys, or connection strings, are not permitted.
848862

0 commit comments

Comments
 (0)