You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/synapse-spark-sql-pool-import-export.md
+7-19Lines changed: 7 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: kalyankadiyala-Microsoft
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: overview
7
7
ms.subservice: spark
8
-
ms.date: 05/10/2022
8
+
ms.date: 01/22/2025
9
9
ms.author: kakadiya
10
10
ms.reviewer: ktuckerdavis, aniket.adnaik
11
11
---
@@ -46,9 +46,9 @@ At a high-level, the connector provides the following capabilities:
46
46
47
47

48
48
49
-
## Pre-requisites
49
+
## Prerequisites
50
50
51
-
Pre-requisites such as setting up required Azure resources and steps to configure them are discussed in this section.
51
+
Prerequisites such as setting up required Azure resources and steps to configure them are discussed in this section.
52
52
53
53
### Azure resources
54
54
@@ -94,7 +94,7 @@ A basic authentication approach requires user to configure `username` and `passw
94
94
There are two ways to grant access permissions to Azure Data Lake Storage Gen2 - Storage Account:
95
95
96
96
* Role based Access Control role - [Storage Blob Data Contributor role](../../role-based-access-control/built-in-roles.md#storage-blob-data-contributor)
97
-
* Assigning the `Storage Blob Data Contributor Role` grants the User permissions to read, write and delete from the Azure Storage Blob Containers.
97
+
* Assigning the `Storage Blob Data Contributor Role` grants the User permissions to read, write, and delete from the Azure Storage Blob Containers.
98
98
* RBAC offers a coarse control approach at the container level.
99
99
* [Access Control Lists (ACL)](../../storage/blobs/data-lake-storage-access-control.md)
100
100
* ACL approach allows for fine-grained controls over specific paths and/or files under a given folder.
@@ -152,7 +152,7 @@ To successfully bootstrap and orchestrate the read or write operation, the Conne
152
152
Following is the list of configuration options based on usage scenario:
153
153
154
154
* **Read using Microsoft Entra ID based authentication**
155
-
* Credentials are auto-mapped, and user isn't required to provide specific configuration options.
155
+
* Credentials are automapped, and user isn't required to provide specific configuration options.
156
156
* Three-part table name argument on `synapsesql` method is required to read from respective table in Azure Synapse Dedicated SQL Pool.
157
157
* **Read using basic authentication**
158
158
* Azure Synapse Dedicated SQL End Point
@@ -281,7 +281,7 @@ dfToReadFromTable.show()
281
281
> * Table name and query cannot be specified at the same time.
282
282
> * Only select queries are allowed. DDL and DML SQLs are not allowed.
283
283
> * The select and filter options on dataframe are not pushed down to the SQL dedicated pool when a query is specified.
284
-
> * Read from a query is only available in Spark 3.1 and 3.2.
284
+
> * Read from a query is only available in Spark 3.
The method signature for the Connector version built for [Spark 2.4.8](./apache-spark-24-runtime.md) has one less argument, than that applied to the Spark 3.1.2 version. Following are the two method signatures:
573
-
574
-
* Spark Pool Version 2.4.8
575
-
576
-
```Scala
577
-
synapsesql(tableName:String,
578
-
tableType:String=Constants.INTERNAL,
579
-
location:Option[String] =None):Unit
580
-
```
581
-
582
-
* Spark Pool Version 3.1.2
583
-
584
572
##### [Scala](#tab/scala3)
585
573
586
574
```Scala
@@ -974,7 +962,7 @@ By default, a write response is printed to the cell output. On failure, the curr
974
962
* When writing large data sets, it's important to factor in the impact of [DWU Performance Level](../../synapse-analytics/sql-data-warehouse/quickstart-scale-compute-portal.md) setting that limits [transaction size](../../synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-transactions.md#transaction-size).
975
963
* Monitor [Azure Data Lake Storage Gen2](../../storage/blobs/data-lake-storage-best-practices.md) utilization trends to spot throttling behaviors that can [impact](../../storage/common/scalability-targets-standard-account.md) read and write performance.
0 commit comments