You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/security/synapse-workspace-ip-firewall.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: ashinMSFT
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: overview
7
7
ms.subservice: security
8
-
ms.date: 09/16/2022
8
+
ms.date: 01/22/2025
9
9
ms.author: danzhang
10
10
ms.reviewer: wiassaf
11
11
---
@@ -23,14 +23,14 @@ IP firewall rules grant or deny access to your Azure Synapse workspace based on
23
23
There are two ways IP firewall rules are added to an Azure Synapse workspace. To add an IP firewall to your workspace, select **Networking** and check **Allow connections from all IP addresses** during workspace creation.
24
24
25
25
> [!Important]
26
-
> This feature is only available to Azure Synapse workspaces not associated with a Managed VNet.
26
+
> This feature is only available to Azure Synapse workspaces not associated with a Managed Virtual Network.
27
27
28
28
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-networking-connections-all-ip-addresses.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-networking-connections-all-ip-addresses.png" alt-text="Screenshot that highlights the Security tab, and the 'Allow connections from all IP addresses' checkbox.":::
29
29
30
30
You can also add IP firewall rules to a Synapse workspace after the workspace is created. Select **Firewalls** under **Security** from Azure portal. To add a new IP firewall rule, give it a name, Start IP, and End IP. Select **Save** when done.
31
31
32
32
> [!NOTE]
33
-
> The Public network access feature is only available to Azure Synapse workspaces associated with Azure Synapse Analytics Managed Virtual Network. However, you can still open your Azure Synapse workspaces to the public network regardless of its association with managed VNet. For more information, see [Public network access](connectivity-settings.md#public-network-access).
33
+
> The Public network access feature is only available to Azure Synapse workspaces associated with Azure Synapse Analytics Managed Virtual Network. However, you can still open your Azure Synapse workspaces to the public network regardless of its association with managed virtual network. For more information, see [Public network access](connectivity-settings.md#public-network-access).
34
34
35
35
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-networking-firewalls-add-client-ip.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-networking-firewalls-add-client-ip.png" alt-text="Screenshot of the Networking page of a Synapse Workspace, highlighting the Add client IP button and rules fields.":::
36
36
@@ -55,7 +55,7 @@ For more information on the methods to manage the firewall programmatically, see
Copy file name to clipboardExpand all lines: articles/synapse-analytics/security/synapse-workspace-managed-vnet.md
+13-17Lines changed: 13 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: ashinMSFT
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: overview
7
7
ms.subservice: security
8
-
ms.date: 08/16/2021
8
+
ms.date: 01/22/2025
9
9
ms.author: seshin
10
10
ms.reviewer: wiassaf
11
11
---
@@ -18,25 +18,25 @@ This article will explain Managed Virtual Network in Azure Synapse Analytics.
18
18
19
19
When you create your Azure Synapse workspace, you can choose to associate it to a Microsoft Azure Virtual Network. The Virtual Network associated with your workspace is managed by Azure Synapse. This Virtual Network is called a *Managed workspace Virtual Network*.
20
20
21
-
Managed workspace Virtual Network provides you value in four ways:
21
+
Managed workspace Virtual Network provides you with value in four ways:
22
22
23
23
- With a Managed workspace Virtual Network you can offload the burden of managing the Virtual Network to Azure Synapse.
24
24
- You don't have to configure inbound NSG rules on your own Virtual Networks to allow Azure Synapse management traffic to enter your Virtual Network. Misconfiguration of these NSG rules causes service disruption for customers.
25
25
- You don't need to create a subnet for your Spark clusters based on peak load.
26
26
- Managed workspace Virtual Network along with Managed private endpoints protects against data exfiltration. You can only create Managed private endpoints in a workspace that has a Managed workspace Virtual Network associated with it.
27
27
28
-
Creating a workspace with a Managed workspace Virtual Network associated with it ensures that your workspace is network isolated from other workspaces. Azure Synapse provides various analytic capabilities in a workspace: Data integration,serverless Apache Spark pool, dedicated SQL pool, and serverless SQL pool.
28
+
Creating a workspace with a Managed workspace Virtual Network associated with it ensures that your workspace is network isolated from other workspaces. Azure Synapse provides various analytic capabilities in a workspace: Data integration,serverless Apache Spark pool, dedicated SQL pool, and serverless SQL pool.
29
29
30
30
If your workspace has a Managed workspace Virtual Network, Data integration and Spark resources are deployed in it. A Managed workspace Virtual Network also provides user-level isolation for Spark activities because each Spark cluster is in its own subnet.
31
31
32
-
Dedicated SQL pool and serverless SQL pool are multi-tenant capabilities and therefore reside outside of the Managed workspace Virtual Network. Intra-workspace communication to dedicated SQL pool and serverless SQL pool use Azure private links. These private links are automatically created for you when you create a workspace with a Managed workspace Virtual Network associated to it.
32
+
Dedicated SQL pool and serverless SQL pool are multitenant capabilities and therefore reside outside of the Managed workspace Virtual Network. Intra-workspace communication to dedicated SQL pool and serverless SQL pool use Azure private links. These private links are automatically created for you when you create a workspace with a Managed workspace Virtual Network associated to it.
33
33
34
34
>[!IMPORTANT]
35
-
>You cannot change this workspace configuration after the workspace is created. For example, you cannot reconfigure a workspace that does not have a Managed workspace Virtual Network associated with it and associate a Virtual Network to it. Similarly, you cannot reconfigure a workspace with a Managed workspace Virtual Network associated to it and disassociate the Virtual Network from it.
35
+
>You can't change this workspace configuration after the workspace is created. For example, you can't reconfigure a workspace that doesn't have a Managed workspace Virtual Network associated with it and associate a Virtual Network to it. Similarly, you can't reconfigure a workspace with a Managed workspace Virtual Network associated to it and disassociate the Virtual Network from it.
36
36
37
37
## Create an Azure Synapse workspace with a Managed workspace Virtual Network
38
38
39
-
If you have not already done so, register the Network resource provider. Registering a resource provider configures your subscription to work with the resource provider. Choose *Microsoft.Network* from the list of resource providers when you [register](../../azure-resource-manager/management/resource-providers-and-types.md).
39
+
If you haven't already done so, register the Network resource provider. Registering a resource provider configures your subscription to work with the resource provider. Choose *Microsoft.Network* from the list of resource providers when you [register](../../azure-resource-manager/management/resource-providers-and-types.md#azure-portal).
40
40
41
41
To create an Azure Synapse workspace that has a Managed workspace Virtual Network associated with it, select the **Networking** tab in Azure portal and check the **Enable managed virtual network** checkbox.
42
42
@@ -47,26 +47,22 @@ If you leave the checkbox unchecked, then your workspace won't have a Virtual Ne
47
47
48
48
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-analytics-networking-managed-virtual-network-outbound-traffic.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-analytics-networking-managed-virtual-network-outbound-traffic.png" alt-text="Screenshot of the Create Synapse workspace networking page, with the Managed virtual network option Enabled and the Allow outbound data traffic only to approved targets option to Yes.":::
49
49
50
-
After you choose to associate a Managed workspace Virtual Network with your workspace, you can protect against data exfiltration by allowing outbound connectivity from the Managed workspace Virtual Network only to approved targets using [Managed private endpoints](./synapse-workspace-managed-private-endpoints.md). Select **Yes** to limit outbound traffic from the Managed workspace Virtual Network to targets through Managed private endpoints.
51
-
52
-
50
+
After you choose to associate a Managed workspace Virtual Network with your workspace, you can protect against data exfiltration by allowing outbound connectivity from the Managed workspace Virtual Network only to approved targets using [Managed private endpoints](./synapse-workspace-managed-private-endpoints.md). Select **Yes** to limit outbound traffic from the Managed workspace Virtual Network to targets through Managed private endpoints.
53
51
54
52
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-allow-outbound-traffic.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-allow-outbound-traffic.png" alt-text="Screenshot of the Managed virtual network page, with the Allow outbound data traffic only to approved targets option to Yes.":::
55
53
56
54
Select **No** to allow outbound traffic from the workspace to any target.
57
55
58
-
You can also control the targets to which Managed private endpoints are created from your Azure Synapse workspace. By default, Managed private endpoints to resources in the same AAD tenant that your subscription belongs to are allowed. If you want to create a Managed private endpoint to a resource in an AAD tenant that is different from the one that your subscription belongs to, then you can add that AAD tenant by selecting **+ Add**. You can either select the AAD tenant from the dropdown or manually enter the AAD tenant ID.
56
+
You can also control the targets to which Managed private endpoints are created from your Azure Synapse workspace. By default, Managed private endpoints to resources in the same Microsoft Entra ID tenant that your subscription belongs to are allowed. If you want to create a Managed private endpoint to a resource in a Microsoft Entra ID tenant that is different from the one that your subscription belongs to, then you can add that Microsoft Entra ID tenant by selecting **+ Add**. You can either select the Microsoft Entra ID tenant from the dropdown or manually enter the Microsoft Entra ID tenant ID.
59
57
60
-
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-private-endpoints-azure-ad.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-private-endpoints-azure-ad.png" alt-text="Screenshot of the Managed virtual network page, with the Add button for Azure Tenant Tenants highlighted.":::
58
+
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-private-endpoints-azure-ad.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-workspace-managed-virtual-network-private-endpoints-azure-ad.png" alt-text="Screenshot of the Managed virtual network page, with the Add button for Azure Tenants highlighted.":::
61
59
62
60
After the workspace is created, you can check whether your Azure Synapse workspace is associated to a Managed workspace Virtual Network by selecting **Overview** from Azure portal.
63
61
64
62
:::image type="content" source="./media/synpase-workspace-ip-firewall/azure-synapse-analytics-overview-managed-virtual-network-enabled.png" lightbox="./media/synpase-workspace-ip-firewall/azure-synapse-analytics-overview-managed-virtual-network-enabled.png" alt-text="Screenshot of the Azure Synapse workspace overview page indicating that a managed virtual network is enabled.":::
65
63
66
-
## Next steps
67
-
68
-
Create an [Azure Synapse Workspace](../quickstart-create-workspace.md)
69
-
70
-
Learn more about [Managed private endpoints](./synapse-workspace-managed-private-endpoints.md)
64
+
## Related content
71
65
72
-
[Create Managed private endpoints to your data sources](./how-to-create-managed-private-endpoints.md)
66
+
- Create an [Azure Synapse Workspace](../quickstart-create-workspace.md)
67
+
- Learn more about [Managed private endpoints](./synapse-workspace-managed-private-endpoints.md)
68
+
-[Create Managed private endpoints to your data sources](./how-to-create-managed-private-endpoints.md)
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/synapse-spark-sql-pool-import-export.md
+7-19Lines changed: 7 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: kalyankadiyala-Microsoft
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: overview
7
7
ms.subservice: spark
8
-
ms.date: 05/10/2022
8
+
ms.date: 01/22/2025
9
9
ms.author: kakadiya
10
10
ms.reviewer: ktuckerdavis, aniket.adnaik
11
11
---
@@ -46,9 +46,9 @@ At a high-level, the connector provides the following capabilities:
46
46
47
47

48
48
49
-
## Pre-requisites
49
+
## Prerequisites
50
50
51
-
Pre-requisites such as setting up required Azure resources and steps to configure them are discussed in this section.
51
+
Prerequisites such as setting up required Azure resources and steps to configure them are discussed in this section.
52
52
53
53
### Azure resources
54
54
@@ -94,7 +94,7 @@ A basic authentication approach requires user to configure `username` and `passw
94
94
There are two ways to grant access permissions to Azure Data Lake Storage Gen2 - Storage Account:
95
95
96
96
* Role based Access Control role - [Storage Blob Data Contributor role](../../role-based-access-control/built-in-roles.md#storage-blob-data-contributor)
97
-
* Assigning the `Storage Blob Data Contributor Role` grants the User permissions to read, write and delete from the Azure Storage Blob Containers.
97
+
* Assigning the `Storage Blob Data Contributor Role` grants the User permissions to read, write, and delete from the Azure Storage Blob Containers.
98
98
* RBAC offers a coarse control approach at the container level.
99
99
* [Access Control Lists (ACL)](../../storage/blobs/data-lake-storage-access-control.md)
100
100
* ACL approach allows for fine-grained controls over specific paths and/or files under a given folder.
@@ -152,7 +152,7 @@ To successfully bootstrap and orchestrate the read or write operation, the Conne
152
152
Following is the list of configuration options based on usage scenario:
153
153
154
154
* **Read using Microsoft Entra ID based authentication**
155
-
* Credentials are auto-mapped, and user isn't required to provide specific configuration options.
155
+
* Credentials are automapped, and user isn't required to provide specific configuration options.
156
156
* Three-part table name argument on `synapsesql` method is required to read from respective table in Azure Synapse Dedicated SQL Pool.
157
157
* **Read using basic authentication**
158
158
* Azure Synapse Dedicated SQL End Point
@@ -281,7 +281,7 @@ dfToReadFromTable.show()
281
281
> * Table name and query cannot be specified at the same time.
282
282
> * Only select queries are allowed. DDL and DML SQLs are not allowed.
283
283
> * The select and filter options on dataframe are not pushed down to the SQL dedicated pool when a query is specified.
284
-
> * Read from a query is only available in Spark 3.1 and 3.2.
284
+
> * Read from a query is only available in Spark 3.
The method signature for the Connector version built for [Spark 2.4.8](./apache-spark-24-runtime.md) has one less argument, than that applied to the Spark 3.1.2 version. Following are the two method signatures:
573
-
574
-
* Spark Pool Version 2.4.8
575
-
576
-
```Scala
577
-
synapsesql(tableName:String,
578
-
tableType:String=Constants.INTERNAL,
579
-
location:Option[String] =None):Unit
580
-
```
581
-
582
-
* Spark Pool Version 3.1.2
583
-
584
572
##### [Scala](#tab/scala3)
585
573
586
574
```Scala
@@ -974,7 +962,7 @@ By default, a write response is printed to the cell output. On failure, the curr
974
962
* When writing large data sets, it's important to factor in the impact of [DWU Performance Level](../../synapse-analytics/sql-data-warehouse/quickstart-scale-compute-portal.md) setting that limits [transaction size](../../synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-transactions.md#transaction-size).
975
963
* Monitor [Azure Data Lake Storage Gen2](../../storage/blobs/data-lake-storage-best-practices.md) utilization trends to spot throttling behaviors that can [impact](../../storage/common/scalability-targets-standard-account.md) read and write performance.
0 commit comments