You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/run-job-in-virtual-network.md
+26-27Lines changed: 26 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,27 +5,27 @@ author: ahartoon
5
5
ms.author: anboisve
6
6
ms.service: azure-stream-analytics
7
7
ms.topic: how-to
8
-
ms.date: 05/23/2023
8
+
ms.date: 10/08/2024
9
9
---
10
10
11
11
# Run your Azure Stream Analytics job in an Azure Virtual Network (Public preview)
12
12
This article describes how to run your Azure Stream Analytics (ASA) job in an Azure virtual network.
13
13
14
14
## Overview
15
-
Virtual network (VNet) support enables you to lock down access to Azure Stream Analytics to your virtual network infrastructure. This capability provides you with the benefits of network isolation and can be accomplished by [deploying a containerized instance of your ASA job inside your Virtual Network](../virtual-network/virtual-network-for-azure-services.md). Your VNet injected ASA job can then privately access your resources within the virtual network via:
15
+
Virtual network support enables you to lock down access to Azure Stream Analytics to your virtual network infrastructure. This capability provides you with the benefits of network isolation and can be accomplished by [deploying a containerized instance of your ASA job inside your Virtual Network](../virtual-network/virtual-network-for-azure-services.md). Your virtual network injected ASA job can then privately access your resources within the virtual network via:
16
16
17
-
-[Private endpoints](../private-link/private-endpoint-overview.md), which connect your VNet injected ASA job to your data sources over private links powered by Azure Private Link.
18
-
-[Service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md), which connect your data sources to your VNet injected ASA job.
17
+
-[Private endpoints](../private-link/private-endpoint-overview.md), which connect your virtual network injected ASA job to your data sources over private links powered by Azure Private Link.
18
+
-[Service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md), which connect your data sources to your virtual network injected ASA job.
19
19
-[Service tags](../virtual-network/service-tags-overview.md), which allow or deny traffic to Azure Stream Analytics.
20
20
21
21
## Availability
22
22
Currently, this capability is only available in select regions: **West US**, **Central Canada**, **East US**, **East US 2**, **Central US**, **West Europe**, and **North Europe**.
23
-
If you're interested in enabling VNet integration in your region, **fill out this [form](https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbRzFwASREnlZFvs9gztPNuTdUMU5INk5VT05ETkRBTTdSMk9BQ0w3OEZDQi4u)**.
23
+
If you're interested in enabling virtual network integration in your region, **fill out this [form](https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbRzFwASREnlZFvs9gztPNuTdUMU5INk5VT05ETkRBTTdSMk9BQ0w3OEZDQi4u)**.
24
24
25
-
## Requirements for VNet integration support
25
+
## Requirements for virtual network integration support
26
26
27
-
- A **General purpose V2 (GPV2) Storage account** is required for VNET injected ASA jobs.
28
-
-VNet injected ASA jobs require access to metadata such as checkpoints to be stored in Azure tables for operational purposes.
27
+
- A **General purpose V2 (GPV2) Storage account** is required for virtual network injected ASA jobs.
28
+
-Virtual network injected ASA jobs require access to metadata such as checkpoints to be stored in Azure tables for operational purposes.
29
29
- If you already have a GPV2 account provisioned with your ASA job, no extra steps are required.
30
30
- Users with higher scale jobs with Premium storage are still required to provide a GPV2 storage account.
31
31
- If you wish to protect storage accounts from public IP based access, consider configuring it using Managed Identity and Trusted Services as well.
@@ -34,48 +34,47 @@ If you're interested in enabling VNet integration in your region, **fill out thi
34
34
- An existing **Azure Virtual Network** or [create one](../virtual-network/quick-create-portal.md).
35
35
36
36
> [!IMPORTANT]
37
-
> ASA VNET injected jobs use an internal container injection technology provided by Azure networking. Azure Networking requires that all customers set up Azure NAT Gateway on subnets for security and reliability.
37
+
> ASA virtual network injected jobs use an internal container injection technology provided by Azure networking. As per networking requirements, Azure NAT Gateway must be configured for virtual network injected ASA jobs for security and reliability purposes.
38
38
>
39
-
> Azure NAT Gateway is a fully managed and highly resilient Network Address Translation (NAT) service. Azure NAT Gateway simplifies outbound Internet connectivity for virtual networks. When configured on a subnet, all outbound connectivity uses the NAT gateway's static public IP addresses.
40
-
39
+
> Azure NAT Gateway is a fully managed and highly resilient Network Address Translation (NAT) service. When configured on a subnet, all outbound connectivity uses the NAT gateway's static public IP addresses.
41
40
:::image type="content" source="./media/run-job-in-virtual-network/vnet-nat.png" alt-text="Diagram showing the architecture of the virtual network.":::
42
41
43
-
To learn about setup and pricing, see [Azure NAT Gateway](../nat-gateway/nat-overview.md).
42
+
For more information about Azure NAT Gateway, see [Azure NAT Gateway](../nat-gateway/nat-overview.md).
44
43
45
44
## Subnet Requirements
46
45
Virtual network integration depends on a dedicated subnet. When you create a subnet, the Azure subnet consumes five IPs from the start.
47
46
48
-
You must take into consideration the IP range associated with your delegated subnet as you think about future needs required to support your ASA workload. Because subnet size can't be changed after assignment, use a subnet that's large enough to accommodate whatever scale your job(s) might reach.
47
+
You must take into consideration the IP range associated with your delegated subnet as you think about future needs required to support your ASA workload. Because subnet size can't be changed after assignment, use a subnet that's large enough to accommodate whatever scale your job might reach.
49
48
50
49
The scale operation affects the real, available supported instances for a given subnet size.
51
50
52
51
### Considerations for estimating IP ranges
53
52
54
-
- Make sure the subnet range doesn't collide with ASA’s subnet range. Avoid IP range 10.0.0.0 to 10.0.255.255 as it's used by ASA.
53
+
- Make sure the subnet range doesn't collide with ASA’s subnet range. Avoid IP range 10.0.0.0 to 10.0.255.255 as it's used by ASA.
55
54
- Reserve:
56
-
- Five IP addresses for Azure Networking
57
-
- One IP address is required to facilitate features such as sample data, test connection and metadata discovery for jobs associated with this subnet.
58
-
- Two IP addresses are required for every six SU or one SU V2 (ASA’s V2 pricing structure is launching July 1, 2023, see [here](https://aka.ms/AzureStreamAnalyticsisLaunchingaNewCompetitivePricingModel) for details)
55
+
-**Five** IP addresses for Azure Networking
56
+
-**One** IP address is required to facilitate features such as sample data, test connection, and metadata discovery for jobs associated with this subnet.
57
+
-**Two** IP addresses are required for every six streaming unit (SU) or one SU V2 (ASA’s V2 pricing structure is launching July 1, 2023, see [here](https://aka.ms/AzureStreamAnalyticsisLaunchingaNewCompetitivePricingModel) for details)
59
58
60
-
When you indicate VNET integration with your Azure Stream Analytics job, Azure portal will automatically delegate the subnet to the ASA service. Azure portal will undelegate the subnet in the following scenarios:
59
+
When you indicate virtual network integration with your Azure Stream Analytics job, Azure portal automatically delegates the subnet to the ASA service. Azure portal undelegates the subnet in the following scenarios:
61
60
62
-
- You inform us that VNET integration is no longer needed for the [last job](#last-job) associated with specified subnet via the ASA portal (see ‘how to’ section).
61
+
- You inform us that virtual network integration is no longer needed for the [last job](#last-job) associated with specified subnet via the ASA portal (see the how-to section).
63
62
- You delete the [last job](#last-job) associated with the specified subnet.
64
63
65
64
### Last job
66
-
Several ASA jobs may utilize the same subnet. The last job here refers to no other jobs utilizing the specified subnet. When the last job has been deleted or removed by associated, Azure Stream Analytics releases the subnet as a resource, which was delegated to ASA as a service. Allow several minutes for this action to be completed.
65
+
Several Stream Analytics jobs can utilize the same subnet. The last job here refers to no other jobs utilizing the specified subnet. When the last job has been deleted or removed by associated, Azure Stream Analytics releases the subnet as a resource, which was delegated to ASA as a service. Allow several minutes for this action to be completed.
67
66
68
-
## Set up VNET integration
67
+
## Set up virtual network integration
69
68
70
69
### Azure portal
71
-
1. From the Azure portal, navigate to **Networking** from menu bar and select **Run this job in virtual network**. This step informs us that your job must work with a VNET:
70
+
1. From the Azure portal, navigate to **Networking** from menu bar and select **Run this job in virtual network**. This step informs us that your job must work with a virtual network:
72
71
1. Configure the settings as prompted and select **Save**.
73
72
74
73
:::image type="content" source="./media/run-job-in-virtual-network/networking-page.png" alt-text="Screenshot of the Networking page for a Stream Analytics job.":::
75
74
76
75
## VS Code
77
76
78
-
1. In Visual Studio Code, reference the subnet within your ASA job. This step tells your job that it must work with a subnet.
77
+
1. In Visual Studio Code, reference the subnet within your ASA job. This step tells your job that it must work with a subnet.
79
78
1. In the `JobConfig.json`, set up your `VirtualNetworkConfiguration` as shown in the following image.
80
79
81
80
:::image type="content" source="./media/run-job-in-virtual-network/virtual-network-configuration.png" alt-text="Screenshot of the sample virtual network configuration." lightbox="./media/run-job-in-virtual-network/virtual-network-configuration.png":::
@@ -102,19 +101,19 @@ You must have at least the following Role-based access control permissions on th
102
101
|`Microsoft.Network/virtualNetworks/read`| Read the virtual network definition |
103
102
|`Microsoft.Network/virtualNetworks/subnets/read`| Read a virtual network subnet definition |
104
103
|`Microsoft.Network/virtualNetworks/subnets/join/action`| Joins a virtual network |
105
-
|`Microsoft.Network/virtualNetworks/subnets/write`| Optional. Required if you need to perform subnet delegation |
104
+
|`Microsoft.Network/virtualNetworks/subnets/write`| Optional. Required if you need to perform subnet delegation |
106
105
107
106
108
107
If the virtual network is in a different subscription than your ASA job, you must ensure that the subscription with the virtual network is registered for the `Microsoft.StreamAnalytics` resource provider. You can explicitly register the provider by following [this documentation](../azure-resource-manager/management/resource-providers-and-types.md#register-resource-provider), but it's automatically registered when creating the job in a subscription.
109
108
110
109
## Limitations
111
110
112
-
-VNET jobs require a minimum of one SU V2 (new pricing model) or six SUs (current)
111
+
-Virtual network jobs require a minimum of one SU V2 (new pricing model) or six SUs (current)
113
112
- Make sure the subnet range doesn't collide with ASA subnet range (that is, don't use subnet range 10.0.0.0/16).
114
-
- ASA job(s) and the virtual network must be in the same region.
113
+
- ASA jobs and the virtual network must be in the same region.
115
114
- The delegated subnet can only be used by Azure Stream Analytics.
116
115
- You can't delete a virtual network when it's integrated with ASA. You must disassociate or remove the last job* on the delegated subnet.
117
-
- We don't support DNS refreshes currently. If DNS configurations of your VNET are changed, you must redeploy all ASA jobs in that VNET (subnets will also need to be disassociated from all jobs and reconfigured). For more information, see [Name resolution for resources in Azure virtual networks](../virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances.md?tabs=redhat) for more information.
116
+
- We don't support Domain Name System (DNS) refreshes currently. If DNS configurations of your virtual network are changed, you must redeploy all ASA jobs in that virtual network (subnets also need to be disassociated from all jobs and reconfigured). For more information, see [Name resolution for resources in Azure virtual networks](../virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances.md?tabs=redhat) for more information.
118
117
119
118
## Access on-premises resources
120
119
No extra configuration is required for the virtual network integration feature to reach through your virtual network to on-premises resources. You simply need to connect your virtual network to on-premises resources by using ExpressRoute or a site-to-site VPN.
0 commit comments