Skip to content

Commit ca10f90

Browse files
authored
Merge pull request #298330 from dlepow/snat
[APIM] Enable NAT gateway
2 parents a463d24 + 616e1fd commit ca10f90

File tree

2 files changed

+40
-25
lines changed

2 files changed

+40
-25
lines changed

articles/api-management/TOC.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -449,7 +449,7 @@
449449
href: diagnose-solve-problems.md
450450
- name: Troubleshoot failed to update hostnames error
451451
href: api-management-troubleshoot-cannot-add-custom-domain.md
452-
- name: Troubleshoot response timeouts and errors
452+
- name: Troubleshoot response time-outs and errors
453453
href: troubleshoot-response-timeout-and-errors.md
454454
- name: Samples
455455
items:
Lines changed: 39 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,99 @@
11
---
2-
title: Troubleshooting client response timeouts and errors with API Management
2+
title: Troubleshooting client response time-outs and errors with API Management
33
description: Troubleshoot intermittent connection errors and related latency issues in API Management
44
author: dlepow
55
ms.topic: troubleshooting
6-
ms.date: 12/04/2020
6+
ms.date: 04/15/2025
77
ms.author: danlep
88
ms.service: azure-api-management
99
---
1010

11-
# Troubleshooting client response timeouts and errors with API Management
11+
# Troubleshooting client response time-outs and errors with API Management
1212

1313
[!INCLUDE [api-management-availability-all-tiers](../../includes/api-management-availability-all-tiers.md)]
1414

15-
This article helps you troubleshoot intermittent connection errors and related latency issues in [Azure API Management](./api-management-key-concepts.md). Specifically, this article will provide information and troubleshooting for the exhaustion of source address network translation (SNAT) ports. If you require more help, contact the Azure experts at [Azure Community Support](https://azure.microsoft.com/support/community/) or file a support request with [Azure Support](https://azure.microsoft.com/support/options/).
15+
This article helps you troubleshoot intermittent connection errors and related latency issues in [Azure API Management](./api-management-key-concepts.md). Specifically, this article provides information and troubleshooting for the exhaustion of source network address translation (SNAT) ports. If you require more help, contact the Azure experts at [Azure Community Support](https://azure.microsoft.com/support/community/) or file a support request with [Azure Support](https://azure.microsoft.com/support/options/).
1616

1717
## Symptoms
1818

19-
Client applications calling APIs through your API Management (APIM) service may exhibit one or more of the following symptoms:
19+
Client applications calling APIs through your API Management service may exhibit one or more of the following symptoms:
2020

2121
* Intermittent HTTP 500 errors
22-
* Timeout error messages
22+
* Time-out error messages
2323

2424
These symptoms manifest as instances of `BackendConnectionFailure` in your [Azure Monitor resource logs](/azure/azure-monitor/essentials/resource-logs).
2525

26+
In certain API Management service tiers, you may also see diagnostic information related to SNAT port exhaustion in the Azure portal on the **Diagnose and solve problems** > **SNAT Port Analysis** page for your API Management instance.
27+
2628
## Cause
2729

28-
This pattern of symptoms often occurs due to network address translation (SNAT) port limits with your APIM service.
30+
This pattern of symptoms often occurs due to SNAT port limits with your API Management service.
2931

30-
Whenever a client calls one of your APIM APIs, Azure API Management service opens a SNAT port to access your backend API. As discussed in [Outbound connections in Azure](../load-balancer/load-balancer-outbound-connections.md), Azure uses source network address translation (SNAT) and a Load Balancer (not exposed to customers) to communicate with end points outside Azure in the public IP address space, as well as end points internal to Azure that aren't using [Virtual Network service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md). This situation is only applicable to backend APIs exposed on public IPs.
32+
Whenever a client calls one of your API Management APIs, Azure API Management service opens a SNAT port to access your backend API. As discussed in [Outbound connections in Azure](../load-balancer/load-balancer-outbound-connections.md), Azure uses source network address translation (SNAT) and a load balancer (not exposed to customers) to communicate with endpoints outside Azure in the public IP address space, and to endpoints internal to Azure that aren't using [Virtual Network service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md). This situation is only applicable to backend APIs exposed on public IPs.
3133

32-
Each instance of API Management service is initially given a pre-allocated number of SNAT ports. That limit affects opening connections to the same host and port combination. SNAT ports are used up when you have repeated calls to the same address and port combination. Once a SNAT port has been released, the port is available for reuse as needed. The Azure Network load balancer reclaims SNAT ports from closed connections only after waiting four minutes.
34+
Each instance of the API Management service is initially given a preallocated number of SNAT ports. That limit affects opening connections to the same host and port combination. SNAT ports are used up when you have repeated calls to the same address and port combination. Once a SNAT port has been released, the port is available for reuse as needed. The Azure network load balancer reclaims SNAT ports from closed connections only after waiting four minutes.
3335

34-
A rapid succession of client requests to your APIs may exhaust the pre-allocated quota of SNAT ports if these ports are not closed and recycled fast enough, preventing your APIM service from processing client requests in a timely manner.
36+
A rapid succession of client requests to your APIs may exhaust the preallocated quota of SNAT ports if these ports aren't closed and recycled fast enough, preventing your API Management service from processing client requests in a timely manner.
3537

3638
## Mitigations and solutions
3739

38-
Addressing the problem of SNAT port exhaustion first requires diagnosing and optimizing the performance of your backend services.
40+
General strategies for mitigating SNAT port exhaustion are discussed in [Troubleshooting outbound connections failures](../load-balancer/troubleshoot-outbound-connection.md) in the Azure Load Balancer documentation. Of these strategies, the following are applicable to API Management.
41+
42+
### Enable Azure NAT Gateway
43+
44+
For a virtual network-injected instance in the Premium tier of API Management, you can enable [Azure NAT Gateway](/azure/virtual-network/nat-gateway/nat-overview) to provide a larger number of SNAT ports (up to 64K) than are available by default in API Management. If supported in your scenario, this solution is the most effective way to avoid SNAT port exhaustion.
45+
46+
To enable Azure NAT Gateway in the API Management instance's virtual network, set the instance's `natGatewayState` property to `enabled` by using the [API Management Service - Create Or Update](/rest/api/apimanagement/api-management-service/create-or-update#apimanagementcreateservicewithnatgatewayenabled) REST API.
47+
48+
> [!NOTE]
49+
> * Currently, to set the `natGatewayState` property, the instance can't be in a zonal or zone-redundant configuration.
50+
> * For an instance injected into a virtual network in internal mode, the NAT gateway works only for outbound traffic to the internet.
51+
> * Azure NAT Gateway may incur extra costs.
52+
53+
The default idle time-out set in the NAT gateway is 4 minutes. You can change the idle time-out to a maximum of 120 minutes. For more information, see [Manage NAT Gateway](/azure/nat-gateway/manage-nat-gateway?tabs=manage-nat-portal).
3954

40-
General strategies for mitigating SNAT port exhaustion are discussed in [Troubleshooting outbound connections failures](../load-balancer/troubleshoot-outbound-connection.md) from *Azure Load Balancer* documentation. Of these strategies, the following are applicable to API Management.
55+
If you're unable to use a NAT gateway for outbound connectivity, refer to the other mitigation options described in this section.
4156

42-
### Scale your APIM instance
57+
### Scale your API Management instance
4358

44-
Each API Management instance is allocated a number of SNAT ports, based on APIM units. You can allocate additional SNAT ports by scaling your API Management instance with additional units. For more info, see [Scale your API Management service](upgrade-and-scale.md#scale-your-api-management-instance).
59+
Each API Management instance is allocated a number of SNAT ports, based on API Management units. You can allocate more SNAT ports by scaling your API Management instance with more units. For more information, see [Scale your API Management service](upgrade-and-scale.md#scale-your-api-management-instance).
4560

4661
> [!NOTE]
4762
> SNAT port usage is currently not available as a metric for autoscaling API Management units.
4863
4964
### Use multiple IPs for your backend URLs
5065

51-
Each connection from your APIM instance to the same destination IP and destination port of your backend service will use a SNAT port, in order to maintain a distinct traffic flow. Without different SNAT ports for the return traffic from your background service, APIM would have no way to separate one response from another.
66+
Each connection from your API Management instance to the same destination IP and destination port of your backend service uses a SNAT port, in order to maintain a distinct traffic flow. Without different SNAT ports for the return traffic from your background service, API Management has no way to separate one response from another.
5267

5368
Because SNAT ports can be reused if the destination IP or destination port are different, another way to avoid SNAT port exhaustion is by using multiple IPs for your backend service URLs.
5469

5570
For more, see [Outbound proxy Azure Load Balancer](../load-balancer/load-balancer-outbound-connections.md).
5671

57-
### Place your APIM and backend service in the same VNet
72+
### Place your API Management and backend service in the same VNet
5873

59-
If your backend API is hosted on an Azure service that supports *service endpoints* such as App Service, you can avoid SNAT port exhaustion issues by placing your APIM instance and backend service in the same virtual network and exposing it through [service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md) or [private endpoints](../private-link/private-endpoint-overview.md). When you use a common VNet and place service endpoints on the integration subnet, outbound traffic from your APIM instance to those services bypasses the internet, thus avoiding SNAT port restrictions. Likewise, if you use a VNet and private endpoints, you will not have any outbound SNAT port issues to that destination.
74+
If your backend API is hosted on an Azure service that supports *service endpoints* such as App Service, you can avoid SNAT port exhaustion issues by placing your API Management instance and backend service in the same virtual network and exposing it through [service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md) or [private endpoints](../private-link/private-endpoint-overview.md). When you use a common VNet and place service endpoints on the integration subnet, outbound traffic from your API Management instance to those services bypasses the internet, thus avoiding SNAT port restrictions. Likewise, if you use a VNet and private endpoints, you will not have any outbound SNAT port issues to that destination.
6075

6176
For details, see [How to use Azure API Management with virtual networks](api-management-using-with-vnet.md) and [Integrate App Service with an Azure virtual network](../app-service/overview-vnet-integration.md).
6277

63-
### Place your APIM in a virtual network and route outbound calls to Azure Firewall
78+
### Place your API Management service in a virtual network and route outbound calls to Azure Firewall
6479

65-
Similar to placing your APIM and backend services in a virtual network, you can employ Azure Firewall in a VNet with your APIM service, then route outbound APIM calls to Azure Firewall. Between APIM and Azure Firewall (being in the same VNet), no SNAT ports are required. For SNAT connections to your backend services, Azure Firewall has 64,000 available ports, a much higher amount than is allocated to APIM instances.
80+
Similar to placing your API Management and backend services in a virtual network, you can employ Azure Firewall in a VNet with your API Management service, then route outbound API Management calls to Azure Firewall. Between API Management and Azure Firewall (when placed in the same VNet), no SNAT ports are required. For SNAT connections to your backend services, Azure Firewall has 64,000 available ports, a much higher amount than is allocated to API Management instances.
6681

67-
Refer to [Azure Firewall](../firewall/overview.md) documentation for more.
82+
Refer to [Azure Firewall](../firewall/overview.md) documentation for more information.
6883

6984
### Consider response caching and other backend performance tuning
7085

71-
Another potential mitigation to consider is improving processing times for your backend APIs. One way to do this is by configuring certain APIs with response caching to reduce latency between client applications calling your API and your APIM backend load.
86+
Another potential mitigation is to improve processing times for your backend APIs. One way to do this is by configuring certain APIs with response caching to reduce latency between client applications calling your API and your API Management backend load.
7287

7388
For more, see [Add caching to improve performance in Azure API Management](api-management-howto-cache.md).
7489

7590
### Consider implementing access restriction policies
7691

77-
If it makes sense for your business scenario, you can implement access restriction policies for your API Management product. For example, the `rate-limit-by-key` policy can be used to prevent API usage spikes on a per key basis by limiting the call rate per a specified time period.
92+
If it makes sense for your business scenario, you can implement access restriction policies for your API Management product. For example, the [rate-limit-by-key](rate-limit-by-key-policy.md) policy can be used to prevent API usage spikes on a per key basis by limiting the call rate per a specified time period.
7893

79-
See [Rate limiting and quota policies](api-management-policies.md#rate-limiting-and-quotas) for more info.
94+
See [Rate limiting and quota policies](api-management-policies.md#rate-limiting-and-quotas) for more information.
8095

81-
## See also
96+
## Related content
8297

8398
* [Azure Load Balancer: Troubleshooting outbound connections failures](../load-balancer/troubleshoot-outbound-connection.md)
8499
* [Azure App Service: Troubleshooting intermittent outbound connection errors](../app-service/troubleshoot-intermittent-outbound-connection-errors.md)

0 commit comments

Comments
 (0)