Updates for self-hosted gateway GA

Author: vladvino

articles/api-management/api-management-features.md

@@ -43,4 +43,4 @@ Each API Management [pricing tier](https://aka.ms/apimpricing) offers a distinct
 
 <sup>1</sup> Enables the use of Azure AD (and Azure AD B2C) as an identity provider for user sign in on the developer portal.<br/>
 <sup>2</sup> Including related functionality e.g. users, groups, issues, applications and email templates and notifications.<br/>
-<sup>3</sup> Limited to a single self-hosted gateway deployment with a single gateway node.<br/>
+<sup>3</sup> In the Developer tier self-hosted gateways are limited to single gateway node.<br/>

articles/api-management/api-management-howto-deploy-self-hosted-gateway-to-k8s.md

@@ -1,68 +1,113 @@
 ---
-title: Deploy a self-hosted Azure API Management gateway to Kubernetes | Microsoft Docs
-description: Learn how to deploy a self-hosted Azure API Management gateway to Kubernetes
+title: Deploy Azure API Management self-hosted gateway to Kubernetes | Microsoft Docs
+description: Learn how to deploy self-hosted gateway to Kubernetes
 services: api-management
-documentationcenter: ''
-author: miaojiang
+author: vladvino
 manager: gwallace
-editor: ''
-
 ms.service: api-management
 ms.workload: mobile
-ms.tgt_pltfrm: na
 ms.topic: article
-ms.date: 03/31/2020
 ms.author: apimpm
-
+ms.date: 04/23/2020
 ---
+# Deploy Azure API Management self-hosted gateway to Kubernetes
 
-# Deploy a self-hosted Azure API Management gateway to Kubernetes
-
-This article provides the steps for deploying self-hosted Azure API Management gateway into a Kubernetes Cluster.
+This article describes the steps for deploying self-hosted gateway to a Kubernetes cluster.
 
 ## Prerequisites
 
 - Complete the following quickstart: [Create an Azure API Management instance](get-started-create-service-instance.md)
-- Create a Kubernetes cluster. [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) is a good option for development and evaluation purposes. For production workloads, you can use [Azure Kubernetes Service](https://azure.microsoft.com/services/kubernetes-service/) or a Kubernetes cluster in a foreign cloud.
-- [Provision a gateway resource in your API Management instance](api-management-howto-provision-self-hosted-gateway.md).
+- Create a Kubernetes cluster.
+> [!TIP]
+> [Single-node clusters](https://kubernetes.io/docs/setup/#learning-environment) work well for development and evaluation purposes. Use [Kubernetes Certified](https://kubernetes.io/partners/#conformance) multi-node clusters on-premises or in the cloud for production workloads.
+- [Provision a self-hosted gateway resource in your API Management instance](api-management-howto-provision-self-hosted-gateway.md).
 
-## Deploy the gateway to Kubernetes
+## Deploy the self-hosted gateway to Kubernetes
 
-1. Select **Gateways** from under **Settings**.
-2. Select the gateway resource you intend to deploy.
+1. Select **Gateways** from under **Deployment and infrastructure**.
+2. Select the self-hosted gateway resource you intend to deploy.
 3. Select **Deployment**.
-4. Note that a new token in the **Token** text box was autogenerated for you using the default **Expiry** and **Secret Key** values. Adjust either or both if desired and select **Generate** to create a new token.
-5. Make sure **Kubernetes** is selected under **Deployment scripts**.
-6. Select **<gateway-name>.yml** file link next to **Deployment** to download the file.
-7. Adjust the port mappings and container name in the yml file as needed.
-8. Select the **copy** icon located at the right end of the **Deploy** text box to save the `kubectl` command to clipboard. 
-9. Paste the command to the terminal (or command) window. Note that the command expects the downloaded environment file to be present in the current directory.
-```console
-    kubectl apply -f <gateway-name>.yaml
-```
-10. Execute the command. The command instructs your Kubernetes cluster to run the container, using self-hosted gateway's image downloaded from the Microsoft Container Registry, and to configure the container to expose HTTP (8080) and HTTPS (443) ports.
-11. Run the below command to check the gateway pod is running. Note that your pod name will be different. 
+4. Note that an access token in the **Token** text box was autogenerated for you using the default **Expiry** and **Secret key** values. If needed, pick desired values in either or both controls to generate a new token.
+5. Select **Kubernetes** tab under **Deployment scripts**.
+6. Click on the **<gateway-name>.yml** file link and download the YAML file.
+7. Select **copy** icon located at the bottom right corner of the **Deploy** text box to save the `kubectl` commands to the clipboard.
+8. Paste commands to the terminal (or command) window. The first command creates a Kubernetes secret containing access token generated in step 4. Second command applies the configuration file downloaded in step 6 to the Kubernetes cluster and expects the the file to be present in the current directory.
+9. Execute the commands to create the necessary Kubernetes objects in the [default namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) and start self-hosted gateway pod(s) from [container image](https://aka.ms/apim/sputnik/repo) downloaded from the Microsoft Container Registry.
+10. Execute command shown bellow to check if the deployment succeeded . Note that it might take a little time for all the objects to be created and for pod(s) to initialize.
 ```console
-kubectl get pods
-NAME                              READY   STATUS    RESTARTS   AGE
-local-gateway-55f774f844-bv9wt    1/1     Running   0          1m
+kubectl get deployments
+NAME             READY   UP-TO-DATE   AVAILABLE   AGE
+<gateway-name>   1/1     1            1           18s
 ```
-12. Run the below command to check the gateway service is running. Note that your service name will be different. 
+11. Execute command shown bellow to check if the service was successfully created. Note that your service IPs and ports will be different.
 ```console
 kubectl get services
-NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
-localgateway     NodePort    10.110.230.87   <none>        80:32504/TCP,443:30043/TCP   1m
+NAME             TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
+<gateway-name>   LoadBalancer   10.99.236.168   <pending>     80:31620/TCP,443:30456/TCP   9m1s
 ```
-13. Go back to the Azure portal and confirm that gateway node you just deployed is reporting healthy status.
+12. Go back to the Azure portal and select **Overview**.
+13. **Status** showing a green checkmark icon followed by the node count matching the number of replicas specified in the YAML file confirms that deployed self-hosted gateway pods are successfully communicating with the API Management service and have a regular "heartbeat".
 
 ![gateway status](media/api-management-howto-deploy-self-hosted-gateway-to-k8s/status.png)
 
 > [!TIP]
-> Use <code>kubectl logs <gateway-pod-name></code> command to view a snapshot of self-hosted gateway log.
+> Execute <code>kubectl logs deployment/<gateway-name></code> command to view logs from a randomly selected pod if there are more than one.
+> Execute <code>kubectl logs -h</code> for a complete set of command options, e.g. how to view logs for a specific pod or container.
 
-## Next steps
+## Production deployment considerations
+
+### Access token
+Without a valid access token self-hosted gateway is unable to access and download configuration from configuration data endpoint of the associated API Management service. Access token can be valid for a maximum of 30 days. It must be regenerated and the cluster configured with a fresh token either manually or via automation before it expires. When automating token refresh use this management API [operation](https://docs.microsoft.com/rest/api/apimanagement/2019-12-01/gateway/generatetoken) to generated a new token. Follow this [link](https://kubernetes.io/docs/concepts/configuration/secret) for information on managing Kubernetes secrets.
+
+### Namespace
+Kubernetes [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) help with dividing a single cluster among multiple teams, projects, or applications. Namespaces provide a scope for resources and names and can be associated with a resource quota and access control policies.
+Commands provided in the Azure portal create self-hosted gateway resources in the **default** namespace which is automatically created, exists in every cluster, and can't be deleted.
+Consider [creating and deploying](https://kubernetesbyexample.com/ns/) self-hosted gateway into a separate namespace in production.
+
+### Number of replicas
+Minimal number of replicas suitable in production is two.
+
+By default, self-hosted gateway is deployed with a **RollingUpdate** deployment [strategy](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy). Review the default values and consider explicitly setting [**maxUnavailable**](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable) and [**maxSurge**](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge) fields, especially when using a high replica count.
+
+### Container resources
+By default, the YAML file provided in the Azure portal doesn't specify container resource requests.
 
-* To learn more about the self-hosted gateway, see [Azure API Management self-hosted gateway overview](self-hosted-gateway-overview.md)
-* Learn more about [Azure Kubernetes Service](https://docs.microsoft.com/azure/aks/intro-kubernetes)
+It's impossible to reliably predict and recommend the amount of per-container CPU and memory resources and the number of replicas required for supporting a specific workload due to many factors at play, e.g.:
 
+- Specific hardware the cluster is running on
+- Presence and type of virtualization
+- Number and rate of concurrent client connections
+- Request rate
+- Kind and number of configured policies
+- Payload size and if payloads are buffered or streamed
+- Backend service latency
+
+We recommend setting resource request to 2 cores and 2 GiB as a starting point, performing a load test and scaling up/out or down/in based the results.
+
+### Container image tag
+The YAML file provided in the Azure portal uses the **latest** tag which always references the most recent version of the self-hosted gateway container image.
+
+Consider using a specific version tag in production to avoid unintentional upgrade to a newer version.
+
+Follow this [link](https://mcr.microsoft.com/v2/azure-api-management/gateway/tags/list) to see the full list of available tags.
+
+### DNS policy
+DNS name resolution plays a critical role in self-hosted gateway's ability to connect to dependencies in Azure and dispatch API calls to backend services.
+
+The YAML file provided in the Azure portal applies the default [**ClusterFirst**](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy) policy which causes name resolution requests not resolved by the cluster DNS to be forwarded to the upstream DNS server inherited from the node.
+
+Follow this [link](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service) to learn about name resolution in Kubernetes and consider customizing [DNS policy](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy) or D[NS configuration](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-config) as appropriate for your setup.
+
+### Configuration backup
+Follow this [link](self-hosted-gateway-overview.md#connectivity-to-azure) to learn about self-hosted gateway behavior in the presence of a temporary Azure connectivity outage.
+Configure local storage volume for the self-hosted gateway container, so it can persist a backup copy of the latest downloaded configuration and, if connectivity is down, use it upon restart. The volume mount path must be  <code>/apim/config</code>. See an example [here](https://github.com/Azure/api-management-self-hosted-gateway/blob/master/examples/self-hosted-gateway-with-configuration-backup.yaml).
+To learn about storage in Kubernetes follow this [link](https://kubernetes.io/docs/concepts/storage/volumes/).
+
+### Local logs and metrics
+Self-hosted gateway sends telemetry to [Azure Monitor](api-management-howto-use-azure-monitor.md) and [Azure Application Insights](api-management-howto-app-insights.md) per configuration settings in the associated API Management service.
+When [connectivity to Azure](self-hosted-gateway-overview.md#connectivity-to-azure) is temporarily lost, the flow of telemetry to Azure is interrupted and the data is lost for the duration of the outage.
+Consider [setting up local monitoring](api-management-howto-configure-local-metrics-and-logs.md) to ensure ability to observe the API traffic and prevent telemetry loss during Azure connectivity outages.
+
+## Next steps
 
+* To learn more about the self-hosted gateway, see [Azure API Management self-hosted gateway overview](self-hosted-gateway-overview.md)

articles/api-management/media/api-management-howto-deploy-self-hosted-gateway-to-k8s/status.PNG

@@ -1,6 +1,6 @@
 ---
-title: Self-hosted Azure API Management gateway overview | Microsoft Docs
-description: Learn how self-hosted Azure API Management gateway helps organizations manage APIs in the hybrid and multicloud environments.
+title: Azure API Management self-hosted gateway overview | Microsoft Docs
+description: Learn how Azure API Management self-hosted gateway helps organizations manage APIs in the hybrid and multicloud environments.
 services: api-management
 documentationcenter: ''
 author: vlvinogr
@@ -11,7 +11,7 @@ ms.service: api-management
 ms.workload: mobile
 ms.tgt_pltfrm: na
 ms.topic: article
-ms.date: 03/31/2020
+ms.date: 04/24/2020
 ms.author: apimpm
 ---
 

articles/api-management/self-hosted-gateway-overview.md

@@ -8,7 +8,7 @@ author: vladvino
 ms.assetid: 1b813833-39c8-46be-8666-fd0960cfbf04
 ms.service: api-management
 ms.topic: include
-ms.date: 01/10/2020
+ms.date: 04/14/2020
 ms.author: vlvinogr
 ms.custom: include file
 ---
@@ -21,21 +21,24 @@ ms.custom: include file
 | Maximum cached response size | 2 MiB |
 | Maximum policy document size | 256 KiB<sup>5</sup> |
 | Maximum custom gateway domains per service instance<sup>6</sup> | 20 |
-| Maximum number of CA certificates per service instance | 10 |
-| Maximum number of service instances per subscription<sup>7</sup> | 20 |
-| Maximum number of subscriptions per service instance<sup>7</sup> | 500 |
-| Maximum number of client certificates per service instance<sup>7</sup> | 50 |
-| Maximum number of APIs per service instance<sup>7</sup> | 50 |
-| Maximum number of API operations per service instance<sup>7</sup> | 1,000 |
-| Maximum total request duration<sup>7</sup> | 30 seconds |
-| Maximum buffered payload size<sup>7</sup> | 2 MiB |
-| Maximum request URL size<sup>8</sup> | 4096 bytes |
+| Maximum number of CA certificates per service instance<sup>7</sup> | 10 |
+| Maximum number of service instances per subscription<sup>8</sup> | 20 |
+| Maximum number of subscriptions per service instance<sup>8</sup> | 500 |
+| Maximum number of client certificates per service instance<sup>8</sup> | 50 |
+| Maximum number of APIs per service instance<sup>8</sup> | 50 |
+| Maximum number of API operations per service instance<sup>8</sup> | 1,000 |
+| Maximum total request duration<sup>8</sup> | 30 seconds |
+| Maximum buffered payload size<sup>8</sup> | 2 MiB |
+| Maximum request URL size<sup>9</sup> | 4096 bytes |
+| Maximum number of self-hosted gateways<sup>10</sup> | 25 |
 
-<sup>1</sup>Scaling limits depend on the pricing tier. To see the pricing tiers and their scaling limits, see [API Management pricing](https://azure.microsoft.com/pricing/details/api-management/).<br/>
+<sup>1</sup>Scaling limits depend on the pricing tier. For details on the pricing tiers and their scaling limits, see [API Management pricing](https://azure.microsoft.com/pricing/details/api-management/).<br/>
 <sup>2</sup>Per unit cache size depends on the pricing tier. To see the pricing tiers and their scaling limits, see [API Management pricing](https://azure.microsoft.com/pricing/details/api-management/).<br/>
 <sup>3</sup>Connections are pooled and reused unless explicitly closed by the back end.<br/>
 <sup>4</sup>This limit is per unit of the Basic, Standard, and Premium tiers. The Developer tier is limited to 1,024. This limit doesn't apply to the Consumption tier.<br/>
 <sup>5</sup>This limit applies to the Basic, Standard, and Premium tiers. In the Consumption tier, policy document size is limited to 4 KiB.<br/>
-<sup>6</sup>This resource is available in the Premium tier only.<br/>
-<sup>7</sup>This resource applies to the Consumption tier only.<br/>
-<sup>8</sup>Applies to the Consumption tier only. Includes an up to 2048 bytes long query string.<br/>
+<sup>6</sup>Multiple custom domains are supported in the Developer and Premium tiers only.<br/>
+<sup>7</sup>CA certificates are not supported in the Consumption tier.<br/>
+<sup>8</sup>This resource applies to the Consumption tier only.<br/>
+<sup>9</sup>Applies to the Consumption tier only. Includes an up to 2048 bytes long query string.<br/>
+<sup>10</sup>Self-hosted gateways are supported in the Developer and Premium tiers only. The limit applies to the number of [self-hosted gateway resources](https://docs.microsoft.com/rest/api/apimanagement/2019-12-01/gateway). To raise this limit please contact [support](https://azure.microsoft.com/support/options/). Note, that the number of nodes (or replicas) associated with a self-hosted gateway resource is unlimited in the Premium tier and capped at a single node in the Developer tier.