Merge branch 'main' into enable_MD010

Author: Blargian

docs/cloud/security/aws-privatelink.md

@@ -6,6 +6,7 @@ description: 'How to set up Azure Private Link'
 keywords: ['azure', 'private link', 'privatelink']
 ---
 
+import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge';
 import azure_pe from '@site/static/images/cloud/security/azure-pe.png';
 import azure_privatelink_pe_create from '@site/static/images/cloud/security/azure-privatelink-pe-create.png';
 import azure_private_link_center from '@site/static/images/cloud/security/azure-private-link-center.png';
@@ -25,9 +26,7 @@ import azure_privatelink_pe_dns from '@site/static/images/cloud/security/azure-p
 
 # Azure Private Link
 
-:::note
-Azure Private Link can be enabled only on ClickHouse Cloud **Production** services. **Development** services are not supported.
-:::
+<ScalePlanFeatureBadge feature="Azure Private Link"/>
 
 This guide shows how to use Azure Private Link to provide private connectivity via a virtual network between Azure (including customer-owned and Microsoft Partner services) and ClickHouse Cloud. Azure Private Link simplifies the network architecture and secures the connection between endpoints in Azure by eliminating data exposure to the public internet.
 
@@ -39,7 +38,7 @@ Unlike AWS and GCP, Azure supports cross-region connectivity via Private Link. T
 Additional charges may be applied to inter-region traffic. Please check latest Azure documentation.
 :::
 
-Please complete the following steps to enable Azure Private Link:
+**Please complete the following steps to enable Azure Private Link:**
 
 1. Obtain Azure connection alias for Private Link
 1. Create a Private Endpoint in Azure
@@ -48,58 +47,67 @@ Please complete the following steps to enable Azure Private Link:
 1. Access your ClickHouse Cloud service using Private Link
 
 
-Find complete Terraform example for Azure Private Link [here](https://github.com/ClickHouse/terraform-provider-clickhouse/tree/main/examples/PrivateLinkAzure).
+## Attention {#attention}
+ClickHouse attempts to group your services to reuse the same published [Private Link service](https://learn.microsoft.com/en-us/azure/private-link/private-link-service-overview) within the Azure region. However, this grouping is not guaranteed, especially if you spread your services across multiple ClickHouse organizations.
+If you already have Private Link configured for other services in your ClickHouse organization, you can often skip most of the steps because of that grouping and proceed directly to the final step: [Add the Private Endpoint GUID to your service(s) allow list](#add-private-endpoint-guid-to-services-allow-list).
+
+Find Terraform examples at the ClickHouse [Terraform Provider repository](https://github.com/ClickHouse/terraform-provider-clickhouse/tree/main/examples/).
 
 ## Obtain Azure connection alias for Private Link {#obtain-azure-connection-alias-for-private-link}
 
 ### Option 1: ClickHouse Cloud console {#option-1-clickhouse-cloud-console}
 
-In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink, then open the **Settings** menu. Click on the **Set up private endpoint** button. Copy the **Service name** which will be used for setting up Private Link.
+In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink, then open the **Settings** menu. Click on the **Set up private endpoint** button. Make a note of the `Service name` and `DNS name`  which will be used for setting up Private Link.
 
 <img src={azure_privatelink_pe_create} alt="Private Endpoints" />
 
+Make a note of the `Service name` and `DNS name`, it will be needed in the next steps.
+
 ### Option 2: API {#option-2-api}
 
-Before you get started, you'll need a ClickHouse Cloud API key. You can [create a new key](/cloud/manage/openapi) or use an existing one. Note that you will need an **Admin** key to manage the Private Link configuration.
+Before you get started, you'll need a ClickHouse Cloud API key. You can [create a new key](/cloud/manage/openapi) or use an existing one.
 
 Once you have your API key, set the following environment variables before running any commands:
 
 ```bash
-REGION=<region code, use Azure format>
+REGION=<region code, use Azure format, for example: westus3>
 PROVIDER=azure
 KEY_ID=<Key ID>
 KEY_SECRET=<Key secret>
 ORG_ID=<set ClickHouse organization ID>
+SERVICE_NAME=<Your ClickHouse service name>
 ```
 
-Obtain an instance ID from your region:
-
-You'll need at least one ClickHouse Cloud service deployed in the specified region to perform this step.
+Get your ClickHouse `INSTANCE_ID` by filtering by region, provider and service name:
 
-```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services | jq ".result[] | select (.region==\"${REGION:?}\" and .provider==\"${PROVIDER:?}\") | .id " -r | head -1 | tee instance_id
-```
-
-Create an `INSTANCE_ID` environment variable using the ID you received in the previous step:
-
-```bash
-INSTANCE_ID=$(cat instance_id)
+```shell
+INSTANCE_ID=$(curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" \
+"https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services" | \
+jq ".result[] | select (.region==\"${REGION:?}\" and .provider==\"${PROVIDER:?}\" and .name==\"${SERVICE_NAME:?}\") | .id " -r)
 ```
 
 Obtain your Azure connection alias and Private DNS hostname for Private Link:
 
 ```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig | jq  .result
+curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig" | jq  .result
 {
-  "endpointServiceId": "production-westus3-0-0.63c890a9-4d32-48cc-a08c-8cd92dfb1ad3.westus3.azure.privatelinkservice",
-  ...
+  "endpointServiceId": "production-westus3-0-0.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.westus3.azure.privatelinkservice",
+  "privateDnsHostname": "xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud"
 }
 ```
 
 Make a note of the `endpointServiceId`. You'll use it in the next step.
 
 ## Create a Private Endpoint in Azure {#create-private-endpoint-in-azure}
 
+:::important
+This section covers ClickHouse-specific details for configuring ClickHouse via Azure Private Link. Azure-specific steps are provided as a reference to guide you on where to look, but they may change over time without notice from the Azure cloud provider. Please consider Azure configuration based on your specific use case.  
+
+Please note that ClickHouse is not responsible for configuring the required Azure private endpoints, DNS records.  
+
+For any issues related to Azure configuration tasks, contact Azure Support directly.
+:::
+
 In this section, we're going to create a Private Endpoint in Azure. You can use either the Azure Portal or Terraform.
 
 ### Option 1: Using Azure Portal to create a Private Endpoint in Azure {#option-1-using-azure-portal-to-create-a-private-endpoint-in-azure}
@@ -204,85 +212,24 @@ Under properties, find `resourceGuid` field and copy this value:
 
 ## Setting up DNS for Private Link {#setting-up-dns-for-private-link}
 
-You need will need to create a Private DNS zone (`${location_code}.privatelink.azure.clickhouse.cloud`) and attach it to your VNet to access resources via Private Link.
-
-### Create Private DNS zone {#create-private-dns-zone}
-
-**Option 1: Using Azure portal**
-
-Please follow the following guide to [create an Azure private DNS zone using the Azure Portal](https://learn.microsoft.com/en-us/azure/dns/private-dns-getstarted-portal).
-
-**Option 2: Using Terraform**
-
-Use the following Terraform template to create a Private DNS zone:
-
-```json
-resource "azurerm_private_dns_zone" "clickhouse_cloud_private_link_zone" {
-  name                = "${var.location}.privatelink.azure.clickhouse.cloud"
-  resource_group_name = var.resource_group_name
-}
-```
-
-### Create a wildcard DNS record {#create-a-wildcard-dns-record}
-
-Create a wildcard record and point to your Private Endpoint:
-
-**Option 1: Using Azure Portal**
-
-1. Open the `MyAzureResourceGroup` resource group and select the `${region_code}.privatelink.azure.clickhouse.cloud` private zone.
-2. Select + Record set.
-3. For Name, type `*`.
-4. For IP Address, type the IP address you see for Private Endpoint.
-5. Select **OK**.
-
-<img src={azure_pl_dns_wildcard} alt="Private Link DNS Wildcard Setup" />
-
-**Option 2: Using Terraform**
-
-Use the following Terraform template to create a wildcard DNS record:
-
-```json
-resource "azurerm_private_dns_a_record" "example" {
-  name                = "*"
-  zone_name           = var.zone_name
-  resource_group_name = var.resource_group_name
-  ttl                 = 300
-  records             = ["10.0.0.4"]
-}
-```
-
-### Create a virtual network link {#create-a-virtual-network-link}
-
-To link the private DNS zone to a virtual network, you'll need create a virtual network link.
-
-**Option 1: Using Azure Portal**
-
-Please follow the following guide to [link the virtual network to your private DNS zone](https://learn.microsoft.com/en-us/azure/dns/private-dns-getstarted-portal#link-the-virtual-network).
-
-**Option 2: Using Terraform**
+:::note
+There are various ways to configure DNS. Please set up DNS according to your specific use case.
+:::
 
-Use the following Terraform template to link the virtual network to your private DNS zone:
+You need to point "DNS name", taken from [Obtain Azure connection alias for Private Link](#obtain-azure-connection-alias-for-private-link) step, to Private Endpoint IP address. This ensures that services/components within your VPC/Network can resolve it properly.
 
-```json
-resource "azurerm_private_dns_zone_virtual_network_link" "example" {
-  name                  = "test"
-  resource_group_name   = var.resource_group_name
-  private_dns_zone_name = var.zone_name
-  virtual_network_id    = var.virtual_network_id
-}
-```
 
 ### Verify DNS setup {#verify-dns-setup}
 
-Any record within the `westus3.privatelink.azure.clickhouse.cloud` domain should be pointed to the Private Endpoint IP. (10.0.0.4 in this example).
+`xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud` domain should be pointed to the Private Endpoint IP. (10.0.0.4 in this example).
 
 ```bash
-nslookup instance-id.westus3.privatelink.azure.clickhouse.cloud.
-Server:         127.0.0.53
-Address:        127.0.0.53#53
+nslookup xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud.
+Server:		127.0.0.53
+Address:	127.0.0.53#53
 
 Non-authoritative answer:
-Name:   instance-id.westus3.privatelink.azure.clickhouse.cloud
+Name:	xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud
 Address: 10.0.0.4
 ```
 
@@ -309,7 +256,7 @@ ENDPOINT_ID=<Private Endpoint resourceGuid>
 REGION=<region code, use Azure format>
 ```
 
-Set the `VPC_ENDPOINT` environment variable using data from the [Obtaining the Private Endpoint `resourceGuid`](#obtaining-private-endpoint-resourceguid) step.
+Set the `ENDPOINT_ID` environment variable using data from the [Obtaining the Private Endpoint `resourceGuid`](#obtaining-private-endpoint-resourceguid) step.
 
 Run the following command to add the Private Endpoint:
 
@@ -351,7 +298,7 @@ EOF
 After adding or removing a Private Endpoint, run the following command to apply it to your organization:
 
 ```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} -X PATCH -H "Content-Type: application/json" https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?} -d @pl_config_org.json
+curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X PATCH -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}" -d @pl_config_org.json
 ```
 
 ## Add the Private Endpoint GUID to your service(s) allow list {#add-private-endpoint-guid-to-services-allow-list}
@@ -414,16 +361,13 @@ EOF
 After adding or removing a Private Endpoint to the services allow list, run the following command to apply it to your organization:
 
 ```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} -X PATCH -H "Content-Type: application/json" https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID?} -d @pl_config.json | jq
+curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X PATCH -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}" -d @pl_config.json | jq
 ```
 
 ## Access your ClickHouse Cloud service using Private Link {#access-your-clickhouse-cloud-service-using-private-link}
 
-Each service with Private Link enabled has a public and private endpoint. In order to connect using Private Link, you need to use a private endpoint which will be `privateDnsHostname`.
+Each service with Private Link enabled has a public and private endpoint. In order to connect using Private Link, you need to use a private endpoint which will be `privateDnsHostname`<sup>API</sup> or `DNS name`<sup>console</sup> taken from [Obtain Azure connection alias for Private Link](#obtain-azure-connection-alias-for-private-link).
 
-:::note
-Private DNS hostname is only available from your Azure VNet. Do not try to resolve the DNS host from a machine that resides outside of Azure VNet.
-:::
 
 ### Obtaining the Private DNS Hostname {#obtaining-the-private-dns-hostname}
 
@@ -447,7 +391,7 @@ INSTANCE_ID=<Instance ID>
 Run the following command:
 
 ```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig | jq  .result
+curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}/privateEndpointConfig" | jq  .result
 ```
 
 You should receive a response similar to the following:
@@ -467,19 +411,18 @@ Use the `privateDnsHostname` to connect to your ClickHouse Cloud service using P
 
 ### Test DNS setup {#test-dns-setup}
 
-All DNS records from the `${region_code}.privatelink.azure.clickhouse.cloud.` zone should be pointed to the internal IP address from the [*Create a Private Endpoint in Azure*](#create-private-endpoint-in-azure) step. In this example, the region is `westus3`.
-
 Run the following command:
 
 ```bash
-nslookup abcd.westus3.privatelink.azure.clickhouse.cloud.
+nslookup <dns name>
 ```
+where "dns name" `privateDnsHostname`<sup>API</sup> or `DNS name`<sup>console</sup> from [Obtain Azure connection alias for Private Link](#obtain-azure-connection-alias-for-private-link)
 
 You should receive the following response:
 
 ```response
 Non-authoritative answer:
-Name:   abcd.westus3.privatelink.azure.clickhouse.cloud
+Name:	<dns name>
 Address: 10.0.0.4
 ```
 
@@ -536,8 +479,7 @@ INSTANCE_ID=<Instance ID>
 Run the following command to check Private Endpoint filters:
 
 ```bash
-curl --silent --user ${KEY_ID:?}:${KEY_SECRET:?} -X GET -H "Content-Type: application/json" https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?} | jq .result.privateEndpointIds
-[]
+curl --silent --user "${KEY_ID:?}:${KEY_SECRET:?}" -X GET -H "Content-Type: application/json" "https://api.clickhouse.cloud/v1/organizations/${ORG_ID:?}/services/${INSTANCE_ID:?}" | jq .result.privateEndpointIds
 ```
 
 ## More information {#more-information}