Update DNS and L3 steps.

Author: EthanChangAED

articles/iot-operations/manage-layered-network/howto-configure-l3-cluster-layered-network.md

@@ -14,7 +14,7 @@ ms.service: azure-iot-operations
 
 # Configure level 3 cluster in an isolated network with Azure IoT Layered Network Management (preview)
 
-You can configure a special isolated network environment for deploying Azure IoT Operations. For example, level 3 or lower in the ISA-95 network architecture. In this article, you set up a Kubernetes cluster to meet all the prerequisites of Azure IoT Operations and Arc-enable the cluster through the Azure IoT Layered Network Management (preview) service in the upper level. Before you start this process, the Layered Network Management (preview) service has to be ready for accepting the connection request from this level.
+You can configure a special isolated network environment for deploying Azure IoT Operations. For example, level 3 or lower in the ISA-95 network architecture. In this article, you set up a Kubernetes cluster and Arc-enable it through the Azure IoT Layered Network Management (preview) service in the upper level. Before you start this process, the Layered Network Management (preview) service has to be ready for accepting the connection request from this level.
 
 You'll complete the following tasks:
 - Set up the host system and install all the required software in an internet facing environment.
@@ -111,6 +111,10 @@ After the device is moved to your level 3 isolated network layer, it's required
     1. Open the **Wi-Fi Settings**.
     1. Select the setting of the current connection.
     1. In the IPv4 tab, disable the **Automatic** setting for DNS and enter the local IP of DNS server.
+    1. Restart the CoreDNS service.
+        ```bash
+        kubectl rollout restart -n kube-system deployment/coredns
+        ```
 
 # [AKS Edge Essentials](#tab/aksee)
 There are few limitations for setting up AKS Edge Essentials as the level 3 cluster.
@@ -129,9 +133,6 @@ If you're using VM to create your Windows 11 machines, use the [VM image](https:
 1. Download the [installer for the validated AKS Edge Essentials](https://aka.ms/aks-edge/msi-k3s-1.2.414.0) version.
 1. Install AKS Edge Essentials. Follow the steps in [Prepare your machines for AKS Edge Essentials](/azure/aks/hybrid/aks-edge-howto-setup-machine). Be sure to use the installer you downloaded in the previous step and not the most recent version.
 1. **Certificates:** For level 3 and lower, you ARC onboard the cluster that isn't connected to the internet. Therefore, you need to install certificates steps in [Prerequisites for AKS Edge Essentials offline installation](/azure/aks/hybrid/aks-edge-howto-offline-install).
-1. Install the following optional software if you plan to try Azure IoT Operations quickstarts or MQTT related scenarios.
-    - [MQTTUI](https://github.com/EdJoPaTo/mqttui/releases) or other MQTT client
-    - [Mosquitto](https://mosquitto.org/)
 1. Install Azure CLI. You can install the Azure CLI directly onto the level 3 machine or on another *developer* or *jumpbox* machine if you plan to access the level 3 cluster remotely. If you choose to access the Kubernetes cluster remotely to keep the cluster host clean, you run the *kubectl* and *az* related commands from the developer machine for the rest of the steps in this article.
     The *AKS Edge Essentials - Single machine deployment* does not support accessing Kubernetes remotely. If you want to enable remote kubectl access, you will need to create the [Full Kubernetes Deployment](/azure/aks/hybrid/aks-edge-howto-multi-node-deployment) instead. Additional configurations are needed when creating this type of Kubernetes cluster.
     - Install Azure CLI. Follow the steps in [Install Azure CLI on Windows](/cli/azure/install-azure-cli-windows).
@@ -150,25 +151,6 @@ To create the AKS Edge Essentials cluster that's compatible with Azure IoT Opera
 1. Complete the steps in [Create a single machine deployment](/azure/aks/hybrid/aks-edge-howto-single-node-deployment).
     Create a [Full Kubernetes Deployment](/azure/aks/hybrid/aks-edge-howto-multi-node-deployment) instead if you plan to remotely access the kubernetes from another machine.
 
-    At the end of [Step 1: single machine configuration parameters](/azure/aks/hybrid/aks-edge-howto-single-node-deployment#step-1-single-machine-configuration-parameters), modify the following values in the *aksedge-config.json* file as follows:
-
-    - `Init.ServiceIPRangeSize` = 10
-    - `LinuxNode.DataSizeInGB` = 30
-    - `LinuxNode.MemoryInMB` = 8192
-
-    In the **Network** section, set the `SkipDnsCheck` property to **true**. Add and set the `DnsServers` to the address of the DNS server in the subnet.
-
-    ```json
-    "DnsServers": ["<IP ADDRESS OF THE DNS SERVER IN SUBNET>"],
-    "SkipDnsCheck": true,
-    ```
-
-1. Install **local-path** storage in the cluster by running the following command:
-
-    ```cmd
-    kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/storage/local-path-provisioner/local-path-storage.yaml
-    ```
-
 ## Move the device to level 3 isolated network
 
 In your isolated network layer, the DNS server was configured in a prerequisite step using [Create sample network environment](./howto-configure-layered-network.md). Complete the step if you haven't done so.
@@ -256,19 +238,6 @@ login.microsoftonline.com. 0    IN      A       100.104.0.165
     ```
     > [!TIP]
     > If the `connectedk8s` commands fail, try using the cmdlets in [Connect your AKS Edge Essentials cluster to Arc](/azure/aks/hybrid/aks-edge-howto-connect-to-arc).
-1. Fetch the `objectId` or `id` of the Microsoft Entra ID application that the Azure Arc service uses.  Run the following command exactly as written, without changing the GUID value. The command you use depends on your version of Azure CLI:
-    ```powershell
-    # If you're using an Azure CLI version lower than 2.37.0, use the following command:
-    az ad sp show --id bc313c14-388c-4e7d-a58e-70017303ee3b --query objectId -o tsv
-    ```
-    ```powershell
-    # If you're using Azure CLI version 2.37.0 or higher, use the following command:
-    az ad sp show --id bc313c14-388c-4e7d-a58e-70017303ee3b --query id -o tsv
-    ```
-1. Use the [az connectedk8s enable-features](/cli/azure/connectedk8s#az-connectedk8s-enable-features) command to enable custom location support on your cluster. Use the `objectId` or `id` value from the previous command to enable custom locations on the cluster:
-   ```bash
-   az connectedk8s enable-features -n $CLUSTER_NAME -g $RESOURCE_GROUP --custom-locations-oid <objectId/id> --features cluster-connect custom-locations
-   ```
 
 ### Configure cluster network
 

articles/iot-operations/manage-layered-network/howto-configure-l4-cluster-layered-network.md

@@ -203,98 +203,54 @@ Create the Layered Network Management custom resource.
       allowList:
         enableArcDomains: true
         domains:
-        - destinationUrl: "*.arc.azure.net"
-          destinationType: external
-        - destinationUrl: "*.data.mcr.microsoft.com"
+        - destinationUrl: "management.azure.com"
           destinationType: external
         - destinationUrl: "*.dp.kubernetesconfiguration.azure.com"
           destinationType: external
-        - destinationUrl: "*.guestnotificationservice.azure.com"
-          destinationType: external
-        - destinationUrl: "*.his.arc.azure.com"
+        - destinationUrl: "login.microsoftonline.com"
           destinationType: external
         - destinationUrl: "*.login.microsoft.com"
           destinationType: external
-        - destinationUrl: "*.login.microsoftonline.com"
-          destinationType: external
-        - destinationUrl: "*.obo.arc.azure.com"
-          destinationType: external
-        - destinationUrl: "*.servicebus.windows.net"
-          destinationType: external
-        - destinationUrl: "graph.microsoft.com"
-          destinationType: external
         - destinationUrl: "login.windows.net"
           destinationType: external
-        - destinationUrl: "management.azure.com"
-          destinationType: external
         - destinationUrl: "mcr.microsoft.com"
           destinationType: external
-        - destinationUrl: "sts.windows.net"
-          destinationType: external
-        - destinationUrl: "*.ods.opinsights.azure.com"
-          destinationType: external
-        - destinationUrl: "graph.windows.net"
-          destinationType: external
-        - destinationUrl: "msit-onelake.pbidedicated.windows.net"
-          destinationType: external
-        - destinationUrl: "*.azurecr.io"
+        - destinationUrl: "*.data.mcr.microsoft.com"
           destinationType: external
-        - destinationUrl: "*.azureedge.net"
+        - destinationUrl: "gbl.his.arc.azure.com"
           destinationType: external
-        - destinationUrl: "*.blob.core.windows.net"
+        - destinationUrl: "*.his.arc.azure.com"
           destinationType: external
-        - destinationUrl: "*.prod.hot.ingestion.msftcloudes.com"
+        - destinationUrl: "k8connecthelm.azureedge.net"
           destinationType: external
-        - destinationUrl: "*.prod.microsoftmetrics.com"
+        - destinationUrl: "guestnotificationservice.azure.com"
           destinationType: external
-        - destinationUrl: "adhs.events.data.microsoft.com"
+        - destinationUrl: "*.guestnotificationservice.azure.com"
           destinationType: external
-        - destinationUrl: "dc.services.visualstudio.com"
+        - destinationUrl: "sts.windows.net"
           destinationType: external
-        - destinationUrl: "go.microsoft.com"
+        - destinationUrl: "k8sconnectcsp.azureedge.net"
           destinationType: external
-        - destinationUrl: "packages.microsoft.com"
+        - destinationUrl: "*.servicebus.windows.net"
           destinationType: external
-        - destinationUrl: "www.powershellgallery.com"
+        - destinationUrl: "graph.microsoft.com"
           destinationType: external
-        - destinationUrl: "*.gw.arc.azure.com"
+        - destinationUrl: "*.arc.azure.net"
           destinationType: external
-        - destinationUrl: "*.gcs.prod.monitoring.core.windows.net"
+        - destinationUrl: "*.obo.arc.azure.com"
           destinationType: external
-        - destinationUrl: "*.prod.warm.ingest.monitor.core.windows.net"
+        - destinationUrl: "linuxgeneva-microsoft.azurecr.io"
           destinationType: external
-        - destinationUrl: "*.prod.hot.ingest.monitor.core.windows.net"
+        - destinationUrl: "graph.windows.net"
           destinationType: external
-        - destinationUrl: "azure.archive.ubuntu.com"
+        - destinationUrl: "*.azurecr.io"
           destinationType: external
-        - destinationUrl: "crl.microsoft.com"
+        - destinationUrl: "*.blob.core.windows.net"
           destinationType: external
-        - destinationUrl: "*.table.core.windows.net"
+        - destinationUrl: "*.vault.azure.net"
           destinationType: external
         - destinationUrl: "*.blob.storage.azure.net"
           destinationType: external
-        - destinationUrl: "*.docker.com"
-          destinationType: external
-        - destinationUrl: "*.docker.io"
-          destinationType: external
-        - destinationUrl: "*.googleapis.com"
-          destinationType: external
-        - destinationUrl: "github.com"
-          destinationType: external
-        - destinationUrl: "collect.traefik.io"
-          destinationType: external
-        - destinationUrl: "contracts.canonical.com"
-          destinationType: external
-        - destinationUrl: "database.clamav.net"
-          destinationType: external
-        - destinationUrl: "esm.ubuntu.com"
-          destinationType: external
-        - destinationUrl: "livepatch.canonical.com"
-          destinationType: external
-        - destinationUrl: "motd.ubuntu.com"
-          destinationType: external
-        - destinationUrl: "update.traefik.io"
-          destinationType: external
         sourceIpRange:
         - addressPrefix: "0.0.0.0"
           prefixLen: 0

articles/iot-operations/manage-layered-network/howto-configure-layered-network.md

@@ -187,64 +187,57 @@ A custom DNS is only needed for levels 3 and below. This example uses a [dnsmasq
     apt install dnsmasq
     systemctl status dnsmasq
     ```
+
+    > [!NOTE]
+    > You might need to disable the *systemd-resolved* service if it causes conflict on port 53.
+    > ```bash
+    > systemctl disable --now systemd-resolved
+    > ```
 1. Modify the `/etc/dnsmasq.conf` file as shown to route these domains to the upper level.
     - Change the IPv4 address from 10.104.0.10 to respective destination address for that level. In this case, the IP address of the Layered Network Management service in the parent level.
-    - Verify the `interface` where you're running the *dnsmasq* and change the value as needed.
 
     The following configuration only contains the necessary endpoints for enabling Azure IoT Operations.
 
     ```conf
-    # Add domains which you want to force to an IP address here.
-    address=/management.azure.com/10.104.0.10
-    address=/dp.kubernetesconfiguration.azure.com/10.104.0.10
-    address=/.dp.kubernetesconfiguration.azure.com/10.104.0.10
-    address=/login.microsoftonline.com/10.104.0.10
-    address=/.login.microsoft.com/10.104.0.10
-    address=/.login.microsoftonline.com/10.104.0.10
-    address=/login.microsoft.com/10.104.0.10
-    address=/mcr.microsoft.com/10.104.0.10
-    address=/.data.mcr.microsoft.com/10.104.0.10
-    address=/gbl.his.arc.azure.com/10.104.0.10
-    address=/.his.arc.azure.com/10.104.0.10
-    address=/k8connecthelm.azureedge.net/10.104.0.10
-    address=/guestnotificationservice.azure.com/10.104.0.10
-    address=/.guestnotificationservice.azure.com/10.104.0.10
-    address=/sts.windows.nets/10.104.0.10
-    address=/k8sconnectcsp.azureedge.net/10.104.0.10
-    address=/.servicebus.windows.net/10.104.0.10
-    address=/servicebus.windows.net/10.104.0.10
-    address=/obo.arc.azure.com/10.104.0.10
-    address=/.obo.arc.azure.com/10.104.0.10
-    address=/adhs.events.data.microsoft.com/10.104.0.10
-    address=/dc.services.visualstudio.com/10.104.0.10
-    address=/go.microsoft.com/10.104.0.10
-    address=/onegetcdn.azureedge.net/10.104.0.10
-    address=/www.powershellgallery.com/10.104.0.10
-    address=/self.events.data.microsoft.com/10.104.0.10
-    address=/psg-prod-eastus.azureedge.net/10.104.0.10
-    address=/.azureedge.net/10.104.0.10
-    address=/api.segment.io/10.104.0.10
-    address=/nw-umwatson.events.data.microsoft.com/10.104.0.10
-    address=/sts.windows.net/10.104.0.10
-    address=/.azurecr.io/10.104.0.10
-    address=/.blob.core.windows.net/10.104.0.10
-    address=/global.metrics.azure.microsoft.scloud/10.104.0.10
-    address=/.prod.hot.ingestion.msftcloudes.com/10.104.0.10
-    address=/.prod.microsoftmetrics.com/10.104.0.10
-    address=/global.metrics.azure.eaglex.ic.gov/10.104.0.10
+    strict-order
+
+    # Arc endpoints
+    address=/management.azure.com/10.0.0.6
+    address=/.dp.kubernetesconfiguration.azure.com/10.0.0.6
+    address=/login.microsoftonline.com/10.0.0.6
+    address=/.login.microsoft.com/10.0.0.6
+    address=/login.windows.net/10.0.0.6
+    address=/mcr.microsoft.com/10.0.0.6
+    address=/.data.mcr.microsoft.com/10.0.0.6
+    address=/gbl.his.arc.azure.com/10.0.0.6
+    address=/.his.arc.azure.com/10.0.0.6
+    address=/k8connecthelm.azureedge.net/10.0.0.6
+    address=/guestnotificationservice.azure.com/10.0.0.6
+    address=/.guestnotificationservice.azure.com/10.0.0.6
+    address=/sts.windows.net/10.0.0.6
+    address=/k8sconnectcsp.azureedge.net/10.0.0.6
+    address=/.servicebus.windows.net/10.0.0.6
+    address=/graph.microsoft.com/10.0.0.6
+    address=/.arc.azure.net/10.0.0.6
+    address=/.obo.arc.azure.com/10.0.0.6
+    address=/linuxgeneva-microsoft.azurecr.io/10.0.0.6
+    # Azure CLI installation endpoints
+    address=/pypi.org/10.0.0.6
+    address=/files.pythonhosted.org/10.0.0.6
+    # Azure CLI endpoints
+    address=/graph.windows.net/10.0.0.6
+    address=/.azurecr.io/10.0.0.6
+    address=/.blob.core.windows.net/10.0.0.6
+    address=/.vault.azure.net/10.0.0.6
+    # AIO endpoints
+    address=/.blob.storage.azure.net/10.0.0.6
 
     # --address (and --server) work with IPv6 addresses too.
     address=/guestnotificationservice.azure.com/fe80::20d:60ff:fe36:f83
     address=/.guestnotificationservice.azure.com/fe80::20d:60ff:fe36:f833
     address=/.servicebus.windows.net/fe80::20d:60ff:fe36:f833
     address=/servicebus.windows.net/fe80::20d:60ff:fe36:f833
 
-    # If you want dnsmasq to listen for DHCP and DNS requests only on
-    # specified interfaces (and the loopback) give the name of the
-    # interface (eg eth0) here.
-    # Repeat the line for more than one interface.
-    interface=enp1s0
-
     listen-address=::1,127.0.0.1,10.102.0.72
 
     no-hosts
@@ -253,18 +246,14 @@ A custom DNS is only needed for levels 3 and below. This example uses a [dnsmasq
 1. As an alternative, you can put `address=/#/<IP of upper level Layered Network Management service>` in the IPv4 address section. For example:
 
     ```conf
+    strict-order
+
     # Add domains which you want to force to an IP address here.
     address=/#/<IP of upper level Layered Network Management service>
 
     # --address (and --server) work with IPv6 addresses too.
     address=/#/fe80::20d:60ff:fe36:f833
 
-    # If you want dnsmasq to listen for DHCP and DNS requests only on
-    # specified interfaces (and the loopback) give the name of the
-    # interface (eg eth0) here.
-    # Repeat the line for more than one interface.
-    interface=enp1s0
-
     listen-address=::1,127.0.0.1,10.102.0.72
 
     no-hosts