IoT Edge Frequently Disconnects from IoT Hub, Requires VM Restart to Restore Connectivity

[mvximvs]

Expected Behavior

I expect the connection to the IoT Hub to remain stable, but it appears to be disconnecting frequently.

Current Behavior

The IoT Edge periodically fails to connect, showing the messages below. However, after restarting the virtual machine, it functions normally:

iotedge logs:

 2025-03-12 02:27:27.333 +00:00 [INF] - Error authenticating token for SITE/Routing because the token is expired or could not be parsed CloudConnectionProvider.cs:line 256
<4> 2025-03-11 19:36:49.410 +00:00 [WRN] - Could not get an active Iot Hub connection for client SITE/SensorData
<6> 2025-03-11 19:36:51.535 +00:00 [INF] - Attempting to connect to IoT Hub for client SITE/SensorData via AMQP...
<6> 2025-03-11 19:37:27.332 +00:00 [INF] - Entering periodic task to reauthenticate connected clients
<4> 2025-03-11 19:37:54.464 +00:00 [WRN] - Error creating cloud connection for client SITE/SensorData
Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry.
 ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
   at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.ConnectAsyncResult.End(IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.<>c.<ConnectAsync>b__17_1(IAsyncResult r)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<<ExecuteWithErrorHandlingAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<<OpenInternalAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync()
   at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 158
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 57
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 62
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<<InvokeFunc>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 194
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 146
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 182
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 137
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 112
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 101
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 285
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 256
<6> 2025-03-11 19:37:54.465 +00:00 [INF] - Error getting cloud connection for device SITE/SensorData
Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry.
 ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
   at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.ConnectAsyncResult.End(IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.<>c.<ConnectAsync>b__17_1(IAsyncResult r)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout)
   at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<<ExecuteWithErrorHandlingAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<<OpenInternalAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken)
   at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync()
   at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 158
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 57
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 62
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<<InvokeFunc>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 194
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 146
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 182
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 137
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 112
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 101
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 285
   at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 256
<4> 2025-03-11 19:37:54.466 +00:00 [WRN] - Could not get an active Iot Hub connection for client SITE/SensorData
<6> 2025-03-11 19:37:57.891 +00:00 [INF] - Attempting to connect to IoT Hub for client SITE/SensorData via AMQP...
<4> 2025-03-11 19:38:59.530 +00:00 [WRN] - Error creating cloud connection for client SITE/SensorData

Steps to Reproduce

Recently, this issue occurs across all modules after the edge device has been operating for an extended period (days or weeks).

Context (Environment)

The machine where IoT Edge is installed is a virtual machine running Ubuntu 20.04.6. It is deployed at the organization’s edge, in an environment without internet access. The machine only has connectivity to specific services in the Azure cloud and is also configured to use a custom DNS.

Output of iotedge check

Click here

Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
× aziot-identity-service package is up-to-date - Error
    could not query https://aka.ms/azure-iotedge-latest-versions for latest available version
‼ host time is close to reference time - Warning
    Could not query NTP server
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
√ read all preloaded certificates from the Certificates Service - OK
√ read all preloaded key pairs from the Keys Service - OK
√ check all EST server URLs utilize HTTPS - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
√ host can connect to and perform TLS handshake with iothub AMQP port - OK
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
× configuration has correct URIs for daemon mgmt endpoint - Error
    Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.5.13' locally
    docker: Error response from daemon: Get "https://mcr.microsoft.com/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers).
    See 'docker run --help'.
× aziot-edge package is up-to-date - Error
    Error while fetching latest versions of edge components: could not send HTTP request
× container time is close to host time - Error
    Could not query local time inside container
√ DNS server - OK
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
    The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
‼ production readiness: Edge Hub's storage directory is persisted on the host filesystem - Warning
    The edgeHub module is not configured to persist its /tmp/edgeHub directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
√ Agent image is valid and can be pulled from upstream - OK
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK

Connectivity checks
-------------------
× container on the default network can connect to upstream AMQP port - Error
    Container on the default network could not connect to iothub.azure-devices.net:5671
× container on the default network can connect to upstream HTTPS / WebSockets port - Error
    Container on the default network could not connect to iothub.azure-devices.net:443
× container on the IoT Edge module network can connect to upstream AMQP port - Error
    Container on the azure-iot-edge network could not connect to iothub.azure-devices.net:5671
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to iothub.azure-devices.net:443
23 check(s) succeeded.
4 check(s) raised warnings. Re-run with --verbose for more details.
8 check(s) raised errors. Re-run with --verbose for more details.
2 check(s) were skipped due to errors from other checks. Re-run with --verbose for more details.

Device Information

• Host OS : Ubuntu 20.04.6 LTS (Focal Fossa)
• Architecture amd64
• Container OS: Linux containers

Runtime Versions

• aziot-edged: iotedge 1.5.13

• Edge Agent: 1.5.10

• Edge Hub: 1.5.10

• Docker/Moby:
  - Client:
  Version: 27.0.3-1
  API version: 1.46
  Go version: go1.21.12
  Git commit: 7d4bcd863a4c863e650eed02a550dfeb98560b83
  Built: Fri Jun 28 14:56:30 UTC 2024
  OS/Arch: linux/amd64
  Context: default

  - Server:
   Engine:
    Version:          27.0.3-1
    API version:      1.46 (minimum version 1.24)
    Go version:       go1.21.12
    Git commit:       662f78c0b1bb5114172427cfcb40491d73159be2
    Built:            Fri Jun 28 21:16:49 2024
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.6.36-1
    GitCommit:        88c3d9bc5b5a193f40b7c14fa996d23532d6f956
   runc:
    Version:          1.1.15-1
    GitCommit:        bc20cb4497af9af01bea4a8044f1678ffca2745c
   docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0
  

Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead

Logs

aziot-edged logs

Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/encrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8330"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - --> 200 {"content-type": "application/json"}
Mar 14 12:31:16 storm-edge-vsrv aziot-edged[89606]: 2025-03-14T11:31:16Z [INFO] - <-- POST /modules/%24edgeHub/genid/638663966182168843/decrypt?api-version=2020-07-07 {"accept": "application/json", "host": "workload.sock:80", "connection": "close", "content-type": "application/json", "content-length": "8355"}

edge-agent logs

  <6> 2025-03-13 17:01:42.536 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 17:01:42.537 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 17:01:42.537 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 17:01:42.538 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 17:01:42.552 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 17:01:42.558 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 17:01:42.558 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 17:02:01.799 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 17:02:02.190 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 17:02:02.191 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 18:01:41.830 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-13 18:01:41.830 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-13 18:01:41.831 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-13 18:01:41.831 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-13 18:01:41.831 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-13 18:01:42.561 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 18:01:42.561 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 18:01:42.561 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 18:01:42.562 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 18:01:42.573 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 18:01:42.585 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 18:01:42.585 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 18:02:02.192 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 18:02:02.699 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 18:02:02.700 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 19:01:42.584 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 19:01:42.584 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 19:01:42.584 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 19:01:42.586 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 19:01:42.597 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 19:01:42.604 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 19:01:42.604 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 19:02:02.700 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 19:02:03.114 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 19:02:03.115 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 19:54:51.620 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 20:01:41.827 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-13 20:01:41.827 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-13 20:01:41.827 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-13 20:01:41.827 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-13 20:01:41.827 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-13 20:01:42.599 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 20:01:42.599 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 20:01:42.599 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 20:01:42.600 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 20:01:42.611 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 20:01:42.617 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 20:01:42.617 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 20:02:03.115 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 20:02:03.630 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 20:02:03.631 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 20:51:42.892 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 21:01:42.615 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 21:01:42.615 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 21:01:42.615 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 21:01:42.617 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 21:01:42.628 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 21:01:42.635 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 21:01:42.635 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 21:02:03.632 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 21:02:03.915 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 21:02:03.917 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 21:04:19.958 +00:00 [INF] - Starting periodic operation Metrics Upload...
  <6> 2025-03-13 21:04:19.959 +00:00 [INF] - Uploading Metrics
  <6> 2025-03-13 21:04:20.224 +00:00 [INF] - Published metrics
  <6> 2025-03-13 21:04:20.230 +00:00 [INF] - Successfully completed periodic operation Metrics Upload
  <6> 2025-03-13 22:01:41.828 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-13 22:01:41.829 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-13 22:01:41.829 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-13 22:01:41.829 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-13 22:01:41.829 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-13 22:01:42.636 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 22:01:42.636 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 22:01:42.636 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 22:01:42.637 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 22:01:42.649 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 22:01:42.655 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 22:01:42.655 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 22:02:03.916 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 22:02:04.215 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 22:02:04.216 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-13 23:01:42.656 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-13 23:01:42.656 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-13 23:01:42.656 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-13 23:01:42.658 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-13 23:01:42.669 +00:00 [INF] - Storing Metrics
  <6> 2025-03-13 23:01:42.676 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-13 23:01:42.676 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-13 23:02:04.218 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-13 23:02:04.488 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-13 23:02:04.490 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 00:01:41.828 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 00:01:41.828 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 00:01:41.828 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 00:01:41.828 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 00:01:41.828 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 00:01:42.676 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 00:01:42.676 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 00:01:42.676 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 00:01:42.678 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 00:01:42.689 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 00:01:42.696 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 00:01:42.696 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 00:02:04.491 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 00:02:04.913 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 00:02:04.915 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 00:13:36.481 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 01:01:42.698 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 01:01:42.698 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 01:01:42.698 +00:00 [INF] - Scraping endpoint http://edge/metrics
  <6> 2025-03-14 01:01:42.699 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 01:01:42.709 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 01:01:42.716 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 01:01:42.716 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 01:02:04.915 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 01:02:05.829 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 01:02:05.830 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 02:01:41.827 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 02:01:41.827 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 02:01:41.827 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 02:01:41.827 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 02:01:41.827 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 02:01:42.716 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 02:01:42.716 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 02:01:42.716 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 02:01:42.718 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 02:01:42.730 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 02:01:42.737 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 02:01:42.737 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 02:02:05.833 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 02:02:06.519 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 02:02:06.521 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 03:01:42.736 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 03:01:42.736 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 03:01:42.736 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 03:01:42.738 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 03:01:42.748 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 03:01:42.755 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 03:01:42.755 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 03:02:06.519 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 03:02:07.147 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 03:02:07.148 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 04:01:41.830 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 04:01:41.830 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 04:01:41.830 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 04:01:41.830 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 04:01:41.830 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 04:01:42.758 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 04:01:42.758 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 04:01:42.758 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 04:01:42.760 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 04:01:42.771 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 04:01:42.778 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 04:01:42.778 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 04:02:07.147 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 04:02:07.669 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 04:02:07.670 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 05:01:42.780 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 05:01:42.780 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 05:01:42.780 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 05:01:42.782 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 05:01:42.796 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 05:01:42.802 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 05:01:42.802 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 05:02:07.673 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 05:02:08.080 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 05:02:08.083 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 06:01:41.830 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 06:01:41.830 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 06:01:41.830 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 06:01:41.830 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 06:01:41.830 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 06:01:42.805 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 06:01:42.805 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 06:01:42.805 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 06:01:42.806 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 06:01:42.817 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 06:01:42.824 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 06:01:42.824 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 06:02:08.086 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 06:02:08.649 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 06:02:08.652 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 07:01:42.826 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 07:01:42.826 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 07:01:42.826 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 07:01:42.828 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 07:01:42.842 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 07:01:42.849 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 07:0 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 07:02:08.652 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 07:02:09.324 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 07:02:09.325 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 08:01:41.830 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 08:01:41.830 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 08:01:41.830 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 08:01:41.830 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 08:01:41.830 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 08:01:42.847 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 08:01:42.848 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 08:01:42.848 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 08:01:42.849 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 08:01:42.861 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 08:01:42.871 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 08:01:42.871 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 08:02:09.325 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 08:02:10.043 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 08:02:10.043 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 08:06:57.823 +00:00 [INF] - Starting periodic operation Metrics Upload...
  <6> 2025-03-14 08:06:57.823 +00:00 [INF] - Uploading Metrics
  <6> 2025-03-14 08:06:58.395 +00:00 [INF] - Published metrics
  <6> 2025-03-14 08:06:58.404 +00:00 [INF] - Successfully completed periodic operation Metrics Upload
  <6> 2025-03-14 09:01:42.874 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 09:01:42.874 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 09:01:42.875 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 09:01:42.876 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 09:01:42.888 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 09:01:42.897 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 09:01:42.897 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 09:02:10.045 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 09:02:10.990 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 09:02:10.993 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 10:01:41.829 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 10:01:41.829 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 10:01:41.829 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 10:01:41.829 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 10:01:41.829 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 10:01:42.897 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 10:01:42.897 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 10:01:42.897 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 10:01:42.899 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 10:01:42.911 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 10:01:42.917 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 10:01:42.917 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 10:02:10.993 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 10:02:11.643 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 10:02:11.645 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 11:01:42.917 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 11:01:42.917 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 11:01:42.917 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 11:01:42.919 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 11:01:42.928 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 11:01:42.935 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 11:01:42.935 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 11:02:11.643 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 11:02:12.043 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 11:02:12.044 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 12:01:41.830 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 12:01:41.830 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 12:01:41.830 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 12:01:41.830 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 12:01:41.830 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 12:01:42.934 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 12:01:42.935 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 12:01:42.935 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 12:01:42.935 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 12:01:42.944 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 12:01:42.953 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 12:01:42.953 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 12:02:12.042 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 12:02:12.880 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 12:02:12.881 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 13:01:42.952 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 13:01:42.952 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 13:01:42.952 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 13:01:42.953 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 13:01:42.964 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 13:01:42.974 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 13:01:42.974 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 13:02:12.881 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 13:02:13.417 +00:00 [INF] - Obtained nt twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 13:02:13.418 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 14:01:41.827 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 14:01:41.827 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 14:01:41.827 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 14:01:41.827 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 14:01:41.827 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 14:01:42.976 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 14:01:42.976 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 14:01:42.976 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 14:01:42.977 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 14:01:42.986 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 14:01:42.993 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 14:01:42.993 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 14:02:13.421 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 14:02:14.209 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 14:02:14.210 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 14:28:50.735 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 15:01:42.992 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 15:01:42.992 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 15:01:42.992 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 15:01:42.993 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 15:01:43.001 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 15:01:43.008 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 15:01:43.008 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 15:02:14.211 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 15:02:14.478 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 15:02:14.479 +00:00 [INF] - Successfully completed periodic operation refresh twin config
  <6> 2025-03-14 16:01:41.827 +00:00 [INF] - Starting compaction of stores
  <6> 2025-03-14 16:01:41.827 +00:00 [INF] - Starting compaction of store moduleState
  <6> 2025-03-14 16:01:41.827 +00:00 [INF] - Starting compaction of store Metrics
  <6> 2025-03-14 16:01:41.827 +00:00 [INF] - Starting compaction of store deploymentConfig
  <6> 2025-03-14 16:01:41.827 +00:00 [INF] - Starting compaction of store default
  <6> 2025-03-14 16:01:43.009 +00:00 [INF] - Starting periodic operation Metrics Scrape...
  <6> 2025-03-14 16:01:43.009 +00:00 [INF] - Scraping Metrics
  <6> 2025-03-14 16:01:43.009 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
  <6> 2025-03-14 16:01:43.010 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
  <6> 2025-03-14 16:01:43.028 +00:00 [INF] - Storing Metrics
  <6> 2025-03-14 16:01:43.036 +00:00 [INF] - Scraped and Stored Metrics
  <6> 2025-03-14 16:01:43.036 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
  <6> 2025-03-14 16:02:14.482 +00:00 [INF] - Starting periodic operation refresh twin config...
  <6> 2025-03-14 16:02:14.741 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 27 and reported properties version 290.
  <6> 2025-03-14 16:02:14.743 +00:00 [INF] - Successfully completed periodic operation refresh twin config

edge-hub logs

<6> 2025-03-14 16:48:31.928 +00:00 [INF] - Client SITE_Sensor-01 in device scope authenticated locally.
<6> 2025-03-14 16:48:31.928 +00:00 [INF] - Successfully generated identity for clientId SITE_Sensor-03 and username site-edge-vsrv/SITE_Sensor-03/?api-version=2020-09-30&DeviceClientType=.NET%2F1.42.3%2B2a97dd06e82cb67c5be8a7c7d189bef8048a3a86%20%28.NET%208.0.13%3B%20Microsoft%20Windows%2010.0.22631%20WindowsProduct%3A0x00000030%3B%20X64%3B%20%7B039D3D25-C8DD-4E11-8970-23F990223FD4%7D%29
<6> 2025-03-14 16:48:31.928 +00:00 [INF] - "ClientAuthenticated, SITE_Sensor-03, 691bd90a"
<6> 2025-03-14 16:48:31.928 +00:00 [INF] - Successfully generated identity for clientId SITE_Sensor-01 and username site-edge-vsrv/SITE_Sensor-01/?api-version=2020-09-30&DeviceClientType=.NET%2F1.42.3%2B2a97dd06e82cb67c5be8a7c7d189bef8048a3a86%20%28.NET%208.0.13%3B%20Microsoft%20Windows%2010.0.22631%20WindowsProduct%3A0x00000030%3B%20X64%3B%20%7B039D3D25-C8DD-4E11-8970-23F990223FD4%7D%29
<6> 2025-03-14 16:48:31.928 +00:00 [INF] - "ClientAuthenticated, SITE_Sensor-01, 090bad2b"
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - New device connection for device SITE_Sensor-03
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - New device connection for device SITE_Sensor-01
<4> 2025-03-14 16:48:31.929 +00:00 [WRN] - "Closing connection for device: SITE_Sensor-01, Microsoft.Azure.Devices.Edge.Hub.Core.MultipleConnectionsException: Multiple connections detected for device SITE_Sensor-01, "
<4> 2025-03-14 16:48:31.929 +00:00 [WRN] - "Closing connection for device: SITE_Sensor-03, Microsoft.Azure.Devices.Edge.Hub.Core.MultipleConnectionsException: Multiple connections detected for device SITE_Sensor-03, "
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Disposing MessagingServiceClient for device Id SITE_Sensor-01 because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.MultipleConnectionsException: Multiple connections detected for device SITE_Sensor-01
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Disposing MessagingServiceClient for device Id SITE_Sensor-03 because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.MultipleConnectionsException: Multiple connections detected for device SITE_Sensor-03
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Closing device proxy for device Id SITE_Sensor-01
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Closing device proxy for device Id SITE_Sensor-03
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Client SITE_Sensor-03 connected to edgeHub, processing existing subscriptions.
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Client SITE_Sensor-01 connected to edgeHub, processing existing subscriptions.
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Skipping SITE_Sensor-03 for subscription processing, as it is currently being processed.
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Skipping SITE_Sensor-01 for subscription processing, as it is currently being processed.
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Bind device proxy for device SITE_Sensor-01
<6> 2025-03-14 16:48:31.929 +00:00 [INF] - Bind device proxy for device SITE_Sensor-03

Additional Information

The IoT Edge runs on a virtualized Ubuntu instance via Hyper-V, with Windows 11 Professional (version 24H2) as the host operating system.

[jlian]

In the checks I see

× container on the default network can connect to upstream AMQP port - Error
    Container on the default network could not connect to iothub.azure-devices.net:5671
× container on the default network can connect to upstream HTTPS / WebSockets port - Error
    Container on the default network could not connect to iothub.azure-devices.net:443
× container on the IoT Edge module network can connect to upstream AMQP port - Error
    Container on the azure-iot-edge network could not connect to iothub.azure-devices.net:5671
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to iothub.azure-devices.net:443

Which seems like the environment has connectivity issues to IoT Hub. IoT Edge needs not only the AMQP port but the 443 HTTP port as well. Can you check your firewall/network rules?

https://learn.microsoft.com/en-us/azure/iot-edge/troubleshoot#check-your-firewall-and-port-configuration-rules

Also, when the problem occurs, can you SSH into the VM and just try something like nslookup iothub.azure-devices.net?

[jlian]

@mvximvs any updates?

[infused-kim]

I ran into this issue too. The root cause was that systctl net.ipv4.ip_forward was set to 0 on my host. Setting it back to 1 restored connectivity of the iot edge containers.

[arsing]

Closing because this seems to be an environment issue. The error in the code block in the OP is EAGAIN, which is usually from DNS resolution failure. If you still have the issue then please verify DNS is working inside the VM, as jlian said in this comment.