Merge pull request #229953 from baanders/3-8-egress

ADT: Add public access requirement for egress

Author: prmerger-automator[bot]

articles/digital-twins/concepts-data-history.md

@@ -5,7 +5,7 @@ titleSuffix: Azure Digital Twins
 description: Understand the data history feature for Azure Digital Twins.
 author: baanders
 ms.author: baanders # Microsoft employees only
-ms.date: 02/23/2023
+ms.date: 03/08/2023
 ms.topic: conceptual
 ms.service: digital-twins
 
@@ -28,9 +28,9 @@ For more of an introduction to data history, including a quick demo, watch the f
 ## Resources and data flow
 
 Data history requires the following resources:
-* Azure Digital Twins instance, with a [system-assigned managed identity](concepts-security.md#managed-identity-for-accessing-other-resources) enabled
-* [Event Hubs](../event-hubs/event-hubs-about.md) namespace containing an event hub
-* [Azure Data Explorer](/azure/data-explorer/data-explorer-overview) cluster containing a database 
+* Azure Digital Twins instance, with a [system-assigned managed identity](concepts-security.md#managed-identity-for-accessing-other-resources) enabled.
+* [Event Hubs](../event-hubs/event-hubs-about.md) namespace containing an event hub.
+* [Azure Data Explorer](/azure/data-explorer/data-explorer-overview) cluster containing a database. The cluster must have public network access enabled.
 
 These resources are connected into the following flow:
 

articles/digital-twins/concepts-data-ingress-egress.md

@@ -5,7 +5,7 @@ titleSuffix: Azure Digital Twins
 description: Learn about the data ingress and egress requirements for integrating Azure Digital Twins with other services.
 author: baanders
 ms.author: baanders # Microsoft employees only
-ms.date: 01/12/2023
+ms.date: 03/08/2023
 ms.topic: conceptual
 ms.service: digital-twins
 
@@ -37,10 +37,13 @@ You can also integrate Azure Digital Twins into a [Microsoft Power Platform](/po
 
 You may want to send Azure Digital Twins data to other downstream services for storage or additional processing. 
 
-Digital twin data can be sent to most Azure services using *endpoints*. If your destination is [Azure Data Explorer](/azure/data-explorer/data-explorer-overview), you can use *data history* instead to automatically send graph updates to an Azure Data Explorer cluster, where they are stored as historical data and can be queried as such. The rest of this section describes these capabilities in more detail.
+There are two main egress options in Azure Digital Twins. Digital twin data can be sent to most Azure services using *endpoints*. Or, if your destination is [Azure Data Explorer](/azure/data-explorer/data-explorer-overview), you can use *data history* to automatically send graph updates to an Azure Data Explorer cluster, where they are stored as historical data and can be queried as such.
 
->[!NOTE]
->Azure Digital Twins implements *at least once* delivery for data emitted to egress services. 
+In order for Azure Digital Twins to send data to other Azure services via endpoints or data history, the receiving service must have public network access enabled. Azure Digital Twins currently does not support any outbound communication to resources that have public network access disabled.
+
+Once the connection is set up, Azure Digital Twins implements *at least once* delivery for data emitted to egress services. 
+
+The rest of this section describes the two egress options in more detail.
 
 ### Endpoints
 

articles/digital-twins/how-to-create-data-history-connection.md

@@ -5,7 +5,7 @@ titleSuffix: Azure Digital Twins
 description: See how to set up a data history connection for historizing Azure Digital Twins updates into Azure Data Explorer.
 author: baanders
 ms.author: baanders # Microsoft employees only
-ms.date: 03/02/2023
+ms.date: 03/08/2023
 ms.topic: how-to
 ms.service: digital-twins
 ms.custom: event-tier1-build-2022
@@ -123,6 +123,9 @@ Next, create a Kusto (Azure Data Explorer) cluster and database to receive the d
 
 As part of the [data history connection setup](#set-up-data-history-connection) later, you'll grant the Azure Digital Twins instance the *Contributor* role on at least the database (it can also be scoped to the cluster), and the *Admin* role on the database.
 
+>[!IMPORTANT]
+>Make sure that the cluster has public network access enabled. If the Azure Data Explorer cluster has [public network access disabled](/azure/data-explorer/security-network-restrict-public-access), Azure Digital Twins will be unable to configure the tables and other required artifacts, and data history setup will fail.
+
 # [CLI](#tab/cli) 
 
 Use the following CLI commands to create the required resources. The commands use several local variables (`$location`, `$resourcegroup`, `$clustername`, and `$databasename`) that were created earlier in [Set up local variables for CLI session](#set-up-local-variables-for-cli-session).
@@ -182,11 +185,6 @@ When executing the above command, you'll be given the option of assigning the ne
 
 For regular data plane operation, these roles can be reduced to a single Azure Event Hubs Data Sender role, if desired.
 
->[!NOTE]
-> If you encounter the error "Could not create Azure Digital Twins instance connection. Unable to create table and mapping rule in database. Check your permissions for the Azure Database Explorer and run `az login` to refresh your credentials," resolve the error by adding yourself as an *AllDatabasesAdmin* under Permissions in your Azure Data Explorer cluster.
->
->If you're using the Cloud Shell and encounter the error "Failed to connect to MSI. Please make sure MSI is configured correctly," try running the command with a local Azure CLI installation instead.
-
 # [Portal](#tab/portal)
 
 Start by navigating to your Azure Digital Twins instance in the Azure portal (you can find the instance by entering its name into the portal search bar). Then complete the following steps.
@@ -231,6 +229,14 @@ After setting up the data history connection, you can optionally remove the role
 >[!NOTE]
 >Once the connection is set up, the default settings on your Azure Data Explorer cluster will result in an ingestion latency of approximately 10 minutes or less. You can reduce this latency by enabling [streaming ingestion](/azure/data-explorer/ingest-data-streaming) (less than 10 seconds of latency) or an [ingestion batching policy](/azure/data-explorer/kusto/management/batchingpolicy). For more information about Azure Data Explorer ingestion latency, see [End-to-end ingestion latency](concepts-data-history.md#end-to-end-ingestion-latency).
 
+### Troubleshoot connection setup
+
+Here are a few common errors you might encounter when setting up a data history connection, and how to resolve them.
+
+* If you have public network access disabled for your Azure Data Explorer cluster, you'll encounter an error that the service failed to create the data history connection, with the message "The resource could not ACT due to an internal server error." Data history setup will fail if the Azure Data Explorer cluster has [public network access disabled](/azure/data-explorer/security-network-restrict-public-access), since Azure Digital Twins will be unable to configure the tables and other required artifacts.
+* (CLI users) If you encounter the error "Could not create Azure Digital Twins instance connection. Unable to create table and mapping rule in database. Check your permissions for the Azure Database Explorer and run `az login` to refresh your credentials," resolve the error by adding yourself as an *AllDatabasesAdmin* under Permissions in your Azure Data Explorer cluster.
+* (Cloud Shell users) If you're using the Cloud Shell and encounter the error "Failed to connect to MSI. Please make sure MSI is configured correctly," try running the command with a local Azure CLI installation instead.
+
 ## Verify with a sample twin graph
 
 Now that your data history connection is set up, you can test it with data from your digital twins.