Merge pull request #221053 from whhender/byo-event-hubs

BYO event hubs

Author: 19BMG00

articles/purview/catalog-private-link-end-to-end.md

@@ -6,7 +6,7 @@ ms.author: zeinam
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 06/21/2022
+ms.date: 12/09/2022
 # Customer intent: As a Microsoft Purview admin, I want to set up private endpoints for my Microsoft Purview account to access purview account and scan data sources from restricted network.
 ---
 
@@ -18,11 +18,11 @@ The Microsoft Purview _account_ private endpoint is used to add another layer of
 
 The Microsoft Purview _portal_ private endpoint is required to enable connectivity to [Microsoft Purview governance portal](https://web.purview.azure.com/resource/) using a private network.
 
-Microsoft Purview can scan data sources in Azure or an on-premises environment by using _ingestion_ private endpoints. Three private endpoint resources are required to be deployed and linked to Microsoft Purview managed resources when ingestion private endpoint is deployed:
+Microsoft Purview can scan data sources in Azure or an on-premises environment by using _ingestion_ private endpoints. Three private endpoint resources are required to be deployed and linked to Microsoft Purview managed or configured resources when ingestion private endpoint is deployed:
 
  - Blob private endpoint is linked to a Microsoft Purview managed storage account.
  - Queue private endpoint is linked to a Microsoft Purview managed storage account.
- - namespace private endpoint is linked to a Microsoft Purview managed Event Hub namespace.
+ - namespace private endpoint is linked to a Microsoft Purview configured Event Hub namespace.
 
   :::image type="content" source="media/catalog-private-link/purview-private-link-architecture.png" alt-text="Diagram that shows Microsoft Purview and Private Link architecture.":::
 

articles/purview/catalog-private-link-faqs.md

@@ -6,7 +6,7 @@ ms.author: zeinam
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 05/06/2022
+ms.date: 12/09/2022
 # Customer intent: As a Microsoft Purview admin, I want to set up private endpoints and managed vnets for my Microsoft Purview account for secure access or ingestion.
 ---
 # FAQ about Microsoft Purview private endpoints and Managed VNets
@@ -44,11 +44,11 @@ The Microsoft Purview portal private endpoint provides private connectivity to t
 
 ### What's the purpose of deploying the Microsoft Purview ingestion private endpoints?
 
-Microsoft Purview can scan data sources in Azure or an on-premises environment by using ingestion private endpoints. Three other private endpoint resources are deployed and linked to Microsoft Purview managed resources when ingestion private endpoints are created:
+Microsoft Purview can scan data sources in Azure or an on-premises environment by using ingestion private endpoints. Three other private endpoint resources are deployed and linked to Microsoft Purview managed or configured resources when ingestion private endpoints are created:
 
 - **Blob** is linked to a Microsoft Purview managed storage account.
 - **Queue** is linked to a Microsoft Purview managed storage account.
-- **namespace** is linked to a Microsoft Purview managed event hub namespace.
+- **namespace** is linked to a Microsoft Purview configured event hub namespace.
 
 ### Can I scan a data source through a public endpoint if a private endpoint is enabled on my Microsoft Purview account?
 

articles/purview/catalog-private-link-name-resolution.md

@@ -6,7 +6,7 @@ ms.author: viseshag
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 11/17/2022
+ms.date: 12/09/2022
 # Customer intent: As a Microsoft Purview admin, I want to set up private endpoints for my Microsoft Purview account, for secure access.
 ---
 
@@ -46,7 +46,7 @@ When you create ingestion, portal and account private endpoints, the DNS CNAME r
 
 - During the deployment of _portal_ private endpoint for your Microsoft Purview account, we also create a new private DNS zone that corresponds to the `privatelink` subdomain for Microsoft Purview as `privatelink.purviewstudio.azure.com` including DNS A resource records for _Web_.
 
-- If you enable ingestion private endpoints, additional DNS zones are required for managed resources. 
+- If you enable ingestion private endpoints, additional DNS zones are required for managed or configured resources. 
 
 The following table shows an example of Azure Private DNS zones and DNS A Records that are deployed as part of configuration of private endpoint for a Microsoft Purview account if you enable _Private DNS integration_ during the deployment: 
 
@@ -198,14 +198,14 @@ If you do not use DNS forwarders and instead you manage A records directly in yo
    |Portal     |Microsoft Purview          |`privatelink.purviewstudio.azure.com`        |Web         |
    |Ingestion     |Microsoft Purview managed Storage Account - Blob          |`privatelink.blob.core.windows.net`          |scaneastusabcd1234         |
    |Ingestion   |Microsoft Purview managed Storage Account - Queue         |`privatelink.queue.core.windows.net`         |scaneastusabcd1234         |
-   |Ingestion     |Microsoft Purview managed Storage Account - Event Hub         |`privatelink.servicebus.windows.net`         |atlas-12345678-1234-1234-abcd-123456789abc         |
+   |Ingestion     |Microsoft Purview configured Event Hubs - Event Hub         |`privatelink.servicebus.windows.net`         |atlas-12345678-1234-1234-abcd-123456789abc         |
 
 2. Create [Virtual network links](../dns/private-dns-virtual-network-links.md) in your Azure Private DNS Zones for your Azure Virtual Networks to allow internal name resolution.
 
 3. From your management PC and self-hosted integration runtime VM, test name resolution and network connectivity to your Microsoft Purview account using tools such as Nslookup.exe and PowerShell
 
 To test name resolution you need to resolve the following FQDNs through their private IP addresses:
-(Instead of Contoso-Purview, scaneastusabcd1234 or atlas-12345678-1234-1234-abcd-123456789abc, use the hostname associated with your purview account name and managed resources names)
+(Instead of Contoso-Purview, scaneastusabcd1234 or atlas-12345678-1234-1234-abcd-123456789abc, use the hostname associated with your purview account name and managed or configured resources names)
 
 - `Contoso-Purview.purview.azure.com`
 - `web.purview.azure.com`
@@ -214,7 +214,7 @@ To test name resolution you need to resolve the following FQDNs through their pr
 - `atlas-12345678-1234-1234-abcd-123456789abc.servicebus.windows.net`
 
 To test network connectivity, from self-hosted integration runtime VM you can launch PowerShell console and test connectivity using `Test-NetConnection`. 
-You must resolve each endpoint by their private endpoint and obtain TcpTestSucceeded as True. (Instead of Contoso-Purview, scaneastusabcd1234 or atlas-12345678-1234-1234-abcd-123456789abc, use the hostname associated with your purview account name and managed resources names)
+You must resolve each endpoint by their private endpoint and obtain TcpTestSucceeded as True. (Instead of Contoso-Purview, scaneastusabcd1234 or atlas-12345678-1234-1234-abcd-123456789abc, use the hostname associated with your purview account name and managed or configured resources names)
 
 - `Test-NetConnection -ComputerName Contoso-Purview.purview.azure.com -port 443`
 - `Test-NetConnection -ComputerName web.purview.azure.com -port 443`

articles/purview/catalog-private-link-troubleshoot.md

@@ -6,7 +6,7 @@ ms.author: zeinam
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 06/21/2022
+ms.date: 12/09/2022
 # Customer intent: As a Microsoft Purview admin, I want to set up private endpoints for my Microsoft Purview account, for secure access.
 ---
 
@@ -36,10 +36,10 @@ This guide summarizes known limitations related to using private endpoints for M
     |Portal     |Microsoft Purview Account         |mypurview-private-portal  |
     |Ingestion     |Managed Storage Account (Blob)         |mypurview-ingestion-blob  |
     |Ingestion     |Managed Storage Account (Queue)         |mypurview-ingestion-queue  |
-    |Ingestion     |Managed Event Hubs Namespace*         |mypurview-ingestion-namespace  |
+    |Ingestion     |Event Hubs Namespace*         |mypurview-ingestion-namespace  |
 
   >[!NOTE]
-  > *Managed Event Hubs Namespace is only needed if it has been enabled on your Microsoft Purview account. You can check in **Managed Resources** under settings on your Microsoft Purview account page in the Azure Portal.
+  > *Event Hubs Namespace is only needed if it has been configured on your Microsoft Purview account. You can check in **Kafka configuration** under settings on your Microsoft Purview account page in the Azure Portal.
 
 2. If portal private endpoint is deployed, make sure you also deploy account private endpoint.
 
@@ -87,7 +87,7 @@ This guide summarizes known limitations related to using private endpoints for M
    
 6. From self-hosted integration runtime VM, test network connectivity and name resolution to Microsoft Purview endpoint.
 
-7. From self-hosted integration runtime, test network connectivity and name resolution to Microsoft Purview managed resources such as blob queue and Event Hubs through port 443 and private IP addresses. (Replace the managed storage account and Event Hubs namespace with corresponding managed resource name assigned to your Microsoft Purview account).
+7. From self-hosted integration runtime, test network connectivity and name resolution to Microsoft Purview managed resources such as blob queue, and secondary resources like Event Hubs through port 443 and private IP addresses. (Replace the managed storage account and Event Hubs namespace with corresponding resource names).
 
     ```powershell
     Test-NetConnection -ComputerName `scansoutdeastasiaocvseab`.blob.core.windows.net -Port 443
@@ -131,9 +131,9 @@ This guide summarizes known limitations related to using private endpoints for M
     TcpTestSucceeded : True
     ```
 
-8. From the network where data source is located, test network connectivity and name resolution to Microsoft Purview endpoint and managed resources endpoints.
+8. From the network where data source is located, test network connectivity and name resolution to Microsoft Purview endpoint and managed or configured resources endpoints.
 
-9.  If data sources are located in on-premises network, review your DNS forwarder configuration. Test name resolution from within the same network where data sources are located to self-hosted integration runtime, Microsoft Purview endpoints and managed resources. It's expected to obtain a valid private IP address from DNS query for each endpoint.
+9.  If data sources are located in on-premises network, review your DNS forwarder configuration. Test name resolution from within the same network where data sources are located to self-hosted integration runtime, Microsoft Purview endpoints and managed or configured resources. It's expected to obtain a valid private IP address from DNS query for each endpoint.
     
     For more information, see [Virtual network workloads without custom DNS server](../private-link/private-endpoint-dns.md#virtual-network-workloads-without-custom-dns-server) and [On-premises workloads using a DNS forwarder](../private-link/private-endpoint-dns.md#on-premises-workloads-using-a-dns-forwarder) scenarios in [Azure Private Endpoint DNS configuration](../private-link/private-endpoint-dns.md).
 
@@ -149,10 +149,10 @@ You may receive the following error message when running a scan:
 `Internal system error. Please contact support with correlationId:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx System Error, contact support.`
 
 ### Cause 
-This can be an indication of issues related to connectivity or name resolution between the VM running self-hosted integration runtime and Microsoft Purview's managed resources storage account or Event Hubs.
+This can be an indication of issues related to connectivity or name resolution between the VM running self-hosted integration runtime and Microsoft Purview's managed storage account or configured Event Hubs.
 
 ### Resolution 
-Validate if name resolution is successful between the VM running the Self-Hosted Integration Runtime and the Microsoft Purview manage resources such as the blob queue and Event Hubs through port 443 and private IP addresses (step 8 above.)
+Validate if name resolution is successful between the VM running the Self-Hosted Integration Runtime and the Microsoft Purview managed blob queue or configured Event Hubs through port 443 and private IP addresses (step 8 above.)
 
 
 ### Issue 

articles/purview/concept-best-practices-automation.md

@@ -6,7 +6,7 @@ ms.author: tarifat
 ms.service: purview
 ms.subservice: purview-data-map
 ms.topic: conceptual
-ms.date: 11/03/2022
+ms.date: 12/09/2022
 ---
 
 # Microsoft Purview automation best practices
@@ -60,19 +60,13 @@ When to use?
 * Custom application development or process automation.
 
 ## Streaming (Apache Atlas)
-Each Microsoft Purview account can enable a fully managed event hub that is accessible via the Atlas Kafka endpoint found via the Azure portal > Microsoft Purview Account > Properties. 
 
-To enable this Event Hubs namespace, you can follow these steps:
-1. Search for and open your Microsoft Purview account in the [Azure portal](https://portal.azure.com).
-1. Select **Managed Resources** under settings on your Microsoft Purview account page in the Azure portal.
-    :::image type="content" source="media/concept-best-practices/enable-disable-event-hubs.png" alt-text="Screenshot showing the Event Hubs namespace toggle highlighted on the Managed resources page of the Microsoft Purview account page in the Azure portal.":::
-1. Select the Enable/Disable toggle to enable your Event Hubs namespace. It can be disabled at any time.
-1. Select **Save** to save the choice and begin the enablement or disablement process. This can take several minutes to complete.
-    :::image type="content" source="media/concept-best-practices/select-save.png" alt-text="Screenshot showing the Managed resources page of the Microsoft Purview account page in the Azure portal with the save button highlighted.":::
+Each Microsoft Purview account can configure Event Hubs that are accessible via their Atlas Kafka endpoint.
 
->[!NOTE]
->Enabling this Event Hubs namespace does incur a cost for the namespace. For specific details, see [the pricing page](https://azure.microsoft.com/pricing/details/purview/).
+[You can follow these steps to configure the Event Hubs namespaces.](configure-event-hubs-for-kafka.md)
 
+>[!NOTE]
+>Enabling this Event Hubs namespace does incur a cost for the namespace. For specific details, see [the pricing page](https://azure.microsoft.com/pricing/details/event-hubs/).
 
 Once the namespace is enabled, Microsoft Purview events can be monitored by consuming messages from the event hub. External systems can also use the event hub to publish events to Microsoft Purview as they occur.
 * **Consume Events** - Microsoft Purview will send notifications about metadata changes to Kafka topic **ATLAS_ENTITIES**. Applications interested in metadata changes can monitor for these notifications. Supported operations include: `ENTITY_CREATE`, `ENTITY_UPDATE`, `ENTITY_DELETE`, `CLASSIFICATION_ADD`, `CLASSIFICATION_UPDATE`, `CLASSIFICATION_DELETE`.
@@ -82,6 +76,7 @@ When to use?
 * Applications or processes that need to publish or consume Apache Atlas events in real time.
 
 ## Monitoring
+
 Microsoft Purview can send platform logs and metrics via "Diagnostic settings" to one or more destinations (Log Analytics Workspace, Storage Account, or Azure Event Hubs). [Available metrics](./how-to-monitor-with-azure-monitor.md#available-metrics) include `Data Map Capacity Units`, `Data Map Storage Size`, `Scan Canceled`, `Scan Completed`, `Scan Failed`, and `Scan Time Taken`.
 
 Once configured, Microsoft Purview automatically sends these events to the destination as a JSON payload. From there, application subscribers that need to consume and act on these events can do so with the option of orchestrating downstream logic.

articles/purview/concept-best-practices-network.md

@@ -6,7 +6,7 @@ ms.author: zeinam
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: conceptual
-ms.date: 03/04/2022
+ms.date: 12/09/2022
 ms.custom: fasttrack-edit
 ---