Merge branch 'main' of https://github.com/MicrosoftDocs/azure-docs-pr into premPlus

Author: roygara

articles/event-hubs/azure-event-hubs-kafka-overview.md

@@ -1,6 +1,6 @@
 ---
-title: Introduction to Apache Kafka on Azure Event Hubs
-description: Learn what Apache Kafka on Azure Event Hubs is and how to use it to stream data from Apache Kafka applications without setting up a Kafka cluster on your own.
+title: Introduction to Apache Kafka in Event Hubs on Azure Cloud
+description: Learn what Apache Kafka in the Event Hubs service on Azure Cloud is and how to use it to stream data from Apache Kafka applications without setting up a Kafka cluster on your own.
 ms.topic: overview
 ms.date: 02/03/2023
 ---

articles/purview/catalog-lineage-user-guide.md

@@ -18,7 +18,7 @@ One of the platform features of Microsoft Purview is the ability to show the lin
 ## Lineage collection
 
  Metadata collected in Microsoft Purview from enterprise data systems are stitched across to show an end to end data lineage. Data systems that collect lineage into Microsoft Purview are broadly categorized into following three types:
- 
+
  - [Data processing systems](#data-processing-systems)
  - [Data storage systems](#data-storage-systems)
  - [Data analytics and reporting systems](#data-analytics-and-reporting-systems)
@@ -60,7 +60,7 @@ Databases & storage solutions such as Oracle, Teradata, and SAP have query engin
 || [SAP S/4HANA](register-scan-saps4hana-source.md) |
 
 ### Data analytics and reporting systems
-Data analytics and reporting systems like Azure ML and Power BI report lineage into Microsoft Purview. These systems will use the datasets from storage systems and process through their meta model to create BI Dashboards, ML experiments and so on.
+Data analytics and reporting systems like Azure Machine Learning and Power BI report lineage into Microsoft Purview. These systems will use the datasets from storage systems and process through their meta model to create BI Dashboards, ML experiments and so on.
 
 | Data analytics & reporting system | Supported scope |
 | ---------------------- | ------------|

articles/purview/concept-data-lineage.md

@@ -13,15 +13,15 @@ This article provides an overview of data lineage in Microsoft Purview Data Cata
 
 - Raw data staged from various platforms
 - Transformed and prepared data
-- Data used by visualization platforms.
+- Data used by visualization platforms
 
 ## Use cases
 
 Data lineage is broadly understood as the lifecycle that spans the data’s origin, and where it moves over time across the data estate. It's used for different kinds of backwards-looking scenarios such as troubleshooting, tracing root cause in data pipelines and debugging. Lineage is also used for data quality analysis, compliance and “what if” scenarios often referred to as impact analysis. Lineage is represented visually to show data moving from source to destination including how the data was transformed. Given the complexity of most enterprise data environments, these views can be hard to understand without doing some consolidation or masking of peripheral data points.
 
 ## Lineage experience in Microsoft Purview Data Catalog
 
-Microsoft Purview Data Catalog will connect with other data processing, storage, and analytics systems to extract lineage information. The information is combined to represent a generic, scenario-specific lineage experience in the Catalog.
+Microsoft Purview Data Catalog will connect with other data processing, storage, and analytics systems to extract lineage information. The information is combined to represent a generic, scenario-specific lineage experience in the catalog.
 
 :::image type="content" source="media/concept-lineage/lineage-end-end-inline.png" alt-text="end-end lineage showing data copied from blob store all the way to Power BI dashboard" lightbox="media/concept-lineage/lineage-end-end.png":::
 

articles/purview/concept-scans-and-ingestion.md

@@ -6,19 +6,24 @@ ms.author: shjia
 ms.service: purview
 ms.subservice: purview-data-map
 ms.topic: conceptual
-ms.date: 02/14/2023
+ms.date: 03/13/2023
 ms.custom: ignite-fall-2021
 ---
 
 # Scans and ingestion in Microsoft Purview
 
 This article provides an overview of the Scanning and Ingestion features in Microsoft Purview. These features connect your Microsoft Purview account to your sources to populate the data map and data catalog so you can begin exploring and managing your data through Microsoft Purview.
 
+- [**Scanning**](#scanning) captures metadata from [data sources](microsoft-purview-connector-overview.md) and brings it to Microsoft Purview.
+- [**Ingestion**](#ingestion) processes metadata and stores it in the data catalog from both:
+    - Data source scans - scanned metadata is added to the Microsoft Purview Data Map.
+    - Lineage connections - transformation resources add metadata about their sources, outputs, and activities to the Microsoft Purview Data Map.
+
 ## Scanning
 
 After data sources are [registered](manage-data-sources.md) in your Microsoft Purview account, the next step is to scan the data sources. The scanning process establishes a connection to the data source and captures technical metadata like names, file size, columns, and so on. It also extracts schema for structured data sources, applies classifications on schemas, and [applies sensitivity labels if your Microsoft Purview Data Map is connected to a Microsoft Purview compliance portal](create-sensitivity-label.md). The scanning process can be triggered to run immediately or can be scheduled to run on a periodic basis to keep your Microsoft Purview account up to date.
 
-For each scan there are customizations you can apply so that you're only scanning your sources for the information you need.
+For each scan, there are customizations you can apply so that you're only scanning information you need, rather than the whole source.
 
 ### Choose an authentication method for your scans
 
@@ -48,15 +53,15 @@ There are [system scan rule sets](create-a-scan-rule-set.md#system-scan-rule-set
 
 ### Schedule your scan
 
-Microsoft Purview gives you a choice of scanning weekly or monthly at a specific time you choose. Weekly scans may be appropriate for data sources with structures that are actively under development or frequently change. Monthly scanning is more appropriate for data sources that change infrequently. A good best practice is to work with the administrator of the source you want to scan to identify a time when compute demands on the source are low.
+Microsoft Purview gives you a choice of scanning weekly or monthly at a specific time you choose. Weekly scans may be appropriate for data sources with structures that are actively under development or frequently change. Monthly scanning is more appropriate for data sources that change infrequently. Best practice is to work with the administrator of the source you want to scan to identify a time when compute demands on the source are low.
 
 ### How scans detect deleted assets
 
 A Microsoft Purview catalog is only aware of the state of a data store when it runs a scan. For the catalog to know if a file, table, or container was deleted, it compares the last scan output against the current scan output. For example, suppose that the last time you scanned an Azure Data Lake Storage Gen2 account, it included a folder named *folder1*. When the same account is scanned again, *folder1* is missing. Therefore, the catalog assumes the folder has been deleted.
 
 #### Detecting deleted files
 
-The logic for detecting missing files works for multiple scans by the same user as well as by different users. For example, suppose a user runs a one-time scan on a Data Lake Storage Gen2 data store on folders A, B, and C. Later, a different user in the same account runs a different one-time scan on folders C, D, and E of the same data store. Because folder C was scanned twice, the catalog checks it for possible deletions. Folders A, B, D, and E, however, were scanned only once, and the catalog won't check them for deleted assets.
+The logic for detecting missing files works for multiple scans by the same user and by different users. For example, suppose a user runs a one-time scan on a Data Lake Storage Gen2 data store on folders A, B, and C. Later, a different user in the same account runs a different one-time scan on folders C, D, and E of the same data store. Because folder C was scanned twice, the catalog checks it for possible deletions. Folders A, B, D, and E, however, were scanned only once, and the catalog won't check them for deleted assets.
 
 To keep deleted files out of your catalog, it's important to run regular scans. The scan interval is important, because the catalog can't detect deleted assets until another scan is run. So, if you run scans once a month on a particular store, the catalog can't detect any deleted data assets in that store until you run the next scan a month later.
 
@@ -67,7 +72,19 @@ When you enumerate large data stores like Data Lake Storage Gen2, there are mult
 
 ## Ingestion
 
-The technical metadata or classifications identified by the scanning process are then sent to Ingestion. The ingestion process is responsible for populating the data map and is managed by Microsoft Purview.  Ingestion analyses the input from scan, [applies resource set patterns](concept-resource-sets.md#how-microsoft-purview-detects-resource-sets), populates available [lineage](concept-data-lineage.md) information, and then loads the data map automatically. Assets/schemas can be discovered or curated only after ingestion is complete. So, if your scan is completed but you haven't seen your assets in the data map or catalog, you'll need to wait for the ingestion process to finish.
+Ingestion is the process responsible for populating the data map with metadata gathered through its various processes.
+
+## Ingestion from scans
+
+The technical metadata or classifications identified by the scanning process are then sent to ingestion. Ingestion analyses the input from scan, [applies resource set patterns](concept-resource-sets.md#how-microsoft-purview-detects-resource-sets), populates available [lineage](concept-data-lineage.md) information, and then loads the data map automatically. Assets/schemas can be discovered or curated only after ingestion is complete. So, if your scan is completed but you haven't seen your assets in the data map or catalog, you'll need to wait for the ingestion process to finish.
+
+## Ingestion from lineage connections
+
+Resources like [Azure Data Factory](how-to-link-azure-data-factory.md) and [Azure Synapse](how-to-lineage-azure-synapse-analytics.md) can be connected to Microsoft Purview to bring data source and lineage information into your Microsoft Purview Data Map. For example, when a copy pipeline runs in an Azure Data Factory that has been connected to Microsoft Purview, metadata about the input sources, the activity, and the output sources are ingested in Microsoft Purview and the information is added to the data map.
+
+If a data source has already been added to the data map through a scan, lineage information about the activity will be added to the existing source. If the data source hasn't yet been added to the data map, the lineage ingestion process will add it to the root collection with its lineage information.
+
+For more information about the available lineage connections, see the [lineage user guide](catalog-lineage-user-guide.md).
 
 ## Next steps
 

articles/purview/how-to-lineage-azure-synapse-analytics.md

@@ -6,11 +6,17 @@ ms.author: jingwang
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 12/14/2022
+ms.date: 03/13/2023
 ---
 # How to get lineage from Azure Synapse Analytics into Microsoft Purview
 
-This document explains the steps required for connecting an Azure Synapse workspace with a Microsoft Purview account to track data lineage. The document also gets into the details of the coverage scope and supported lineage capabilities.
+This document explains the steps required for connecting an Azure Synapse workspace with a Microsoft Purview account to track [data lineage](concept-data-lineage.md) and [ingest data sources](concept-scans-and-ingestion.md#ingestion). The document also gets into the details of the activity coverage scope and supported lineage capabilities.
+
+When you connect Azure Synapse Analytics to Microsoft Purview, whenever a [supported pipeline activity](#supported-azure-synapse-capabilities) is run, metadata about the activity's source data, output data, and the activity will be automatically [ingested](concept-scans-and-ingestion.md#ingestion) into the Microsoft Purview Data Map.
+
+If a data source has already been scanned and exists in the data map, the ingestion process will add the lineage information from Azure Synapse Analytics to that existing source. If the source or output doesn't exist in the data map and is [supported by Azure Synapse Analytics lineage](#supported-azure-synapse-capabilities) Microsoft Purview will automatically add their metadata from Synapse Analytics into the data map under the root collection.
+
+This can be an excellent way to monitor your data estate as users move and transform information using Azure Synapse Analytics.
 
 ## Supported Azure Synapse capabilities
 
@@ -25,7 +31,7 @@ Currently, Microsoft Purview captures runtime lineage from the following Azure S
 [!INCLUDE[azure-synapse-supported-activity-lineage-capabilities](includes/data-factory-common-supported-capabilities.md)]
 
 ## Access secured Microsoft Purview account
-      
+
 If your Microsoft Purview account is protected by firewall, learn how to let Azure Synapse [access a secured Microsoft Purview account](../synapse-analytics/catalog-and-governance/how-to-access-secured-purview-account.md) through Microsoft Purview private endpoints.
 
 ## Bring Azure Synapse lineage into Microsoft Purview

articles/purview/how-to-link-azure-data-factory.md

@@ -6,11 +6,17 @@ ms.author: jingwang
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 12/14/2022
+ms.date: 03/13/2023
 ---
 # How to connect Azure Data Factory and Microsoft Purview
 
-This document explains the steps required for connecting an Azure Data Factory account with a Microsoft Purview account to track data lineage. The document also gets into the details of the coverage scope and supported lineage patterns.
+This document explains the steps required for connecting an Azure Data Factory account with a Microsoft Purview account to track [data lineage](concept-data-lineage.md) and [ingest data sources](concept-scans-and-ingestion.md#ingestion). The document also gets into the details of the activity coverage scope and supported lineage patterns.
+
+When you connect an Azure Data Factory to Microsoft Purview, whenever a [supported Azure Data Factory activity](#supported-azure-data-factory-activities) is run, metadata about the activity's source data, output data, and the activity will be automatically [ingested](concept-scans-and-ingestion.md#ingestion) into the Microsoft Purview Data Map.
+
+If a data source has already been scanned and exists in the data map, the ingestion process will add the lineage information from Azure Data Factory to that existing source. If the source or output doesn't exist in the data map and is [supported by Azure Data Factory lineage](#supported-azure-data-factory-activities) Microsoft Purview will automatically add their metadata from Azure Data Factory into the data map under the root collection.
+
+This can be an excellent way to monitor your data estate as users move and transform information using Azure Data Factory.
 
 ## View existing Data Factory connections
 
@@ -98,7 +104,7 @@ The integration between Data Factory and Microsoft Purview supports only a subse
 Refer to [supported data stores](how-to-lineage-sql-server-integration-services.md#supported-data-stores).
 
 ## Access secured Microsoft Purview account
-      
+
 If your Microsoft Purview account is protected by firewall, learn how to let Data Factory [access a secured Microsoft Purview account](../data-factory/how-to-access-secured-purview-account.md) through Microsoft Purview private endpoints.
 
 ## Bring Data Factory lineage into Microsoft Purview