Merge pull request #239418 from whhender/patch-110

Asset normalization limitations

Author: v-regandowner

articles/purview/catalog-asset-details.md

@@ -1,31 +1,63 @@
 ---
-title: Asset details page in the Microsoft Purview Data Catalog
-description: View relevant information and take action on assets in the data catalog
+title: Asset management in the Microsoft Purview Data Catalog
+description: View relevant information and take action on assets in the Microsoft Purview Data Catalog.
 author: nayenama
 ms.author: nayenama
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: how-to
-ms.date: 07/25/2022
+ms.date: 05/26/2023
 ---
-# Asset details page in the Microsoft Purview Data Catalog
+# Asset management in the Microsoft Purview Data Catalog
 
-This article discusses how assets are displayed in the Microsoft Purview Data Catalog, and all the features and details available to them. It describes how you can view relevant information or take  action on assets in your catalog.
+This article discusses how assets are displayed in the Microsoft Purview Data Catalog, and all the features and details available to them. It describes how you can view relevant information or take action on assets in your catalog.
 
 ## Prerequisites
 
 - Set up your data sources and scan the assets into your catalog.
 - *Or* Use the Microsoft Purview Atlas APIs to ingest assets into the catalog.
 
-## Open an asset details page
+## Discover assets
 
 You can discover your assets in the Microsoft Purview Data Catalog by either:
 - [Browsing the data catalog](how-to-browse-catalog.md)
 - [Searching the data catalog](how-to-search-catalog.md)
 
-Once you find the asset you're looking for, you can view all of the asset information or take action on them as described in following sections.
+Once you've discovered an asset, select it to view the asset details page, which contains all the asset details, and more actions you can take.
 
-## Asset details tabs explained
+## Editing assets
+
+To edit an asset, other than adding a rating, you will need the [data curator](catalog-permissions.md) role on the collection where the asset is housed.
+
+To edit assets you can either:
+
+- Select multiple assets in the catalog to [bulk edit assets](how-to-bulk-edit-assets.md)
+- Select a single asset and select the **Edit** button at the top of the [asset details page](#asset-details-page)
+
+#### Scan behavior after editing assets
+
+ Microsoft Purview works to reflect the truth of the source system whenever possible. For example, if you edit a column and later it's deleted from the source table. A scan will remove the column metadata from the asset in Microsoft Purview.
+
+Both column-level and asset-level updates such as adding a description, glossary term or classification don't impact scan updates. Scans will update new columns and classifications regardless if these changes are made.
+
+If you update the **name** or **data type** of a column, subsequent scans **won't** update the asset schema. New columns and classifications **won't** be detected.
+
+## Delete asset
+
+If you're a data curator on the collection containing an asset, you can delete an asset by selecting the delete icon under the name of the asset.
+
+> [!IMPORTANT]
+> You cannot delete an asset that has child assets.
+> 
+> Currently, Microsoft Purview doesn't support cascaded deletes. For example, if you attempt to delete a storage account asset in your catalog the containers, folders and files within them will still exist in the data map and the the storage account asset will still exist in relation to them.
+
+Any asset you delete using the delete button is permanently deleted in Microsoft Purview. However, if you run a **full scan** on the source from which the asset was ingested into the catalog, then the asset is reingested and you can discover it using the Microsoft Purview Data Catalog.
+
+If you have a scheduled scan (weekly or monthly) on the source, the **deleted asset won't get re-ingested** into the catalog unless the asset is modified by an end user since the previous run of the scan. For example, say you manually delete a SQL table from the Microsoft Purview Data Map. Later, a data engineer adds a new column to the source table. When Microsoft Purview scans the database, the table will be reingested into the data map and be discoverable in the data catalog.
+
+## Asset details page
+
+At the top of the asset details page there are several tabs:
 
 :::image type="content" source="media/catalog-asset-details/asset-tabs.png" alt-text="Asset details tabs":::
 
@@ -99,24 +131,6 @@ Below are a list of actions you can take from an asset details page. Actions ava
 
 :::image type="content" source="media/catalog-asset-details/asset-details-actions.png" alt-text="Screenshot that shows actions available on the asset details page.":::
 
-### Editing assets
-
-If you're a data curator on the collection containing an asset, you can edit an asset by selecting the edit icon on the top-left corner of the asset.
-
-At the asset level you can edit or add a description, classification, or glossary term by staying on the overview tab of the edit screen.
-
-You can navigate to the schema tab on the edit screen to update column name, data type, column level classification, terms, or asset description.
-
-You can navigate to the contact tab of the edit screen to update owners and experts on the asset. You can search by full name, email or alias of the person within your Azure active directory.
-
-#### Scan behavior after editing assets
-
- Microsoft Purview works to reflect the truth of the source system whenever possible. For example, if you edit a column and later it's deleted from the source table. A scan will remove the column metadata from the asset in Microsoft Purview.
-
-Both column-level and asset-level updates such as adding a description, glossary term or classification don't impact scan updates. Scans will update new columns and classifications regardless if these changes are made.
-
-If you update the **name** or **data type** of a column, subsequent scans **won't** update the asset schema. New columns and classifications **won't** be detected.
-
 ### Request access to data
 
 If a [self-service data access workflow](how-to-workflow-self-service-data-access-hybrid.md) has been created, you can request access to a desired asset directly from the asset details page! To learn more about Microsoft Purview's data policy applications, see [how to enable data use management](how-to-enable-data-use-management.md).
@@ -138,19 +152,6 @@ Microsoft Purview makes it easy to work with useful data you find the data catal
 - SQL Server
 - Teradata
 
-### Deleting assets
-
-If you're a data curator on the collection containing an asset, you can delete an asset by selecting the delete icon under the name of the asset.
-
-> [!IMPORTANT]
-> You cannot delete an asset that has child assets.
-> 
-> Currently, Microsoft Purview doesn't support cascaded deletes. For example, if you attempt to delete a storage account asset in your catalog the containers, folders and files within them will still exist in the data map and the the storage account asset will still exist in relation to them.
-
-Any asset you delete using the delete button is permanently deleted in Microsoft Purview. However, if you run a **full scan** on the source from which the asset was ingested into the catalog, then the asset is reingested and you can discover it using the Microsoft Purview catalog.
-
-If you have a scheduled scan (weekly or monthly) on the source, the **deleted asset won't get re-ingested** into the catalog unless the asset is modified by an end user since the previous run of the scan. For example, say you manually delete a SQL table from the Microsoft Purview Data Map. Later, a data engineer adds a new column to the source table. When Microsoft Purview scans the database, the table will be reingested into the data map and be discoverable in the data catalog.
-
 ## Ratings
 
 Assets can be rated by all users with read access, or better, to that asset in Microsoft Purview.
@@ -176,7 +177,7 @@ These ratings can be seen by all users with read access, and rating can be [adde
 1. Choose a star rating, add a comment, and select **Submit**.
     :::image type="content" source="media/catalog-asset-details/rate-asset.png" alt-text="Screenshot of a rating, showing five start selected and a comment about the quality of the data.":::
 
-## Edit or delete your rating
+### Edit or delete your rating
 
 1. Select the ratings button in the asset's header.
 1. Select the **Open ratings** button.
@@ -215,7 +216,14 @@ If you have [data curator](catalog-permissions.md) permissions Microsoft Purview
     :::image type="content" source="media/catalog-asset-details/remove-tag.png" alt-text="Screenshot that shows the remove tag button highlighted next to an existing page.":::
 1. Confirm the removal of the tag.
 
+## Duplicate assets
+
+If you notice duplicate assets in your Microsoft Purview Data Catalog, review the [asset normalization](concept-asset-normalization.md) documentation. Microsoft Purview normalizes assets to prevent duplication, but not all possible scenarios are covered.
+
+Compare the fully qualified asset names for duplicate assets, and update ingestion points to resolve capitalization or character differences. Then, [delete the duplicated asset](#delete-asset) in your catalog.
+
 ## Next steps
 
 - [Browse the Microsoft Purview Data Catalog](how-to-browse-catalog.md)
 - [Search the Microsoft Purview Data Catalog](how-to-search-catalog.md)
+- [Asset normalization](concept-asset-normalization.md)

articles/purview/concept-asset-normalization.md

@@ -1,21 +1,24 @@
 ---
 title: Asset normalization
-description: Learn how Microsoft Purview prevents duplicate assets in your data map through asset normalization
+description: Learn how Microsoft Purview prevents duplicating assets in your data map through asset normalization.
 author: nayenama
 ms.author: nayenama
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: conceptual
-ms.date: 02/17/2023
+ms.date: 05/26/2023
 ms.custom: ignite-fall-2021
 ---
 
 # Asset normalization
 
-When ingesting assets into the Microsoft Purview data map, different sources updating the same data asset may send similar, but slightly different qualified names. While these qualified names represent the same asset, slight differences such as an extra character or different capitalization may cause these assets on the surface to appear different. To avoid storing duplicate entries and causing confusion when consuming the data catalog, Microsoft Purview applies normalization during ingestion to ensure all fully qualified names of the same entity type are in the same format.
+When ingesting assets into the Microsoft Purview data map, different sources updating the same data asset may send similar, but slightly different qualified names. While these qualified names represent the same asset, slight differences such as an extra character may cause these assets on the surface to appear different and cause duplicate entries in Microsoft Purview. To avoid storing duplicate entries and causing confusion when consuming the data catalog, Microsoft Purview applies normalization during ingestion to ensure all fully qualified names of the same entity type are in the same format.
 
 For example, you scan in an Azure Blob with the qualified name `https://myaccount.file.core.windows.net/myshare/folderA/folderB/my-file.parquet`. This blob is also consumed by an Azure Data Factory pipeline that will then add lineage information to the asset. The ADF pipeline may be configured to read the file as `https://myAccount.file.core.windows.net//myshare/folderA/folderB/my-file.parquet`. While the qualified name is different, this ADF pipeline is consuming the same piece of data. Normalization ensures that all the metadata from both Azure Blob Storage and Azure Data Factory is visible on a single asset, `https://myaccount.file.core.windows.net/myshare/folderA/folderB/my-file.parquet`.
 
+>[!IMPORTANT]
+>The rules listed below are the only kinds of potential dupilcation Microsoft Purview currently recognizes. If you are experiencing accidental asset duplication, compare the assets fully qualified names to check for caplitalization differences or additional characters. Update any ingestion points, for example your ADF pipelines, so that the qualified names match.
+
 ## Normalization rules
 
 Below are the normalization rules applied by Microsoft Purview.

articles/purview/concept-best-practices-asset-lifecycle.md

@@ -6,12 +6,12 @@ ms.author: jubairpatel
 ms.service: purview
 ms.subservice: purview-data-catalog
 ms.topic: conceptual
-ms.date: 01/06/2022
+ms.date: 05/26/2023
 ---
 
 # Business processes for managing data effectively
 
-As data and content has a lifecycle that requires active management (for example, acquisition - processing - disposal) assets in the Microsoft Purview data catalog need active management in a similar way. "Assets" in the catalog include the technical metadata that describes collection, lineage and scan information. Metadata describing the business structure of data such as glossary, classifications and ownership also needs to be managed. 
+As data and content has a lifecycle that requires active management (for example, acquisition - processing - disposal) assets in the Microsoft Purview Data Catalog need active management in a similar way. "Assets" in the catalog include the technical metadata that describes collection, lineage and scan information. Metadata describing the business structure of data such as glossary, classifications and ownership also needs to be managed. 
 
 To manage data assets, responsible people in the organization must understand how and when to apply data governance processes and manage workflows. 
 
@@ -21,7 +21,7 @@ An organization employing [Microsoft Purview data governance solutions](/purview
 
 ### Benefits
 
-- Agreed definition and structure of data is required for the Microsoft Purview data catalog to provide effective data search and protection functionality at scale across organizations' data estates. 
+- Agreed definition and structure of data is required for the Microsoft Purview Data Catalog to provide effective data search and protection functionality at scale across organizations' data estates. 
 
 - Defining and using processes for asset lifecycle management is key to maintaining accurate asset metadata, which will improve usability of the catalog and the ability to protect relevant data. 
 
@@ -54,7 +54,7 @@ The [Data Curator](catalog-permissions.md) role in Microsoft Purview controls re
 
 ## 1. Capture and maintain assets
 
-This process describes the high-level steps and suggested roles to capture and maintain assets in the Microsoft Purview data catalog.
+This process describes the high-level steps and suggested roles to capture and maintain assets in the Microsoft Purview Data Catalog.
 
 :::image type="content" source="media/concept-best-practices/assets-capturing-asset-metadata.png" alt-text="Business Process 1 - Capturing and Maintaining Assets."lightbox="media/concept-best-practices/assets-capturing-asset-metadata.png" border="true":::
 
@@ -66,13 +66,13 @@ This process describes the high-level steps and suggested roles to capture and m
 | 2 | [How to create and manage collections](how-to-create-and-manage-collections.md)
 | 3 & 4 | [Understand Microsoft Purview access and permissions](catalog-permissions.md)
 | 5 | [Microsoft Purview supported sources](purview-connector-overview.md) <br> [Microsoft Purview private endpoint networking](catalog-private-link.md) |
-| 6 | [How to manage multi-cloud data sources](manage-data-sources.md)
+| 6 | [How to manage multicloud data sources](manage-data-sources.md)
 | 7 | [Best practices for scanning data sources in Microsoft Purview](concept-best-practices-scanning.md)
 | 8, 9 & 10 | [Search the data catalog](how-to-search-catalog.md)  <br>   [Browse the data catalog](how-to-browse-catalog.md)
 
 ## 2. Glossary and classification maintenance
 
-This process describes the high-level steps and roles to manage and define the business glossary and classifications metadata to enrich the Microsoft Purview data catalog. 
+This process describes the high-level steps and roles to manage and define the business glossary and classifications metadata to enrich the Microsoft Purview Data Catalog. 
 
 :::image type="content" source="media/concept-best-practices/assets-maintaining-glossary-and-classifications.png" alt-text="Business Process 2 - Maintaining glossary and classifications"lightbox="media/concept-best-practices/assets-maintaining-glossary-and-classifications.png" border="true":::
 
@@ -94,7 +94,7 @@ This process describes the high-level steps and roles to manage and define the b
 
 ## 3. Moving assets between collections
 
-This process describes the high-level steps and roles to move assets between collections using the Microsoft Purview portal. 
+This process describes the high-level steps and roles to move assets between collections using the Microsoft Purview compliance portal. 
 
 :::image type="content" source="media/concept-best-practices/assets-moving-assets-between-collections.png" alt-text="Business Process 3 - Moving assets between collections"lightbox="media/concept-best-practices/assets-moving-assets-between-collections.png" border="true":::
 
@@ -110,11 +110,11 @@ This process describes the high-level steps and roles to move assets between col
 | 7 | [Browse the Microsoft Purview Catalog](how-to-browse-catalog.md) 
 
 > [!Note]
-> It is not currently possible to bulk move assets from one collection to another using the Microsoft Purview portal. 
+> It is not currently possible to bulk move assets from one collection to another using the Microsoft Purview compliance portal. 
 
 ## 4. Deleting asset metadata
 
-This process describes the high-level steps and roles to delete asset metadata from the data catalog using the Microsoft Purview portal. 
+This process describes the high-level steps and roles to delete asset metadata from the data catalog using the Microsoft Purview compliance portal. 
 
 Asset Metadata may need to be deleted manually for many reasons: 
 
@@ -124,7 +124,7 @@ Asset Metadata may need to be deleted manually for many reasons:
 
 
 > [!Note]
-> Before deleting assets, please refer to the how-to guide to review considerations: [How to delete assets](catalog-asset-details.md#deleting-assets)
+> Before deleting assets, please refer to the how-to guide to review considerations: [How to delete assets](catalog-asset-details.md#delete-asset)
 
 :::image type="content" source="media/concept-best-practices/assets-deleting-asset-metadata.png" alt-text="Business Process 4 - Deleting Assets in Microsoft Purview"lightbox="media/concept-best-practices/assets-deleting-asset-metadata.png" border="true":::
 
@@ -135,7 +135,7 @@ Asset Metadata may need to be deleted manually for many reasons:
 | 1 & 2 | Manual steps |  
 | 3 | [Data catalog lineage user guide](catalog-lineage-user-guide.md)
 | 4 | Manual step
-| 5 | [How to view, edit and delete assets](catalog-asset-details.md#deleting-assets) 
+| 5 | [How to view, edit and delete assets](catalog-asset-details.md#delete-asset) 
 | 6 | [Scanning best practices](concept-best-practices-scanning.md)
 
 > [!Note]