You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/purview/catalog-lineage-user-guide.md
+36-3Lines changed: 36 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,8 @@ This article provides an overview of the data lineage features in Microsoft Purv
15
15
16
16
One of the platform features of Microsoft Purview is the ability to show the lineage between datasets created by data processes. Systems like Data Factory, Data Share, and Power BI capture the lineage of data as it moves. Custom lineage reporting is also supported via Atlas hooks and REST API.
17
17
18
-
## Lineage collection
18
+
## Lineage collection
19
+
19
20
Metadata collected in Microsoft Purview from enterprise data systems are stitched across to show an end to end data lineage. Data systems that collect lineage into Microsoft Purview are broadly categorized into following three types:
@@ -26,7 +27,7 @@ Each system supports a different level of lineage scope. Check the sections belo
26
27
27
28
### Known limitations
28
29
29
-
* Database Views used as source of process activity(Azure Data Factory, Synapse Pipelines, Azure SQL Database, Azure Data Share) are currently captured as Database Table objects in Microsoft Purview. If the Database is also scanned, the View assets are discovered separately in Microsoft Purview. In this scenario, two assets with same name captured in Microsoft Purview, one as a Table with data lineage and another as a View.
30
+
* Database Views used as source of process activity(Azure Data Factory, Synapse Pipelines, Azure SQL Database, Azure Data Share) are currently captured as Database Table objects in Microsoft Purview. If the Database is also scanned, the View assets are discovered separately in Microsoft Purview. In this scenario, two assets with same name captured in Microsoft Purview, one as a Table with data lineage and another as a View.
30
31
31
32
### Data processing systems
32
33
Data integration and ETL tools can push lineage into Microsoft Purview at execution time. Tools such as Data Factory, Data Share, Synapse, Azure Databricks, and so on, belong to this category of data processing systems. The data processing systems reference datasets as source from different databases and storage solutions to create target datasets. The list of data processing systems currently integrated with Microsoft Purview for lineage are listed in below table.
@@ -90,7 +91,39 @@ To access lineage information for an asset in Microsoft Purview, follow the step
90
91
91
92
Microsoft Purview supports asset level lineage for the datasets and processes. To see the asset level lineage go to the **Lineage** tab of the current asset in the catalog. Select the current dataset asset node. By default the list of columns belonging to the data appears in the left pane.
92
93
93
-
:::image type="content" source="./media/catalog-lineage-user-guide/view-columns-from-lineage-inline.png" alt-text="Screenshot showing how to select View columns in the lineage page" lightbox="./media/catalog-lineage-user-guide/view-columns-from-lineage.png"border="true":::
94
+
:::image type="content" source="./media/catalog-lineage-user-guide/view-columns-from-lineage-inline.png" alt-text="Screenshot showing how to select View columns in the lineage page." lightbox="./media/catalog-lineage-user-guide/view-columns-from-lineage.png"border="true":::
95
+
96
+
## Manual lineage (preview)
97
+
98
+
Data lineage in Microsoft Purview is [automated](#lineage-collection) for many assets in on-premises, multicloud, and SaaS environments. While we continue to add more automated sources, manual lineage allows you to document lineage metadata for sources where automation isn't yet supported, without using any code.
99
+
100
+
To add manual lineage for any of your assets, follow these steps:
101
+
102
+
1.[Search for your asset in the data catalog](how-to-search-catalog.md) and select it to view details.
103
+
1. Select **Edit** and navigate to the **Lineage** tab.
104
+
:::image type="content" source="./media/catalog-lineage-user-guide/select-edit.png" alt-text="Screenshot of a data asset in Microsoft Purview, with the edit option highlighted.":::
105
+
:::image type="content" source="./media/catalog-lineage-user-guide/select-lineage.png" alt-text="Screenshot of a data asset edit page, with the Lineage tab highlighted.":::
106
+
1. Select **Add Lineage** in the list panel to add an asset as part of the manual lineage.
107
+
:::image type="content" source="./media/catalog-lineage-user-guide/add-lineage.png" alt-text="Screenshot of a data asset lineage page, with the add lineage button highlighted.":::
108
+
1. Select the relationship type:
109
+
1. For upstream lineage - select the relationship as **Consumes**
110
+
1. For downstream lineage - select the relationship as **Produces**
111
+
1. Select the asset dropdown to find the asset from the suggested list or **View more** to search the full catalog. Select your asset.
112
+
:::image type="content" source="./media/catalog-lineage-user-guide/select-asset-dropdown.png" alt-text="Screenshot of a data asset lineage page, with the asset dropdown highlighted.":::
113
+
1. Now you can see the lineage relationship. You can add another by selecting the **Add Lineage** button again, or delete the newly added lineage by selecting the trash can icon. When you're finished, select the **Save** button to save your lineage and exit edit mode.
114
+
:::image type="content" source="./media/catalog-lineage-user-guide/delete-or-save.png" alt-text="Screenshot of a data asset lineage page, the delete and save buttons highlighted.":::
115
+
116
+
### Known limitations of manual lineage
117
+
118
+
* Current asset picker experience allows selecting only one asset at a time.
119
+
* Column level manual lineage is currently not supported.
120
+
* Data curation access required for both source and target assets.
121
+
* These asset types don't currently allow manual lineage because they support automated lineage:
0 commit comments