Skip to content

Commit b929416

Browse files
committed
Acrolinx
1 parent 9278a72 commit b929416

File tree

1 file changed

+19
-17
lines changed

1 file changed

+19
-17
lines changed

articles/purview/tutorial-custom-types.md

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,9 @@ author: adinastoll
55
ms.author: adnegrau
66
ms.service: purview
77
ms.topic: how-to
8-
ms.date: 03/08/2023
8+
ms.date: 03/14/2023
99
---
1010

11-
1211
# Type definitions and how to create custom types
1312

1413
This tutorial will explain what type definitions are, how to create custom types, and how to initialize assets of custom types in Microsoft Purview.
@@ -31,15 +30,16 @@ For this tutorial you'll need:
3130
* Apache Atlas endpoint of your Microsoft Purview account. To get your Apache Atlas endpoint, follow the *Apache Atlas endpoint* section from [here](tutorial-atlas-2-2-apis.md#atlas-endpoint).
3231

3332
> [!NOTE]
34-
> Before moving to the hands-on part of the tutorial, the first four sections will explain what System Type is and how it is used in Microsoft Purview.
33+
> Before moving to the hands-on part of the tutorial, the first four sections will explain what a System Type is and how it is used in Microsoft Purview.
3534
> All the REST API calls described further will use the **bearer token** and the **endpoint** which are described in the prerequisites.
3635
>
3736
> To skip directly to the steps, use these links:
3837
>
3938
>* [Create custom type definitions](#create-definitions)
4039
>* [Initialize assets of custom types](#initialize-assets-of-custom-types)
4140
42-
## What is *asset* and *type* in Microsoft Purview
41+
## What are *asset* and *type* in Microsoft Purview?
42+
4343
An *asset* is a metadata element that describes a digital or physical resource. The digital or physical resources that are expected to be cataloged as assets include:
4444

4545
* Data sources such as databases, files, and data feed.
@@ -51,8 +51,8 @@ Microsoft Purview provides users a flexible *type system* to expand the definiti
5151

5252
Essentially, a *Type* can be seen as a *Class* from Object Oriented Programming (OOP):
5353

54-
* It defines the properties that represent that type
55-
* Each type is uniquely identified by its *name*
54+
* It defines the properties that represent that type.
55+
* Each type is uniquely identified by its *name*.
5656
* A *type* can inherit from a *supertType*. This is an equivalent concept as inheritance from OOP. A type that extends a superType will inherit the attributes of the superType.
5757

5858
You can see all type definitions in your Microsoft Purview account by sending a `GET` request to the [All Type Definitions](/rest/api/purview/catalogdataplane/types/get-all-type-definitions) endpoint:
@@ -61,7 +61,7 @@ You can see all type definitions in your Microsoft Purview account by sending a
6161
GET https://{{ENDPOINT}}/catalog/api/atlas/v2/types/typedefs
6262
```
6363

64-
Apache Atlas has few pre-defined system types that are commonly used as supertypes.
64+
Apache Atlas has few predefined system types that are commonly used as supertypes.
6565

6666
For example:
6767

@@ -73,7 +73,7 @@ For example:
7373

7474
* **Lineage**: Lineage information helps one understand the origin of data and the transformations it may have gone through before arriving in a file or table. Lineage is calculated through *DataSet* and *Process*: DataSets (input of process) impact some other DataSets (output of process) through Process.
7575

76-
:::image type="content" source="./media/tutorial-custom-types/base-model-diagram.png" alt-text="Diagram showing the relationships between system types." border="false":::
76+
:::image type="content" source="./media/tutorial-custom-types/base-model-diagram.png" alt-text="Diagram showing the relationships between system types." border="false" lightbox="./media/tutorial-custom-types/base-model-diagram.png":::
7777

7878
## Example of a *Type* definition
7979

@@ -86,7 +86,7 @@ GET https://{{ENDPOINT}}/catalog/api/atlas/v2/types/typedef/name/{name}
8686
```
8787

8888
>[!TIP]
89-
> The **{name}** property tells which defintion you are interested in. In this case, you should use **azure_sql_table**.
89+
> The **{name}** property tells which definition you are interested in. In this case, you should use **azure_sql_table**.
9090
9191
Below you can see a simplified JSON result:
9292

@@ -149,7 +149,7 @@ Based on the JSON type definition, let's look at some properties:
149149

150150
Below you can see an example of how the **Schema** tab looks like for an asset of type Azure SQL Table:
151151

152-
:::image type="content" source="./media/tutorial-custom-types/schema-tab.png" alt-text="Screenshot of the schema tab for an Azure SQL Table asset.":::
152+
:::image type="content" source="./media/tutorial-custom-types/schema-tab.png" alt-text="Screenshot of the schema tab for an Azure SQL Table asset." lightbox="./media/tutorial-custom-types/schema-tab.png":::
153153

154154
* **relationshipAttributeDefs** are calculated through the relationship type definitions. In our JSON, we can see that **schemaElementsAttributes** points to the relationship attribute called **columns** - which is one of elements from **relationshipAttributeDefs** array, as shown below:
155155

@@ -231,8 +231,9 @@ Below you can see a simplified JSON result:
231231

232232
## Schema tab
233233

234-
### What is **Schema** in Microsoft Purview?
235-
Schema is an important concept that reflects how data is stored and organized in the data store. It reflects the structure of the data and the data restrictions of the elements that construct the structure.
234+
### What is *Schema* in Microsoft Purview?
235+
236+
Schema is an important concept that reflects how data is stored and organized in the data store. It reflects the structure of the data and the data restrictions of the elements that construct the structure.
236237

237238
Elements on the same schema can be classified differently (due to their content). Also, different transformation (lineage) can happen to only a subset of elements. Due to these aspects, Purview can model schema and schema elements **as entities**, hence schema is usually a relationship attribute to the data asset entity. Examples of schema elements are: **columns** of a table, **json properties** of json schema, **xml elements** of xml schema etc.
238239

@@ -242,20 +243,21 @@ There are two types of schemas:
242243

243244
For data store with predefined schema, Purview uses the corresponding relationship between the data asset and the schema elements to reflect the schema. This relationship attribute is specified by the keyword **schemaElementsAttribute** in **options** property of the entity type definition.
244245

245-
* **Non Intrinsic Schema** - Some systems don't enforce such schema restrictions, but users can use it to store structural data by applying some schema protocols to the data. For example, Azure Blobs store binary data and do not care about the data in the binary stream. Therefore, it's unaware of any schema, but the user can serialize their data with schema protocols like json before storing it in the blob. In this sense, schema is maintained by some extra protocols and corresponding validation enforced by the user.
246+
* **Non Intrinsic Schema** - Some systems don't enforce such schema restrictions, but users can use it to store structural data by applying some schema protocols to the data. For example, Azure Blobs store binary data and don't care about the data in the binary stream. Therefore, it's unaware of any schema, but the user can serialize their data with schema protocols like json before storing it in the blob. In this sense, schema is maintained by some extra protocols and corresponding validation enforced by the user.
247+
248+
For data store without inherent schema, schema model is independent of this data store. For such cases, Purview defines an interface for schema and a relationship between DataSet and schema, called **dataset_attached_schemas** - this extends any entity type that inherits from DataSet to have an **attachedSchema** relationship attribute to link to their schema representation.
246249

247-
For data store without inherent schema, schema model is independent of this data store. For such cases, Purview defines an interface for schema and a relationship between DataSet and schema, called **dataset_attached_schemas** - this extends any entity type that inherits form DataSet to have an **attachedSchema** relationship attribute to link to their schema representation.
250+
### Example of *Schema tab*
248251

249-
### Example of **Schema tab**
250252
The Azure SQL Table example from above has an intrinsic schema. The information that shows up in the Schema tab of the Azure SQL Table comes from the Azure SQL Column themselves.
251253

252254
Selecting one column item, we would see the following:
253255

254-
:::image type="content" source="./media/tutorial-custom-types/azure-sql-column.png" alt-text="Screenshot of the addressID column page with the properties tab open and the data type highlighted.":::
256+
:::image type="content" source="./media/tutorial-custom-types/azure-sql-column.png" alt-text="Screenshot of the addressID column page with the properties tab open and the data type highlighted." lightbox="./media/tutorial-custom-types/azure-sql-column.png":::
255257

256258
The question is, how did Microsoft Purview select the *data_tye* property from the column and showed it in the Schema tab of the table?
257259

258-
:::image type="content" source="./media/tutorial-custom-types/schema-tab-data-type.png" alt-text="Screenshot of the Azure SQL Table page with the schema page open.":::
260+
:::image type="content" source="./media/tutorial-custom-types/schema-tab-data-type.png" alt-text="Screenshot of the Azure SQL Table page with the schema page open." lightbox="./media/tutorial-custom-types/schema-tab-data-type.png":::
259261

260262
You can get the type definition of an Azure SQL Column by making a `GET` request to the [endpoint](/rest/api/purview/catalogdataplane/types/get-type-definition-by-name):
261263

0 commit comments

Comments
 (0)