You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -20,85 +20,293 @@ In this tutorial, you learn how to:
20
20
21
21
> [!div class="checklist"]
22
22
>
23
-
> * Ingest sample manifests into an Azure Data Manager for Energy instance by using Postman.
24
-
> * Search for storage metadata records created during manifest ingestion by using Postman.
23
+
> * Ingest sample manifests into an Azure Data Manager for Energy instance by using cURL.
24
+
> * Search for storage metadata records created during manifest ingestion by using cURL.
25
25
26
26
## Prerequisites
27
27
28
-
Before you start this tutorial, complete the following prerequisites.
28
+
* An Azure subscription
29
+
* An instance of [Azure Data Manager for Energy](quickstart-create-microsoft-energy-data-services-instance.md) created in your Azure subscription
30
+
* cURL command-line tool installed on your machine
31
+
* Generate the service principal access token to call the Seismic APIs. See [How to generate auth token](how-to-generate-auth-token.md).
29
32
30
33
### Get details for the Azure Data Manager for Energy instance
31
34
32
35
* You need an Azure Data Manager for Energy instance. If you don't already have one, create one by following the steps in [Quickstart: Create an Azure Data Manager for Energy instance](quickstart-create-microsoft-energy-data-services-instance.md).
33
36
* For this tutorial, you need the following parameters:
34
37
35
-
| Parameter | Value to use | Example | Where to find this value |
|`CLIENT_ID`| Application (client) ID |`00001111-aaaa-2222-bbbb-3333cccc4444`| You use this app or client ID when registering the application with the Microsoft identity platform. See [Register an application](../active-directory/develop/quickstart-register-app.md#register-an-application). |
38
-
|`CLIENT_SECRET`| Client secrets |`_fl******************`| Sometimes called an *application password*, a client secret is a string value that your app can use in place of a certificate to identity itself. See [Add a client secret](../active-directory/develop/quickstart-register-app.md#add-a-client-secret).|
39
-
|`TENANT_ID`| Directory (tenant) ID |`72f988bf-86f1-41af-91ab-xxxxxxxxxxxx`| Hover over your account name in the Azure portal to get the directory or tenant ID. Alternately, search for and select **Microsoft Entra ID** > **Properties** > **Tenant ID** in the Azure portal. |
40
-
|`SCOPE`| Application (client) ID |`00001111-aaaa-2222-bbbb-3333cccc4444`| This value is the same as the app or client ID mentioned earlier. |
41
-
|`refresh_token`| Refresh token value |`0.ATcA01-XWHdJ0ES-qDevC6r...........`| Follow [How to generate auth token](how-to-generate-auth-token.md) to create a refresh token and save it. You need this refresh token later to generate a user token. |
42
-
|`DNS`| URI |`<instance>.energy.Azure.com`| Find this value on the overview page of the Azure Data Manager for Energy instance.|
43
-
|`data-partition-id`| Data partitions |`<data-partition-id>`| Find this value on the overview page of the Azure Data Manager for Energy instance.|
38
+
| Parameter | Value to use | Example | Where to find this value |
39
+
|----|----|----|----|
40
+
|`DNS`| URI |`<instance>.energy.azure.com`| Find this value on the overview page of the Azure Data Manager for Energy instance. |
41
+
|`data-partition-id`| Data partitions |`<data-partition-id>`| Find this value on the Data Partitions section within the Azure Data Manager for Energy instance. |
42
+
|`access_token`| Access token value |`0.ATcA01-XWHdJ0ES-qDevC6r...........`| Follow [How to generate auth token](how-to-generate-auth-token.md) to create an access token and save it.|
44
43
45
44
Follow the [Manage users](how-to-manage-users.md) guide to add appropriate entitlements for the user who's running this tutorial.
46
45
47
-
### Set up Postman and execute requests
48
-
49
-
1. Download and install the [Postman](https://www.postman.com/) desktop app.
To import the Postman collection and environment variables, follow the steps in [Importing data into Postman](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#importing-data-into-postman).
57
-
58
-
1. Update **CURRENT VALUE** for the Postman environment with the information that you obtained in the details of the Azure Data Manager for Energy instance.
59
-
60
-
1. The Postman collection for manifest ingestion contains multiple requests that you must execute sequentially.
61
-
62
-
Be sure to choose **Ingestion Workflow Environment** before you trigger the Postman collection.
63
-
64
-
:::image type="content" source="media/tutorial-manifest-ingestion/tutorial-postman-choose-environment.png" alt-text="Screenshot of the Postman environment." lightbox="media/tutorial-manifest-ingestion/tutorial-postman-choose-environment.png":::
65
-
66
-
1. Trigger each request by selecting the **Send** button.
67
-
68
-
On every request, Postman validates the actual API response code against the expected response code. If there's any mismatch, the test section indicates failures.
69
-
70
-
Here's an example of a successful Postman request:
71
-
72
-
:::image type="content" source="media/tutorial-manifest-ingestion/tutorial-postman-test-success.png" alt-text="Screenshot of a successful Postman call." lightbox="media/tutorial-manifest-ingestion/tutorial-postman-test-success.png":::
73
-
74
-
Here's an example of a failed Postman request:
75
-
76
-
:::image type="content" source="media/tutorial-manifest-ingestion/tutorial-postman-test-failure.png" alt-text="Screenshot of a failed Postman call." lightbox="media/tutorial-manifest-ingestion/tutorial-postman-test-failure.png":::
77
-
78
-
## Ingest sample manifests by using Postman
79
-
80
-
To ingest sample manifests into the Azure Data Manager for Energy instance by using the Postman collection, complete the following steps:
81
-
82
-
1.**Get a User Access Token**: Generate the user token, which will be used to authenticate further API calls.
83
-
1.**Create a Legal Tag**: Create a legal tag that will be added to the manifest data for data compliance purposes.
84
-
1.**Get a Signed URL for uploading a file**: Get the signed URL path to which the manifest file will be uploaded.
85
-
1.**Upload a file**: Download the [Wellbore.csv](https://github.com/microsoft/meds-samples/blob/main/test-data/wellbore.csv) sample to your local machine. It could be any file type, including CSV, LAS, or JSON. Select this file in Postman by clicking the **Select File** button.
86
-
87
-
:::image type="content" source="media/tutorial-manifest-ingestion/tutorial-select-manifest-file.png" alt-text="Screenshot of uploading a manifest file." lightbox="media/tutorial-manifest-ingestion/tutorial-select-manifest-file.png":::
88
-
1.**Upload File Metadata**: Upload the file metadata information, such as file location and other relevant fields.
89
-
1.**Get the File Metadata**: Call to validate if the metadata was created successfully.
90
-
1.**Ingest Master, Reference and Work Product Component (WPC) data**: Ingest the master, reference, and work product component (WPC) manifest metadata.
91
-
1.**Get Manifest Ingestion Workflow status**: The workflow starts and is in the **running** state. Keep querying until the state changes to **finished** (typically 20 to 30 seconds).
92
-
93
-
## Search for ingested data by using Postman
94
-
95
-
To search for storage metadata records created during the manifest ingestion by using the Postman collection, complete the following steps:
96
-
97
-
1.**Search Work Products**: Call the search service to retrieve the work product metadata records.
98
-
1.**Search Work Product Components**: Call the search service to retrieve the WPC metadata records.
99
-
1.**Search for Dataset**: Call the search service to retrieve the dataset metadata records.
100
-
1.**Search for Master data**: Call the search service to retrieve the master metadata records.
101
-
1.**Search for Reference Data**: Call the search service to retrieve the reference metadata records.
46
+
### Set up your environment
47
+
48
+
Ensure you have `cURL` installed on your system. You will use it to make API calls.
49
+
50
+
## Ingest sample manifests by using `cURL`
51
+
52
+
To ingest sample manifests into the Azure Data Manager for Energy instance, complete the following steps:
53
+
54
+
1.**Create a Legal Tag**: Use the following `cURL` command to create a legal tag that will be added to the manifest data for data compliance purposes:
55
+
56
+
```bash
57
+
curl -X POST "https://<DNS>/api/legal/v1/legaltags" \
58
+
-H "Authorization: Bearer <access_token>" \
59
+
-H "Content-Type: application/json" \
60
+
-H "data-partition-id: <data-partition-id>" \
61
+
-d '{
62
+
"name": "<tagName>",
63
+
"description": "Legal Tag added for Well",
64
+
"properties": {
65
+
"contractId": "123456",
66
+
"countryOfOrigin": ["US", "CA"],
67
+
"dataType": "Third Party Data",
68
+
"exportClassification": "EAR99",
69
+
"originator": "Schlumberger",
70
+
"personalData": "No Personal Data",
71
+
"securityClassification": "Private",
72
+
"expirationDate": "2025-12-25"
73
+
}
74
+
}'
75
+
```
76
+
77
+
**Sample Response:**
78
+
```json
79
+
{
80
+
"name": "abcd",
81
+
"description": "Legal Tag added for Well",
82
+
"properties": {
83
+
"contractId": "123456",
84
+
"countryOfOrigin": ["US", "CA"],
85
+
"dataType": "Third Party Data",
86
+
"exportClassification": "EAR99",
87
+
"originator": "Schlumberger",
88
+
"personalData": "No Personal Data",
89
+
"securityClassification": "Private",
90
+
"expirationDate": "2025-12-25"
91
+
}
92
+
}
93
+
```
94
+
95
+
2. **Ingest Master, Reference, and Work Product Component (WPC) data**: Use the following `cURL`command to ingest the master, reference, and work product component (WPC) manifest metadata:
96
+
97
+
```bash
98
+
curl -X POST "https://<DNS>/api/workflow/v1/workflow/Osdu_ingest/workflowRun" \
"description": "Reference data for facility types"
115
+
}
116
+
],
117
+
"MasterData": [
118
+
{
119
+
"id": "osdu:wks:master-data--Well:1.0.0",
120
+
"name": "Well",
121
+
"description": "Master data for wells"
122
+
}
123
+
],
124
+
"Data": {
125
+
"kind": "osdu:wks:dataset--File.Generic:1.0.0",
126
+
"name": "Sample Dataset",
127
+
"description": "Dataset for testing purposes"
128
+
}
129
+
}
130
+
}
131
+
}'
132
+
```
133
+
134
+
**Sample Response:**
135
+
```json
136
+
{
137
+
"workflowId": "Osdu_ingest",
138
+
"runId":"5d6c4e37-ab53-4c5e-9c27-49dd77895377",
139
+
"status": "In Progress",
140
+
"message": "Workflow started successfully."
141
+
}
142
+
```
143
+
Save the `runId` from the response foruse as run_idin the next steps.
144
+
145
+
3. **Get Manifest Ingestion Workflow status**: Use the following `cURL`command to check the workflow status (replace `<run_id>` with the workflow run ID):
146
+
147
+
```bash
148
+
curl -X GET "https://<DNS>/api/workflow/v1/workflow/Osdu_ingest/workflowRun/<run_id>" \
149
+
-H "Authorization: Bearer <access_token>" \
150
+
-H "data-partition-id: <data-partition-id>" \
151
+
```
152
+
153
+
**Sample Response:**
154
+
```json
155
+
{
156
+
"workflowId": "Osdu_ingest",
157
+
"runId":"5d6c4e37-ab53-4c5e-9c27-49dd77895377",
158
+
"status": "finished"
159
+
}
160
+
```
161
+
162
+
## Search for ingested data by using `cURL`
163
+
164
+
To search for storage metadata records created during the manifest ingestion, complete the following steps:
165
+
166
+
1. **Search Work Products**: Use the following `cURL`command to retrieve the work product metadata records:
167
+
168
+
```bash
169
+
curl -X POST "https://<DNS>/api/search/v2/query" \
0 commit comments