Skip to content

Commit 5c51177

Browse files
author
Preeti Singh
committed
remving postman ref
1 parent 2f89bd6 commit 5c51177

File tree

1 file changed

+45
-40
lines changed

1 file changed

+45
-40
lines changed

articles/energy-data-services/how-to-convert-segy-to-ovds.md

Lines changed: 45 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,13 @@ ms.custom: template-concept
1111

1212
# How to convert a SEG-Y file to oVDS
1313

14-
In this article, you learn how to convert SEG-Y formatted data to the Open VDS (oVDS) format. Seismic data stored in the industry standard SEG-Y format can be converted to oVDS format for use in applications via the Seismic DMS. See here for OSDU® community here: [SEG-Y to oVDS conversation](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/tree/master). This tutorial is a step by step guideline how to perform the conversion. Note the actual production workflow may differ and use as a guide for the required set of steps to achieve the conversion.
14+
In this article, you will learn how to convert SEG-Y formatted data to the Open VDS (oVDS) format. Seismic data stored in the industry standard SEG-Y format can be converted to oVDS format for use in applications via the Seismic DDMS. See here for OSDU® community reference: [SEG-Y to oVDS conversation](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/tree/master). This tutorial is a step by step guideline on how to perform the conversion. Note the actual production workflow may differ and use it as a guide for the required set of steps to achieve the conversion.
1515

1616
## Prerequisites
1717
* An Azure subscription
1818
* An instance of [Azure Data Manager for Energy](quickstart-create-microsoft-energy-data-services-instance.md) created in your Azure subscription
1919
* cURL command-line tool installed on your machine
20-
* Generate the service principal access token to call the Seismic APIs. See [How to generate auth token](how-to-generate-auth-token.md).
20+
* Generate the service principal access_token to call the Seismic APIs. See [How to generate auth token](how-to-generate-auth-token.md).
2121
- A SEG-Y File
2222
- You may use any of the following files from the Volve dataset as a test. The Volve data set itself is available from [Equinor](https://www.equinor.com/energy/volve-data-sharing).
2323
- [Small < 100 MB](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/azure/m16-master/source/ddms-smoke-tests/ST0202R08_PSDM_DELTA_FIELD_DEPTH.MIG_FIN.POST_STACK.3D.JS-017534.segy)
@@ -32,7 +32,7 @@ In this article, you learn how to convert SEG-Y formatted data to the Open VDS (
3232
|----|----|----|----|
3333
| `DNS` | URI | `<instance>.energy.azure.com` | Find this value on the overview page of the Azure Data Manager for Energy instance. |
3434
| `data-partition-id` | Data partitions | `<data-partition-id>` | Find this value on the Data Partitions section within the Azure Data Manager for Energy instance. |
35-
| `access_token` | Access token value | `0.ATcA01-XWHdJ0ES-qDevC6r...........`| Follow [How to generate auth token](how-to-generate-auth-token.md) to create an access token and save it.|
35+
| `access_token` | access_token value | `0.ATcA01-XWHdJ0ES-qDevC6r...........`| Follow [How to generate auth token](how-to-generate-auth-token.md) to create an access_token and save it.|
3636

3737
Follow the [Manage users](how-to-manage-users.md) guide to add appropriate entitlements for the user who's running this tutorial.
3838

@@ -49,8 +49,8 @@ Create a legal tag that is automatically added to the Seismic DDMS environment f
4949
API: **Setup** > **Create Legal Tag for SDMS**
5050

5151
```bash
52-
curl --request POST \
53-
--url https://{base_url}/api/legal/v1/legaltags \
52+
cURL --request POST \
53+
--url https://{DNS}/api/legal/v1/legaltags \
5454
--header 'Authorization: Bearer {access_token}' \
5555
--header 'Content-Type: application/json' \
5656
--header 'Data-Partition-Id: {data_partition_id}' \
@@ -102,10 +102,10 @@ This file contains the sample [Vector Header Mapping](https://github.com/microso
102102

103103
### Validate User Access
104104

105-
Use the following `curl` command to get user groups:
105+
Use the following `cURL` command to get user groups:
106106

107107
```bash
108-
curl -X GET "https://<DNS>/api/entitlements/v2/groups" \
108+
cURL -X GET "https://<DNS>/api/entitlements/v2/groups" \
109109
-H "Authorization: Bearer <access_token>" \
110110
-H "Content-Type: application/json" \
111111
-H "data-partition-id: <data_partition_id>"
@@ -124,10 +124,10 @@ curl -X GET "https://<DNS>/api/entitlements/v2/groups" \
124124

125125
### Add User to Admin Group
126126

127-
Use the following `curl` command to add a user to the admin group:
127+
Use the following `cURL` command to add a user to the admin group:
128128

129129
```bash
130-
curl -X POST "https://<DNS>/api/entitlements/v2/groups/users.datalake.admins@<data_partition_id>.<domain>/members" \
130+
cURL -X POST "https://<DNS>/api/entitlements/v2/groups/users.datalake.admins@<data_partition_id>.<domain>/members" \
131131
-H "Authorization: Bearer <access_token>" \
132132
-H "Content-Type: application/json" \
133133
-H "data-partition-id: <data_partition_id>"
@@ -148,14 +148,15 @@ curl -X POST "https://<DNS>/api/entitlements/v2/groups/users.datalake.admins@<da
148148
If you haven't yet created entitlements groups, follow the directions as outlined in [How to manage users](how-to-manage-users.md). If you would like to see what groups you have, use [Get entitlements groups for a given user](how-to-manage-users.md#get-osdu-groups-for-a-given-user-in-a-data-partition). Data access isolation is achieved with this dedicated ACL (access control list) per object within a given data partition.
149149

150150
Follow this [tutorial](tutorial-seismic-ddms.md) to Prepare Subproject which involves following steps:
151-
Register Data Partition to Seismic
152-
Create Subproject
153-
Create dataset
151+
152+
1. Register Data Partition to Seismic -Create a tenant
153+
2. Create a Subproject
154+
3. Register a Dataset
154155

155156
### Upload the File
156157

157-
There are two ways to upload a SEGY file. One option is to use the sasurl through curl call. You need to setup Curl on your OS.
158-
The second method is to use [SDUTIL](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/tags/azure-stable). To log in to your instance for ADME via the tool you need to generate a refresh token for the instance. See [How to generate auth token](how-to-generate-auth-token.md). Alternatively, you can modify the code of SDUTIL to use client credentials instead to log in. If you haven't already, you need to setup SDUTIL. Download the codebase and edit the `config.yaml` at the root. Replace the contents of this config file with the following yaml.
158+
There are two ways to upload a SEGY file. One option is to use the sasurl through cURL call. You need to setup cURL on your OS.
159+
The second method is to use [SDUTIL](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/tags/azure-stable). To log in to your instance for ADME via the tool you need to generate a refresh token for the instance. See [How to generate auth token](how-to-generate-auth-token.md). Alternatively, you can modify the code of SDUTIL to use client credentials instead to log in. If you haven't already, you need to setup SDUTIL.Check this guide for setting up [SDUTIL](tutorial-seismic-ddms-sdutil.md) Download the codebase and edit the `config.yaml` at the root. Replace the contents of this config file with the following yaml.
159160

160161
```yaml
161162
seistore:
@@ -178,12 +179,12 @@ azure:
178179
empty: none
179180
```
180181
181-
#### Method 1: curl
182+
#### Method 1: cURL
182183
183184
##### Get gcs url:
184-
curl -X GET \
185+
cURL -X GET \
185186
-H "content-type: application/json" \
186-
-H "Authorization: Bearer <access>" \
187+
-H "Authorization: Bearer <access_token>" \
187188
"https://<DNS>/seistore-svc/api/v3/dataset/tenant/<datapartition>/subproject/<vdssubprojectname>/dataset/<datasetname>"
188189
189190
@@ -193,13 +194,13 @@ Should be a string. we call it gcsstring.
193194
194195
##### Get the sasurl:
195196
196-
Use the following `curl` command to get a SAS upload URL:
197+
Use the following `cURL` command to get a SAS upload URL:
197198

198199
```bash
199-
curl -X 'GET' \
200+
cURL -X 'GET' \
200201
'https://<DNS>/seistore-svc/api/v3/utility/upload-connection-string?sdpath=sd://<tenant>/<vdssubprojectname>' \
201202
-H 'accept: application/json' \
202-
-H 'Authorization: Bearer <access token>'
203+
-H 'Authorization: Bearer <access_token>'
203204
```
204205

205206
**Sample Response:**
@@ -218,12 +219,12 @@ container=$(echo "$filepath" | cut -d'/' -f1)
218219
sasurl=$(echo "<sas token>" | sed "s|$container|$filepath|")
219220
```
220221

221-
##### Upload the file:
222+
##### Upload the SEG-Y file:
222223

223-
Use the following `curl` command:
224+
Use the following `cURL` command:
224225

225226
```bash
226-
curl -X PUT -T "<local_file_path>" "<sas_url>" \
227+
cURL -X PUT -T "<local_file_path>" "<sas_url>" \
227228
-H "x-ms-blob-type: BlockBlob"
228229
```
229230

@@ -237,19 +238,19 @@ curl -X PUT -T "<local_file_path>" "<sas_url>" \
237238

238239
##### Verify upload
239240

240-
Use the following `curl` command to verify file upload:
241+
Use the following `cURL` command to verify file upload:
241242

242243
```bash
243-
curl -X 'GET' \
244-
'https://<DNS>/seistore-svc/api/v3/utility/ls?sdpath=sd%3A%2F%2F<tenant>%2F<vdssubprojectname>' \
244+
cURL -X 'GET' \
245+
'https://<DNS>/seistore-svc/api/v3/utility/ls?sdpath=sd://<tenant>/<vdssubprojectname>' \
245246
-H 'accept: application/json' \
246-
-H 'Authorization: Bearer <access token>'
247+
-H 'Authorization: Bearer <access_token>'
247248
```
248249

249250
**Sample Response:**
250251
```json
251252
[
252-
list of file.
253+
list of files.
253254
]
254255
```
255256

@@ -279,10 +280,12 @@ python sdutil cp ST10010ZC11_PZ_PSDM_KIRCH_FULL_T.MIG_FIN.POST_STACK.3D.JS-01753
279280

280281
### Create Header Vector Mapping
281282

282-
Use the following `curl` command:
283+
This file contains the sample [Vector Header Mapping](https://github.com/microsoft/adme-samples/blob/main/postman/CreateVectorHeaderMappingKeys_SEGYtoVDS.json). You can get the request payload from sample.
284+
285+
Use the following `cURL` command:
283286

284287
```bash
285-
curl -X PUT "https://<DNS>/api/storage/v2/records" \
288+
cURL -X PUT "https://<DNS>/api/storage/v2/records" \
286289
-H "Authorization: Bearer <access_token>" \
287290
-H "Content-Type: application/json" \
288291
-H "data-partition-id: <data_partition_id>" \
@@ -316,10 +319,12 @@ curl -X PUT "https://<DNS>/api/storage/v2/records" \
316319

317320
### Create Storage Records
318321

319-
Use the following `curl` command to create storage records:
322+
This file contains the sample [Storage Records](https://github.com/microsoft/adme-samples/blob/main/postman/StorageRecord_SEGYtoVDS.json) for the VDS conversion.
323+
324+
Use the following `cURL` command to create storage records:
320325

321326
```bash
322-
curl --request PUT \
327+
cURL --request PUT \
323328
--url 'https://<DNS>/api/storage/v2/records' \
324329
--header 'Accept: application/json' \
325330
--header 'Authorization: Bearer {{access_token}}' \
@@ -367,16 +372,16 @@ curl --request PUT \
367372

368373
1. Trigger the VDS Conversion DAG to convert your data using the execution context values you had saved above.
369374

370-
Fetch the ID token from sdutil for the uploaded file or use an access/bearer token from curl.
375+
Fetch the ID token from sdutil for the uploaded file or use an access/bearer token from cURL.
371376

372377
```markdown
373378
python sdutil auth idtoken
374379
```
375380

376-
Use the following `curl` command:
381+
Use the following `cURL` command to trigger workflow:
377382

378383
```bash
379-
curl -X POST "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion" \
384+
cURL -X POST "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion" \
380385
-H "Authorization: Bearer <access_token>" \
381386
-H "Content-Type: application/json" \
382387
-d '{
@@ -405,10 +410,10 @@ curl -X POST "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion" \
405410

406411
2. Let the DAG run to the `succeeded` state. You can check the status using the workflow status call. The run ID is in the response of the above call
407412

408-
Use the following `curl` command:
413+
Use the following `cURL` command:
409414

410415
```bash
411-
curl -X GET "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion/<vds_run_id>" \
416+
cURL -X GET "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion/<vds_run_id>" \
412417
-H "Authorization: Bearer <access_token>" \
413418
-H "Content-Type: application/json"
414419
```
@@ -424,10 +429,10 @@ curl -X GET "https://<DNS>/api/workflow/v1/workflow/segy-to-vds-conversion/<vds_
424429

425430
### Verify File Conversion
426431

427-
Use the following `curl` command to verify file conversion:
432+
Use the following `cURL` command to verify file conversion:
428433

429434
```bash
430-
curl --request GET \
435+
cURL --request GET \
431436
--url 'http://{{seismic_ddms_host}}/utility/ls?sdpath=sd://{{tenant}}/{{vdsTestSubprojectName}}' \
432437
--header 'Authorization: Bearer {{access_token}}'
433438
@@ -440,7 +445,7 @@ curl --request GET \
440445
}
441446
```
442447

443-
3. You can see if the converted file is present using the following command in sdutil or in the curl API call:
448+
3. You can see if the converted file is present using the following command in sdutil or in the cURL API call:
444449

445450
```bash
446451
python sdutil ls sd://<data-partition-id>/vdssubprojectname

0 commit comments

Comments
 (0)