Skip to content

Commit bc12d40

Browse files
Merge pull request #282108 from jonburchel/2024-07-29-postman-updates
Postman fixes for ADF
2 parents cad5866 + 9c86253 commit bc12d40

File tree

6 files changed

+41
-39
lines changed

6 files changed

+41
-39
lines changed

articles/data-factory/connector-http.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ You can use this HTTP connector to:
4242
- Copy the HTTP response as-is or parse it by using [supported file formats and compression codecs](supported-file-formats-and-compression-codecs.md).
4343

4444
> [!TIP]
45-
> To test an HTTP request for data retrieval before you configure the HTTP connector, learn about the API specification for header and body requirements. You can use tools like Postman or a web browser to validate.
45+
> To test an HTTP request for data retrieval before you configure the HTTP connector, learn about the API specification for header and body requirements. You can use tools like Visual Studio, PowerShell's Invoke-RestMethod, or a web browser to validate.
4646
4747
## Prerequisites
4848

articles/data-factory/connector-odata.md

Lines changed: 17 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -358,21 +358,23 @@ Project Online requires user-based OAuth, which is not supported by Azure Data F
358358
359359
1. Use **Postman** to get the access token:
360360

361-
1. Navigate to **Authorization** tab on the Postman Website.
362-
1. In the **Type** box, select **OAuth 2.0**, and in the **Add authorization data to** box, select **Request Headers**.
363-
1. Fill the following information in the **Configure New Token** page to get a new access token:
364-
- **Grant type**: Select **Authorization Code**.
365-
- **Callback URL**: Enter `https://www.localhost.com/`
366-
- **Auth URL**: Enter `https://login.microsoftonline.com/common/oauth2/authorize?resource=https://<your tenant name>.sharepoint.com`. Replace `<your tenant name>` with your own tenant name.
367-
- **Access Token URL**: Enter `https://login.microsoftonline.com/common/oauth2/token`.
368-
- **Client ID**: Enter your Microsoft Entra service principal ID.
369-
- **Client Secret**: Enter your service principal secret.
370-
- **Client Authentication**: Select **Send as Basic Auth header**.
371-
372-
1. You will be asked to sign in with your username and password.
373-
1. Once you get your access token, please copy and save it for the next step.
374-
375-
:::image type="content" source="./media/connector-odata/odata-project-online-postman-access-token-inline.png" alt-text="Screenshot of using Postman to get the access token." lightbox="./media/connector-odata/odata-project-online-postman-access-token-expanded.png":::
361+
> [!NOTE]
362+
> Postman is a widely used applicatoin among developers for testing remote web APIs. However, there are some security and privacy risks associated with its usage. This article does not endorse the use of Postman for production environments. Please use it at your own risk.
363+
364+
1. Navigate to **Authorization** tab on the Postman Website.
365+
1. In the **Type** box, select **OAuth 2.0**, and in the **Add authorization data to** box, select **Request Headers**.
366+
1. Fill the following information in the **Configure New Token** page to get a new access token:
367+
- **Grant type**: Select **Authorization Code**.
368+
- **Callback URL**: Enter `https://www.localhost.com/`.
369+
- **Auth URL**: Enter `https://login.microsoftonline.com/common/oauth2/authorize?resource=https://<your tenant name>.sharepoint.com`. Replace `<your tenant name>` with your own tenant name.
370+
- **Access Token URL**: Enter `https://login.microsoftonline.com/common/oauth2/token`.
371+
- **Client ID**: Enter your Microsoft Entra service principal ID.
372+
- **Client Secret**: Enter your service principal secret.
373+
- **Client Authentication**: Select **Send as Basic Auth header**.
374+
1. You will be asked to sign in with your username and password.
375+
1. Once you get your access token, please copy and save it for the next step.
376+
377+
:::image type="content" source="./media/connector-odata/odata-project-online-postman-access-token-inline.png" alt-text="Screenshot of using Postman to get the access token." lightbox="./media/connector-odata/odata-project-online-postman-access-token-expanded.png":::
376378

377379
1. Create the OData linked service:
378380
- **Service URL**: Enter `https://<your tenant name>.sharepoint.com/sites/pwa/_api/Projectdata`. Replace `<your tenant name>` with your own tenant name.

articles/data-factory/connector-rest.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ Specifically, this generic REST connector supports:
4343
- For REST as source, copying the REST JSON response [as-is](#export-json-response-as-is) or parse it by using [schema mapping](copy-activity-schema-and-type-mapping.md#schema-mapping). Only response payload in **JSON** is supported.
4444

4545
> [!TIP]
46-
> To test a request for data retrieval before you configure the REST connector in Data Factory, learn about the API specification for header and body requirements. You can use tools like Postman or a web browser to validate.
46+
> To test a request for data retrieval before you configure the REST connector in Data Factory, learn about the API specification for header and body requirements. You can use tools like Visual Studio, PowerShell's Invoke-RestMethod or a web browser to validate.
4747
4848
## Prerequisites
4949

articles/data-factory/connector-troubleshoot-rest.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ This article provides suggestions to troubleshoot common problems with the REST
5858

5959
- Note that 'curl' might not be suitable to reproduce an SSL certificate validation issue. In some scenarios, the 'curl' command was executed successfully without encountering any SSL certificate validation issues. But when the same URL is executed in a browser, no SSL certificate is actually returned for the client to establish trust with server.
6060

61-
Tools like **Postman** and **Fiddler** are recommended for the preceding case.
61+
Tools like **Fiddler** are recommended for the preceding case.
6262

6363
## Related content
6464

articles/data-factory/data-factory-troubleshoot-guide.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1076,7 +1076,7 @@ If the HDI activity is stuck in preparing for cluster, follow the guidelines bel
10761076
10771077
- **Cause**: The request failed due to an underlying issue such as network connectivity, a DNS failure, a server certificate validation, or a timeout.
10781078
1079-
- **Recommendation**: Use Fiddler/Postman/Netmon/Wireshark to validate the request.
1079+
- **Recommendation**: Use Fiddler/Netmon/Wireshark to validate the request.
10801080
10811081
**Using Fiddler**
10821082

articles/data-factory/security-and-access-control-troubleshoot-guide.md

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -50,21 +50,21 @@ The problem is usually caused by one of the following factors:
5050

5151
* If you're using an **Azure IR**, try to disable the firewall setting of the datastore. This approach can resolve the issues in the following two situations:
5252

53-
* [Azure IR IP addresses](./azure-integration-runtime-ip-addresses.md) are not in the allow list.
53+
* [Azure IR IP addresses](./azure-integration-runtime-ip-addresses.md) aren't in the allowlist.
5454
* The *Allow trusted Microsoft services to access this storage account* feature is turned off for [Azure Blob Storage](./connector-azure-blob-storage.md#supported-capabilities) and [Azure Data Lake Storage Gen 2](./connector-azure-data-lake-storage.md#supported-capabilities).
5555
* The *Allow access to Azure services* setting isn't enabled for Azure Data Lake Storage Gen1.
5656

5757
If none of the preceding methods works, contact Microsoft for help.
5858

59-
### Deleted or rejected private end point still shows Aprroved in ADF
59+
### Deleted or rejected private end point still shows Approved in ADF
6060

6161
#### Symptoms
6262

6363
You created managed private endpoint from ADF and obtained an approved private endpoint. But, after deleting or rejecting the private endpoint later, the managed private endpoint in ADF still persists to exist and shows "Approved".
6464

6565
#### Cause
6666

67-
Currently, ADF stops pulling private end point status after it is approved. Hence the status shown in ADF is stale.
67+
Currently, ADF stops pulling private end point status after it's approved. Hence the status shown in ADF is stale.
6868

6969
##### Resolution
7070

@@ -118,7 +118,7 @@ To resolve the issue, do the following:
118118

119119
You're unable to register the IR authentication key on the self-hosted VM because the private link is enabled. You receive the following error message:
120120

121-
"Failed to get service token from ADF service with key *************** and time cost is: 0.1250079 second, the error code is: InvalidGatewayKey, activityId is: XXXXXXX and detailed error message is Client IP address is not valid private ip Cause Data factory couldn't access the public network thereby not able to reach out to the cloud to make the successful connection."
121+
"Failed to get service token from ADF service with key *************** and time cost is: 0.1250079 second, the error code is: InvalidGatewayKey, activityId is: XXXXXXX and detailed error message is Client IP address isn't valid private ip Cause Data factory couldn't access the public network thereby not able to reach out to the cloud to make the successful connection."
122122

123123
#### Cause
124124

@@ -170,18 +170,18 @@ Try to enable public network access on the user interface, as shown in the follo
170170
### Service private DNS zone overrides Azure Resource Manager DNS resolution causing 'Not found' error
171171
172172
#### Cause
173-
Both Azure Resource Manager and the service are using the same private zone creating a potential conflict on customer's private DNS with a scenario where the Azure Resource Manager records will not be found.
173+
Both Azure Resource Manager and the service are using the same private zone creating a potential conflict on customer's private DNS with a scenario where the Azure Resource Manager records won't be found.
174174
175175
#### Resolution
176176
1. Find Private DNS zones **privatelink.azure.com** in Azure portal.
177177
:::image type="content" source="media/security-access-control-troubleshoot-guide/private-dns-zones.png" alt-text="Screenshot of finding Private DNS zones.":::
178-
2. Check if there is an A record **adf**.
178+
2. Check if there's an A record **adf**.
179179
:::image type="content" source="media/security-access-control-troubleshoot-guide/a-record.png" alt-text="Screenshot of A record.":::
180180
3. Go to **Virtual network links**, delete all records.
181181
:::image type="content" source="media/security-access-control-troubleshoot-guide/virtual-network-link.png" alt-text="Screenshot of virtual network link.":::
182182
4. Navigate to your service in Azure portal and recreate the private endpoint for the portal.
183183
:::image type="content" source="media/security-access-control-troubleshoot-guide/create-private-endpoint.png" alt-text="Screenshot of recreating private endpoint.":::
184-
5. Go back to Private DNS zones, and check if there is a new private DNS zone **privatelink.adf.azure.com**.
184+
5. Go back to Private DNS zones, and check if there's a new private DNS zone **privatelink.adf.azure.com**.
185185
:::image type="content" source="media/security-access-control-troubleshoot-guide/check-dns-record.png" alt-text="Screenshot of new DNS record.":::
186186
187187
### Connection error in public endpoint
@@ -190,7 +190,7 @@ Both Azure Resource Manager and the service are using the same private zone crea
190190
191191
When copying data with Azure Blob Storage account public access, pipeline runs randomly fail with following error.
192192
193-
For example: The Azure Blob Storage sink was using Azure IR (public, not Managed VNet) and the Azure SQL Database source was using the Managed VNet IR. Or source/sink use Managed VNet IR only with storage public access.
193+
For example: The Azure Blob Storage sink was using Azure IR (public, not Managed virtual network) and the Azure SQL Database source was using the Managed virtual network IR. Or source/sink use Managed virtual network IR only with storage public access.
194194
195195
`
196196
<LogProperties><Text>Invoke callback url with req:
@@ -199,12 +199,12 @@ For example: The Azure Blob Storage sink was using Azure IR (public, not Managed
199199
200200
#### Cause
201201
202-
The service may still use Managed VNet IR, but you could encounter such error because the public endpoint to Azure Blob Storage in Managed VNet is not reliable based on the testing result, and Azure Blob Storage and Azure Data Lake Gen2 are not supported to be connected through public endpoint from the service's Managed Virtual Network according to [Managed virtual network & managed private endpoints](./managed-virtual-network-private-endpoint.md#outbound-communications-through-public-endpoint-from-a-data-factory-managed-virtual-network).
202+
The service might still use Managed virtual network IR, but you could encounter such error because the public endpoint to Azure Blob Storage in Managed virtual network isn't reliable based on the testing result, and Azure Blob Storage and Azure Data Lake Gen2 aren't supported to be connected through public endpoint from the service's Managed Virtual Network according to [Managed virtual network & managed private endpoints](./managed-virtual-network-private-endpoint.md#outbound-communications-through-public-endpoint-from-a-data-factory-managed-virtual-network).
203203
204204
#### Resolution
205205
206-
- Having private endpoint enabled on the source and also the sink side when using the Managed VNet IR.
207-
- If you still want to use the public endpoint, you can switch to public IR only instead of using the Managed VNet IR for the source and the sink. Even if you switch back to public IR, the service may still use the Managed VNet IR if the Managed VNet IR is still there.
206+
- Having private endpoint enabled on the source and also the sink side when using the Managed virtual network IR.
207+
- If you still want to use the public endpoint, you can switch to public IR only instead of using the Managed virtual network IR for the source and the sink. Even if you switch back to public IR, the service may still use the Managed virtual network IR if the Managed virtual network IR is still there.
208208
209209
### Internal error while trying to Delete a data factory or Synapse workspace with Customer Managed Key (CMK) and User Assigned Managed Identity (UA-MI)
210210
@@ -213,35 +213,35 @@ The service may still use Managed VNet IR, but you could encounter such error be
213213
214214
#### Cause
215215
216-
If you are performing any operations related to CMK, you should complete all operations related to the service first, and then external operations (like Managed Identities or Key Vault operations). For example, if you want to delete all resources, you need to delete the service instance first, and then delete the key vault. If you delete the key vault first, this error will occur since the service can't read the required objects anymore, and it won't be able to validate if deletion is possible or not.
216+
If you're performing any operations related to CMK, you should complete all operations related to the service first, and then external operations (like Managed Identities or Key Vault operations). For example, if you want to delete all resources, you need to delete the service instance first, and then delete the key vault. If you delete the key vault first, this error occurs since the service can't read the required objects anymore, and it won't be able to validate if deletion is possible or not.
217217
218218
#### Resolution
219219
220220
There are three possible ways to solve the issue. They are as follows:
221221
222222
* You revoked the service's access to Key vault where the CMK key was stored.
223-
You can reassign access to the following permissions: **Get, Unwrap Key, and Wrap Key**. These permissions are required to enable customer-managed keys. Please refer to [Grant access to customer-managed keys](enable-customer-managed-key.md#grant-data-factory-access-to-azure-key-vault). Once the permission is provided, you should be able to delete the service.
223+
You can reassign access to the following permissions: **Get, Unwrap Key, and Wrap Key**. These permissions are required to enable customer-managed keys. Refer to [Grant access to customer-managed keys](enable-customer-managed-key.md#grant-data-factory-access-to-azure-key-vault). Once the permission is provided, you should be able to delete the service.
224224
225225
* Customer deleted Key Vault / CMK before deleting the service.
226226
CMK in the service should have "Soft Delete" enabled and "Purge Protect" enabled which has default retention policy of 90 days. You can restore the deleted key.
227-
Please review [Recover deleted Key](../key-vault/general/key-vault-recovery.md?tabs=azure-portal#list-recover-or-purge-soft-deleted-secrets-keys-and-certificates) and [Deleted Key Value](../key-vault/general/key-vault-recovery.md?tabs=azure-portal#list-recover-or-purge-a-soft-deleted-key-vault)
227+
Review [Recover deleted Key](../key-vault/general/key-vault-recovery.md?tabs=azure-portal#list-recover-or-purge-soft-deleted-secrets-keys-and-certificates) and [Deleted Key Value](../key-vault/general/key-vault-recovery.md?tabs=azure-portal#list-recover-or-purge-a-soft-deleted-key-vault)
228228
229-
* User Assigned Managed Identity (UA-MI) was deleted before the service.
230-
You can recover from this by using REST API calls, you can do this in an http client of your choice in any programming language. If you have not anything already set up for REST API calls with Azure authentication, the easiest way to do this would be by using POSTMAN/Fiddler. Please follow following steps.
229+
* User Assigned Managed Identity (UA-MI) was deleted before the service.
230+
You can recover from this by using REST API calls. You can do this in an http client of your choice in any programming language. If you have not anything already set up for REST API calls with Azure authentication, the easiest way to do this 'd be by using Fiddler. Follow following steps.
231231
232232
1. Make a GET call using Method: GET Url like `https://management.azure.com/subscriptions/YourSubscription/resourcegroups/YourResourceGroup/providers/Microsoft.DataFactory/factories/YourFactoryName?api-version=2018-06-01`
233233
234-
2. You need to create a new User Managed Identity with a different Name (same name may work, but just to be sure, it's safer to use a different name than the one in the GET response)
234+
2. You need to create a new User Managed Identity with a different Name (the same name might work, but just to be sure, it's safer to use a different name than the one in the GET response)
235235
236236
3. Modify the encryption.identity property and identity.userassignedidentities to point to the newly created managed identity. Remove the clientId and principalId from the userAssignedIdentity object.
237237
238-
4. Make a PUT call to the same url passing the new body. It is very important that you are passing whatever you got in the GET response, and only modify the identity. Otherwise they would override other settings unintentionally.
238+
4. Make a PUT call to the same url passing the new body. It's important that you're passing whatever you got in the GET response, and only modify the identity. Otherwise they would override other settings unintentionally.
239239
240-
5. After the call succeeds, you will be able to see the entities again and retry deleting.
240+
5. After the call succeeds, you'll be able to see the entities again and retry deleting.
241241
242242
## Sharing Self-hosted Integration Runtime
243243
244-
### Sharing a self-hosted IR from a different tenant is not supported
244+
### Sharing a self-hosted IR from a different tenant isn't supported
245245
246246
#### Symptoms
247247

0 commit comments

Comments
 (0)