You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/connector-rest.md
+45-75Lines changed: 45 additions & 75 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ ms.service: data-factory
7
7
ms.subservice: data-movement
8
8
ms.custom: synapse
9
9
ms.topic: conceptual
10
-
ms.date: 06/07/2022
10
+
ms.date: 06/13/2022
11
11
ms.author: makromer
12
12
---
13
13
@@ -79,10 +79,17 @@ The following properties are supported for the REST linked service:
79
79
| type | The **type** property must be set to **RestService**. | Yes |
80
80
| url | The base URL of the REST service. | Yes |
81
81
| enableServerCertificateValidation | Whether to validate server-side TLS/SSL certificate when connecting to the endpoint. | No<br /> (the default is **true**) |
82
-
| authenticationType | Type of authentication used to connect to the REST service. Allowed values are **Anonymous**, **Basic**, **AadServicePrincipal**, and **ManagedServiceIdentity**. User-based OAuth isn't supported. You can additionally configure authentication headers in `authHeader` property. Refer to corresponding sections below on more properties and examples respectively.| Yes |
82
+
| authenticationType | Type of authentication used to connect to the REST service. Allowed values are **Anonymous**, **Basic**, **AadServicePrincipal**, **OAuth2ClientCredential**, and **ManagedServiceIdentity**. You can additionally configure authentication headers in `authHeader` property. Refer to corresponding sections below on more properties and examples respectively.| Yes |
83
83
| authHeaders | Additional HTTP request headers for authentication.<br/> For example, to use API key authentication, you can select authentication type as “Anonymous” and specify API key in the header. | No |
84
84
| connectVia | The [Integration Runtime](concepts-integration-runtime.md) to use to connect to the data store. Learn more from [Prerequisites](#prerequisites) section. If not specified, this property uses the default Azure Integration Runtime. |No |
85
85
86
+
For different authentication types, see the corresponding sections for details.
Set the **authenticationType** property to **Basic**. In addition to the generic properties that are described in the preceding section, specify the following properties:
@@ -128,7 +135,7 @@ Set the **authenticationType** property to **AadServicePrincipal**. In addition
128
135
| aadResourceId | Specify the AAD resource you are requesting for authorization, for example, `https://management.core.windows.net`.| Yes |
129
136
| azureCloudType | For service principal authentication, specify the type of Azure cloud environment to which your AAD application is registered. <br/> Allowed values are **AzurePublic**, **AzureChina**, **AzureUsGovernment**, and **AzureGermany**. By default, the data factory's cloud environment is used. | No |
130
137
131
-
**Example**
138
+
**Example**
132
139
133
140
```json
134
141
{
@@ -153,6 +160,41 @@ Set the **authenticationType** property to **AadServicePrincipal**. In addition
153
160
}
154
161
}
155
162
```
163
+
### Use OAuth2 Client Credential authentication
164
+
165
+
Set the **authenticationType** property to **OAuth2ClientCredential**. In addition to the generic properties that are described in the preceding section, specify the following properties:
166
+
167
+
| Property | Description | Required |
168
+
|:--- |:--- |:--- |
169
+
| tokenEndpoint| The token endpoint of the authorization server to acquire the access token. | Yes |
170
+
| clientId | The client ID associated with your application. | Yes |
171
+
| clientSecret| The client secret associated with your application. Mark this field as a **SecureString** type to store it securely in Data Factory. You can also [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | Yes |
172
+
| scope | The scope of the access required. It describes what kind of access will be requested. | No |
173
+
| resource | The target service or resource to which the access will be requested. | No |
174
+
175
+
**Example**
176
+
177
+
```json
178
+
{
179
+
"name": "RESTLinkedService",
180
+
"properties": {
181
+
"type": "RestService",
182
+
"typeProperties": {
183
+
"url": "<REST endpoint e.g. https://www.example.com/>",
184
+
"enableServerCertificateValidation": true,
185
+
"authenticationType": "OAuth2ClientCredential",
186
+
"clientId": "<client ID>",
187
+
"clientSecret": {
188
+
"type": "SecureString",
189
+
"value": "<client secret>"
190
+
},
191
+
"tokenEndpoint": "<token endpoint>",
192
+
"scope": "<scope>",
193
+
"resource": "<resource>"
194
+
}
195
+
}
196
+
}
197
+
```
156
198
157
199
### Use user-assigned managed identity authentication
158
200
Set the **authenticationType** property to **ManagedServiceIdentity**. In addition to the generic properties that are described in the preceding section, specify the following properties:
@@ -827,78 +869,6 @@ The pagination rule syntax is the same as in Example 8 and should be set as belo
827
869
828
870
:::image type="content" source="media/connector-rest/pagination-rule-example-9.png" alt-text="Screenshot showing setting the pagination rule for Example 9.":::
829
871
830
-
831
-
## Use OAuth
832
-
This section describes how to use a solution template to copy data from REST connector into Azure Data Lake Storage in JSON format using OAuth.
833
-
834
-
### About the solution template
835
-
836
-
The template contains two activities:
837
-
-**Web** activity retrieves the bearer token and then pass it to subsequent Copy activity as authorization.
838
-
-**Copy** activity copies data from REST to Azure Data Lake Storage.
839
-
840
-
The template defines two parameters:
841
-
-**SinkContainer** is the root folder path where the data is copied to in your Azure Data Lake Storage.
842
-
-**SinkDirectory** is the directory path under the root where the data is copied to in your Azure Data Lake Storage.
843
-
844
-
### How to use this solution template
845
-
846
-
1. Go to the **Copy from REST or HTTP using OAuth** template. Create a new connection for Source Connection.
847
-
:::image type="content" source="media/solution-template-copy-from-rest-or-http-using-oauth/source-connection.png" alt-text="Create new connections":::
848
-
849
-
Below are key steps for new linked service (REST) settings:
850
-
851
-
1. Under **Base URL**, specify the url parameter for your own source REST service.
852
-
2. For **Authentication type**, choose *Anonymous*.
:::image type="content" source="media/solution-template-copy-from-rest-or-http-using-oauth/use-this-template.png" alt-text="Use this template":::
860
-
861
-
4. You would see the pipeline created as shown in the following example:
862
-
:::image type="content" source="media/solution-template-copy-from-rest-or-http-using-oauth/pipeline.png" alt-text="Screenshot shows the pipeline created from the template.":::
863
-
864
-
5. Select **Web** activity. In **Settings**, specify the corresponding **URL**, **Method**, **Headers**, and **Body** to retrieve OAuth bearer token from the login API of the service that you want to copy data from. The placeholder in the template showcases a sample of Azure Active Directory (AAD) OAuth. Note AAD authentication is natively supported by REST connector, here is just an example for OAuth flow.
865
-
866
-
| Property | Description |
867
-
|:--- |:--- |
868
-
| URL |Specify the url to retrieve OAuth bearer token from. for example, in the sample here it's https://login.microsoftonline.com/microsoft.onmicrosoft.com/oauth2/token|
869
-
| Method | The HTTP method. Allowed values are **Post** and **Get**. |
870
-
| Headers | Header is user-defined, which references one header name in the HTTP request. |
6. In **Copy data** activity, select *Source* tab, you could see that the bearer token (access_token) retrieved from previous step would be passed to Copy data activity as **Authorization** under Additional headers. Confirm settings for following properties before starting a pipeline run.
876
-
877
-
| Property | Description |
878
-
|:--- |:--- |
879
-
| Request method | The HTTP method. Allowed values are **Get** (default) and **Post**. |
10. Click the "Input" icon of CopyActivity in **Actions** column, you would see the access_token retrieved by WebActivity is passed to CopyActivity for authentication.
>To avoid token being logged in plain text, enable "Secure output" in Web activity and "Secure input" in Copy activity.
900
-
901
-
902
872
## Export JSON response as-is
903
873
904
874
You can use this REST connector to export REST API JSON response as-is to various file-based stores. To achieve such schema-agnostic copy, skip the "structure" (also called *schema*) section in dataset and schema mapping in copy activity.
0 commit comments