You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/connector-azure-data-explorer.md
+29-29Lines changed: 29 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,9 +15,9 @@ ms.date: 08/01/2019
15
15
ms.author: orspodek
16
16
---
17
17
18
-
# Copy data to or from Azure Data Explorer using Azure Data Factory
18
+
# Copy data to or from Azure Data Explorer by using Azure Data Factory
19
19
20
-
This article outlines how to use the Copy Activity in Azure Data Factory to copy data to or from [Azure Data Explorer](../data-explorer/data-explorer-overview.md). It builds on the [copy activity overview](copy-activity-overview.md) article that presents a general overview of copy activity.
20
+
This article describes how to use the copy activity in Azure Data Factory to copy data to or from [Azure Data Explorer](../data-explorer/data-explorer-overview.md). It builds on the [copy activity overview](copy-activity-overview.md) article, which offers a general overview of copy activity.
21
21
22
22
## Supported capabilities
23
23
@@ -26,12 +26,12 @@ This Azure Data Explorer connector is supported for the following activities:
26
26
-[Copy activity](copy-activity-overview.md) with [supported source/sink matrix](copy-activity-overview.md)
You can copy data from any supported source data store to Azure Data Explorer. You can also copy data from Azure Data Explorer to any supported sink data store. For a list of data stores that are supported as sources or sinks by the copy activity, see the [Supported data stores](copy-activity-overview.md) table.
29
+
You can copy data from any supported source data store to Azure Data Explorer. You can also copy data from Azure Data Explorer to any supported sink data store. For a list of data stores that the copy activity supports as sources or sinks, see the [Supported data stores](copy-activity-overview.md#supported-data-stores-and-formats) table.
30
30
31
31
>[!NOTE]
32
-
>Copying data to/from Azure Data Explorer from/to on premises data store using Self-hosted Integration Runtime is supported since version 3.14.
32
+
>Copying data to or from Azure Data Explorer through an on-premises data store by using self-hosted integration runtime is supported in version 3.14 and later.
33
33
34
-
The Azure Data Explorer connector allows you to do the following:
34
+
With the Azure Data Explorer connector, you can do the following:
35
35
36
36
* Copy data by using Azure Active Directory (Azure AD) application token authentication with a **service principal**.
37
37
* As a source, retrieve data by using a KQL (Kusto) query.
@@ -40,42 +40,42 @@ The Azure Data Explorer connector allows you to do the following:
40
40
## Getting started
41
41
42
42
>[!TIP]
43
-
>For a walkthrough of using Azure Data Explorer connector, see [Copy data to/from Azure Data Explorer using Azure Data Factory](../data-explorer/data-factory-load-data.md).
43
+
>For a walkthrough of Azure Data Explorer connector, see [Copy data to/from Azure Data Explorer using Azure Data Factory](../data-explorer/data-factory-load-data.md).
The following sections provide details about properties that are used to define Data Factory entities specific to Azure Data Explorer connector.
48
48
49
49
## Linked service properties
50
50
51
-
The Azure Data Explorer connector uses service principal authentication. Follow these steps to get a service principal and grant permissions:
51
+
The Azure Data Explorer connector uses service principal authentication. Follow these steps to get a service principal and to grant permissions:
52
52
53
-
1. Register an application entity in Azure Active Directory (Azure AD) by following [Register your application with an Azure AD tenant](../storage/common/storage-auth-aad-app.md#register-your-application-with-an-azure-ad-tenant). Make note of the following values, which you use to define the linked service:
53
+
1. Register an application entity in Azure Active Directory by following the steps in[Register your application with an Azure AD tenant](../storage/common/storage-auth-aad-app.md#register-your-application-with-an-azure-ad-tenant). Make note of the following values, which you use to define the linked service:
54
54
55
55
- Application ID
56
56
- Application key
57
57
- Tenant ID
58
58
59
-
2. Grant the service principal proper permission in Azure Data Explorer. Refer to [Manage Azure Data Explorer database permissions](../data-explorer/manage-database-permissions.md)with detailed information on roles and permissions as well as walk through on managing permissions. In general, you need to
59
+
2. Grant the service principal the correct permissions in Azure Data Explorer. See [Manage Azure Data Explorer database permissions](../data-explorer/manage-database-permissions.md)for detailed information about roles and permissions and about managing permissions. In general, you must:
60
60
61
-
-**As source**, grant at least **Database viewer** role to your database.
62
-
-**As sink**, grant at least **Database ingestor** role to your database.
61
+
-**As source**, grant at least the **Database viewer** role to your database
62
+
-**As sink**, grant at least the **Database ingestor** role to your database
63
63
64
64
>[!NOTE]
65
-
>When using ADF UI to author, your login user account is used to list Azure Data Explorer clusters, databases and tables. Manually input the name if you don’t have permission for such operation.
65
+
>When you use the Data Factory UI to author, your login user account is used to list Azure Data Explorer clusters, databases, and tables. Manually enter the name if you don’t have permission for these operations.
66
66
67
-
The following properties are supported for Azure Data Explorer linked service:
67
+
The following properties are supported for the Azure Data Explorer linked service:
68
68
69
69
| Property | Description | Required |
70
70
|:--- |:--- |:--- |
71
-
| type | The **type** property must be set to **AzureDataExplorer**| Yes |
71
+
| type | The **type** property must be set to **AzureDataExplorer**.| Yes |
72
72
| endpoint | Endpoint URL of the Azure Data Explorer cluster, with the format as `https://<clusterName>.<regionName>.kusto.windows.net`. | Yes |
73
73
| database | Name of database. | Yes |
74
-
| tenant | Specify the tenant information (domain name or tenant ID) under which your application resides. This is what you normally know as "**Authority ID**" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). Retrieve it by hovering with the mouse in the top-right corner of the Azure portal. | Yes |
75
-
| servicePrincipalId | Specify the application's client ID. This is what you normally know as "**AAD application client ID**" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). | Yes |
76
-
| servicePrincipalKey | Specify the application's key. This is what you normally know as "**AAD application key**" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). Mark this field as a **SecureString** to store it securely in Data Factory, or [reference a secret stored in Azure Key Vault](store-credentials-in-key-vault.md). | Yes |
74
+
| tenant | Specify the tenant information (domain name or tenant ID) under which your application resides. This is known as "Authority ID" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). Retrieve it by hovering the mouse pointer in the upper-right corner of the Azure portal. | Yes |
75
+
| servicePrincipalId | Specify the application's client ID. This is known as "AAD application client ID" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). | Yes |
76
+
| servicePrincipalKey | Specify the application's key. This is known as "AAD application key" in [Kusto connection string](https://docs.microsoft.com/azure/kusto/api/connection-strings/kusto#application-authentication-properties). Mark this field as a **SecureString** to store it securely in Data Factory, or [reference secure data stored in Azure Key Vault](store-credentials-in-key-vault.md). | Yes |
77
77
78
-
**Linked Service Properties Example:**
78
+
**Linked service properties example:**
79
79
80
80
```json
81
81
{
@@ -98,18 +98,18 @@ The following properties are supported for Azure Data Explorer linked service:
98
98
99
99
## Dataset properties
100
100
101
-
For a full list of sections and properties available for defining datasets, see the [datasets](concepts-datasets-linked-services.md) article. This section provides a list of properties that are supported by the Azure Data Explorer dataset.
101
+
For a full list of sections and properties available for defining datasets, see [Datasets in Azure Data Factory](concepts-datasets-linked-services.md). This section lists properties that the Azure Data Explorer dataset supports.
102
102
103
103
To copy data to Azure Data Explorer, set the type property of the dataset to **AzureDataExplorerTable**.
104
104
105
105
The following properties are supported:
106
106
107
107
| Property | Description | Required |
108
108
|:--- |:--- |:--- |
109
-
| type | The **type** property must be set to **AzureDataExplorerTable**| Yes |
109
+
| type | The **type** property must be set to **AzureDataExplorerTable**.| Yes |
110
110
| table | The name of the table that the linked service refers to. | Yes for sink; No for source |
111
111
112
-
**Dataset Properties Example**
112
+
**Dataset properties example:**
113
113
114
114
```json
115
115
{
@@ -130,7 +130,7 @@ The following properties are supported:
130
130
131
131
## Copy activity properties
132
132
133
-
For a full list of sections and properties available for defining activities, see the [Pipelines](concepts-pipelines-activities.md) article. This section provides a list of properties supported by Azure Data Explorer source and sink.
133
+
For a full list of sections and properties available for defining activities, see [Pipelines and activities in Azure Data Factory](concepts-pipelines-activities.md). This section provides a list of properties that Azure Data Explorer sources and sinks support.
134
134
135
135
### Azure Data Explorer as source
136
136
@@ -141,10 +141,10 @@ To copy data from Azure Data Explorer, set the **type** property in the Copy act
141
141
| type | The **type** property of the copy activity source must be set to: **AzureDataExplorerSource**| Yes |
142
142
| query | A read-only request given in a [KQL format](/azure/kusto/query/). Use the custom KQL query as a reference. | Yes |
143
143
| queryTimeout | The wait time before the query request times out. Default value is 10 min (00:10:00); allowed max value is 1 hour (01:00:00). | No |
144
-
| noTruncation | Indicates whether to truncate the returned result set. By default, result is truncated after 500,000 records or 64 MB. Truncation is strongly recommended for a proper behavior of the activity. |No |
144
+
| noTruncation | Indicates whether to truncate the returned result set. By default, result is truncated after 500,000 records or 64 megabytes (MB). Truncation is strongly recommended to ensure the correct behavior of the activity. |No |
145
145
146
146
>[!NOTE]
147
-
>Azure Data Explorer source by default has a size limit of 500,000 records or 64 MB. To retrieve all the records without truncation, you can specify `set notruncation;` at the beginning of your query. Refer to [Query limits](https://docs.microsoft.com/azure/kusto/concepts/querylimits) on more details.
147
+
>By default, Azure Data Explorer source has a size limit of 500,000 records or 64 MB. To retrieve all the records without truncation, you can specify `set notruncation;` at the beginning of your query. For more information, see [Query limits](https://docs.microsoft.com/azure/kusto/concepts/querylimits).
148
148
149
149
**Example:**
150
150
@@ -185,8 +185,8 @@ To copy data to Azure Data Explorer, set the type property in the copy activity
185
185
186
186
| Property | Description | Required |
187
187
|:--- |:--- |:--- |
188
-
| type | The **type** property of the copy activity sink must be set to: **AzureDataExplorerSink**| Yes |
189
-
| ingestionMappingName | Name of a pre-created **[mapping](/azure/kusto/management/mappings#csv-mapping)** on a Kusto table. To map the columns from source to Azure Data Explorer - which applies to **[all supported source stores/formats](copy-activity-overview.md#supported-data-stores-and-formats)** including CSV/JSON/Avro formats etc., you can use the Copy activity [column mapping](copy-activity-schema-and-type-mapping.md) (implicitly by name or explicitly as configured) and/or Azure Data Explorer mappings. | No |
188
+
| type | The **type** property of the copy activity sink must be set to: **AzureDataExplorerSink**.| Yes |
189
+
| ingestionMappingName | Name of a pre-created [mapping](/azure/kusto/management/mappings#csv-mapping) on a Kusto table. To map the columns from source to Azure Data Explorer (which applies to [all supported source stores and formats](copy-activity-overview.md#supported-data-stores-and-formats), including CSV/JSON/Avro formats), you can use the copy activity [column mapping](copy-activity-schema-and-type-mapping.md) (implicitly by name or explicitly as configured) and/or Azure Data Explorer mappings. | No |
190
190
191
191
**Example:**
192
192
@@ -222,10 +222,10 @@ To copy data to Azure Data Explorer, set the type property in the copy activity
222
222
223
223
## Lookup activity properties
224
224
225
-
To learn details about the properties, check[Lookup activity](control-flow-lookup-activity.md).
225
+
For more information about the properties, see[Lookup activity](control-flow-lookup-activity.md).
226
226
227
227
## Next steps
228
228
229
-
* For a list of data stores supported as sources and sinks by the copy activity in Azure Data Factory, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
229
+
* For a list of data stores that the copy activity in Azure Data Factory supports as sources and sinks, see [supported data stores](copy-activity-overview.md#supported-data-stores-and-formats).
230
230
231
-
* Learn more about [Copy data from Azure Data Factory to Azure Data Explorer](/azure/data-explorer/data-factory-load-data).
231
+
* Learn more about how to [copy data from Azure Data Factory to Azure Data Explorer](/azure/data-explorer/data-factory-load-data).
0 commit comments