Skip to content

Commit 9549356

Browse files
authored
Merge pull request #114471 from Samantha-Yu/adfupdate0508-3-1
Add supported linked services for activities
2 parents 8e01d78 + ee5bac3 commit 9549356

5 files changed

+11
-11
lines changed

articles/data-factory/transform-data-using-hadoop-hive.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ author: nabhishek
1010
ms.author: abnarain
1111
manager: anandsub
1212
ms.custom: seo-lt-2019
13-
ms.date: 01/15/2019
13+
ms.date: 05/08/2019
1414
---
1515

1616
# Transform data using Hadoop Hive activity in Azure Data Factory
@@ -59,7 +59,7 @@ If you are new to Azure Data Factory, read through [Introduction to Azure Data F
5959
| description | Text describing what the activity is used for | No |
6060
| type | For Hive Activity, the activity type is HDinsightHive | Yes |
6161
| linkedServiceName | Reference to the HDInsight cluster registered as a linked service in Data Factory. To learn about this linked service, see [Compute linked services](compute-linked-services.md) article. | Yes |
62-
| scriptLinkedService | Reference to an Azure Storage Linked Service used to store the Hive script to be executed. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
62+
| scriptLinkedService | Reference to an Azure Storage Linked Service used to store the Hive script to be executed. Only **[Azure Blob Storage](https://docs.microsoft.com/azure/data-factory/connector-azure-blob-storage)** and **[ADLS Gen2](https://docs.microsoft.com/azure/data-factory/connector-azure-data-lake-storage)** linked services are supported here. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
6363
| scriptPath | Provide the path to the script file stored in the Azure Storage referred by scriptLinkedService. The file name is case-sensitive. | Yes |
6464
| getDebugInfo | Specifies when the log files are copied to the Azure Storage used by HDInsight cluster (or) specified by scriptLinkedService. Allowed values: None, Always, or Failure. Default value: None. | No |
6565
| arguments | Specifies an array of arguments for a Hadoop job. The arguments are passed as command-line arguments to each task. | No |

articles/data-factory/transform-data-using-hadoop-map-reduce.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ author: nabhishek
99
ms.author: abnarain
1010
manager: shwang
1111
ms.custom: seo-lt-2019
12-
ms.date: 01/16/2018
12+
ms.date: 05/08/2020
1313
---
1414

1515
# Transform data using Hadoop MapReduce activity in Azure Data Factory
@@ -64,7 +64,7 @@ See [Pig](transform-data-using-hadoop-pig.md) and [Hive](transform-data-using-ha
6464
| type | For MapReduce Activity, the activity type is HDinsightMapReduce | Yes |
6565
| linkedServiceName | Reference to the HDInsight cluster registered as a linked service in Data Factory. To learn about this linked service, see [Compute linked services](compute-linked-services.md) article. | Yes |
6666
| className | Name of the Class to be executed | Yes |
67-
| jarLinkedService | Reference to an Azure Storage Linked Service used to store the Jar files. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
67+
| jarLinkedService | Reference to an Azure Storage Linked Service used to store the Jar files. Only **[Azure Blob Storage](https://docs.microsoft.com/azure/data-factory/connector-azure-blob-storage)** and **[ADLS Gen2](https://docs.microsoft.com/azure/data-factory/connector-azure-data-lake-storage)** linked services are supported here. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
6868
| jarFilePath | Provide the path to the Jar files stored in the Azure Storage referred by jarLinkedService. The file name is case-sensitive. | Yes |
6969
| jarlibs | String array of the path to the Jar library files referenced by the job stored in the Azure Storage defined in jarLinkedService. The file name is case-sensitive. | No |
7070
| getDebugInfo | Specifies when the log files are copied to the Azure Storage used by HDInsight cluster (or) specified by jarLinkedService. Allowed values: None, Always, or Failure. Default value: None. | No |
@@ -107,7 +107,7 @@ You can use the HDInsight MapReduce Activity to run any MapReduce jar file on an
107107
}
108108
}
109109
```
110-
You can specify any arguments for the MapReduce program in the **arguments** section. At runtime, you see a few extra arguments (for example: mapreduce.job.tags) from the MapReduce framework. To differentiate your arguments with the MapReduce arguments, consider using both option and value as arguments as shown in the following example (-s, --input, --output etc., are options immediately followed by their values).
110+
You can specify any arguments for the MapReduce program in the **arguments** section. At runtime, you see a few extra arguments (for example: mapreduce.job.tags) from the MapReduce framework. To differentiate your arguments with the MapReduce arguments, consider using both option and value as arguments as shown in the following example (-s,--input,--output etc., are options immediately followed by their values).
111111

112112
## Next steps
113113
See the following articles that explain how to transform data in other ways:

articles/data-factory/transform-data-using-hadoop-pig.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ author: nabhishek
1010
ms.author: abnarain
1111
manager: shwang
1212
ms.custom: seo-lt-2019
13-
ms.date: 01/16/2018
13+
ms.date: 05/08/2020
1414
---
1515

1616
# Transform data using Hadoop Pig activity in Azure Data Factory
@@ -60,7 +60,7 @@ If you are new to Azure Data Factory, read through [Introduction to Azure Data F
6060
| description | Text describing what the activity is used for | No |
6161
| type | For Hive Activity, the activity type is HDinsightPig | Yes |
6262
| linkedServiceName | Reference to the HDInsight cluster registered as a linked service in Data Factory. To learn about this linked service, see [Compute linked services](compute-linked-services.md) article. | Yes |
63-
| scriptLinkedService | Reference to an Azure Storage Linked Service used to store the Pig script to be executed. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
63+
| scriptLinkedService | Reference to an Azure Storage Linked Service used to store the Pig script to be executed. Only **[Azure Blob Storage](https://docs.microsoft.com/azure/data-factory/connector-azure-blob-storage)** and **[ADLS Gen2](https://docs.microsoft.com/azure/data-factory/connector-azure-data-lake-storage)** linked services are supported here. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
6464
| scriptPath | Provide the path to the script file stored in the Azure Storage referred by scriptLinkedService. The file name is case-sensitive. | No |
6565
| getDebugInfo | Specifies when the log files are copied to the Azure Storage used by HDInsight cluster (or) specified by scriptLinkedService. Allowed values: None, Always, or Failure. Default value: None. | No |
6666
| arguments | Specifies an array of arguments for a Hadoop job. The arguments are passed as command-line arguments to each task. | No |

articles/data-factory/transform-data-using-hadoop-streaming.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.service: data-factory
99
ms.workload: data-services
1010
ms.topic: conceptual
1111
ms.custom: seo-lt-2019
12-
ms.date: 01/16/2018
12+
ms.date: 05/08/2020
1313
---
1414

1515
# Transform data using Hadoop Streaming activity in Azure Data Factory
@@ -73,7 +73,7 @@ If you are new to Azure Data Factory, read through [Introduction to Azure Data F
7373
| mapper | Specifies the name of the mapper executable | Yes |
7474
| reducer | Specifies the name of the reducer executable | Yes |
7575
| combiner | Specifies the name of the combiner executable | No |
76-
| fileLinkedService | Reference to an Azure Storage Linked Service used to store the Mapper, Combiner, and Reducer programs to be executed. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
76+
| fileLinkedService | Reference to an Azure Storage Linked Service used to store the Mapper, Combiner, and Reducer programs to be executed. Only **[Azure Blob Storage](https://docs.microsoft.com/azure/data-factory/connector-azure-blob-storage)** and **[ADLS Gen2](https://docs.microsoft.com/azure/data-factory/connector-azure-data-lake-storage)** linked services are supported here. If you don't specify this Linked Service, the Azure Storage Linked Service defined in the HDInsight Linked Service is used. | No |
7777
| filePath | Provide an array of path to the Mapper, Combiner, and Reducer programs stored in the Azure Storage referred by fileLinkedService. The path is case-sensitive. | Yes |
7878
| input | Specifies the WASB path to the input file for the Mapper. | Yes |
7979
| output | Specifies the WASB path to the output file for the Reducer. | Yes |

articles/data-factory/transform-data-using-spark.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ author: nabhishek
99
ms.author: abnarain
1010
manager: shwang
1111
ms.custom: seo-lt-2019
12-
ms.date: 05/31/2018
12+
ms.date: 05/08/2020
1313
---
1414

1515
# Transform data using Spark activity in Azure Data Factory
@@ -60,7 +60,7 @@ The following table describes the JSON properties used in the JSON definition:
6060
| description | Text describing what the activity does. | No |
6161
| type | For Spark Activity, the activity type is HDInsightSpark. | Yes |
6262
| linkedServiceName | Name of the HDInsight Spark Linked Service on which the Spark program runs. To learn about this linked service, see [Compute linked services](compute-linked-services.md) article. | Yes |
63-
| SparkJobLinkedService | The Azure Storage linked service that holds the Spark job file, dependencies, and logs. If you do not specify a value for this property, the storage associated with HDInsight cluster is used. The value of this property can only be an Azure Storage linked service. | No |
63+
| SparkJobLinkedService | The Azure Storage linked service that holds the Spark job file, dependencies, and logs. Only **[Azure Blob Storage](https://docs.microsoft.com/azure/data-factory/connector-azure-blob-storage)** and **[ADLS Gen2](https://docs.microsoft.com/azure/data-factory/connector-azure-data-lake-storage)** linked services are supported here. If you do not specify a value for this property, the storage associated with HDInsight cluster is used. The value of this property can only be an Azure Storage linked service. | No |
6464
| rootPath | The Azure Blob container and folder that contains the Spark file. The file name is case-sensitive. Refer to folder structure section (next section) for details about the structure of this folder. | Yes |
6565
| entryFilePath | Relative path to the root folder of the Spark code/package. The entry file must be either a Python file or a .jar file. | Yes |
6666
| className | Application's Java/Spark main class | No |

0 commit comments

Comments
 (0)