You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-go-sdk-overview.md
+29-16Lines changed: 29 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,13 @@
1
1
---
2
2
title: Azure HDInsight SDK for Go
3
3
description: Reference material for using Azure HDInsight SDK for Go and Apache Hadoop clusters
4
-
author: tylerfox
5
-
4
+
author: hrasheed-msft
5
+
ms.author: hrasheed
6
+
ms.reviewer: jasonh
6
7
ms.service: hdinsight
7
8
ms.topic: conceptual
8
-
ms.date: 05/8/2019
9
-
ms.author: tyfox
10
-
ms.reviewer: jasonh
11
9
ms.custom: seodec18
10
+
ms.date: 01/03/2020
12
11
---
13
12
14
13
# HDInsight SDK for Go (Preview)
@@ -19,9 +18,11 @@ The HDInsight SDK for Go provides classes and functions that allow you to manage
19
18
> [!NOTE]
20
19
>GoDoc reference material for this SDK is also [available here](https://godoc.org/github.com/Azure/azure-sdk-for-go/services/preview/hdinsight/mgmt/2018-06-01-preview/hdinsight).
21
20
21
+
If you don’t have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
22
+
22
23
## Prerequisites
23
24
24
-
*An Azure account. If you don't have one, [get a free trial](https://azure.microsoft.com/free/).
@@ -30,14 +31,14 @@ From your GOPATH location, run `go get github.com/Azure/azure-sdk-for-go/tree/ma
30
31
31
32
## Authentication
32
33
33
-
The SDK first needs to be authenticated with your Azure subscription. Follow the example below to create a service principal and use it to authenticate. After this is done, you will have an instance of a `ClustersClient`, which contains many functions (outlined in below sections) that can be used to perform management operations.
34
+
The SDK first needs to be authenticated with your Azure subscription. Follow the example below to create a service principal and use it to authenticate. After this is done, you'll have an instance of a `ClustersClient`, which contains many functions (outlined in below sections) that can be used to perform management operations.
34
35
35
36
> [!NOTE]
36
37
> There are other ways to authenticate besides the below example that could potentially be better suited for your needs. All functions are outlined here: [Authentication functions in the Azure SDK for Go](https://docs.microsoft.com/azure/go/azure-sdk-go-authorization)
37
38
38
39
### Authentication example using a service principal
39
40
40
-
First, login to [Azure Cloud Shell](https://shell.azure.com/bash). Verify you are currently using the subscription in which you want the service principal created.
41
+
First, login to [Azure Cloud Shell](https://shell.azure.com/bash). Verify you're currently using the subscription in which you want the service principal created.
41
42
42
43
```azurecli-interactive
43
44
az account show
@@ -94,6 +95,7 @@ The service principal information is displayed as JSON.
Copy the below snippet and fill in `TENANT_ID`, `CLIENT_ID`, `CLIENT_SECRET`, and `SUBSCRIPTION_ID` with the strings from the JSON that was returned after running the command to create the service principal.
98
100
99
101
```golang
@@ -138,29 +140,35 @@ A new cluster can be created by calling `client.Create()`.
138
140
139
141
#### Example
140
142
141
-
This example demonstrates how to create an [Apache Spark](https://spark.apache.org/) cluster with 2 head nodes and 1 worker node.
143
+
This example demonstrates how to create an [Apache Spark](https://spark.apache.org/) cluster with two head nodes and one worker node.
142
144
143
145
> [!NOTE]
144
146
> You first need to create a Resource Group and Storage Account, as explained below. If you have already created these, you can skip these steps.
145
147
146
148
##### Creating a resource group
147
149
148
150
You can create a resource group using the [Azure Cloud Shell](https://shell.azure.com/bash) by running
151
+
149
152
```azurecli-interactive
150
153
az group create -l <Region Name (i.e. eastus)> --n <Resource Group Name>
151
154
```
155
+
152
156
##### Creating a storage account
153
157
154
158
You can create a storage account using the [Azure Cloud Shell](https://shell.azure.com/bash) by running:
159
+
155
160
```azurecli-interactive
156
161
az storage account create -n <Storage Account Name> -g <Existing Resource Group Name> -l <Region Name (i.e. eastus)> --sku <SKU i.e. Standard_LRS>
157
162
```
158
-
Now run the following command to get the key for your storage account (you will need this to create a cluster):
163
+
164
+
Now run the following command to get the key for your storage account (you'll need this to create a cluster):
165
+
159
166
```azurecli-interactive
160
167
az storage account keys list -n <Storage Account Name>
161
168
```
169
+
162
170
---
163
-
The below Go snippet creates a Spark cluster with 2 head nodes and 1 worker node. Fill in the blank variables as explained in the comments and feel free to change other parameters to suit your specific needs.
171
+
The below Go snippet creates a Spark cluster with two head nodes and one worker node. Fill in the blank variables as explained in the comments and feel free to change other parameters to suit your specific needs.
164
172
165
173
```golang
166
174
// The name for the cluster you are creating
@@ -251,7 +259,7 @@ client.Get(context.Background(), "<Resource Group Name>", "<Cluster Name>")
251
259
252
260
#### Example
253
261
254
-
You can use `get` to confirm that you have successfully created your cluster.
262
+
You can use `get` to confirm that you've successfully created your cluster.
@@ -272,10 +280,13 @@ The output should look like:
272
280
### List clusters
273
281
274
282
#### List clusters under the subscription
283
+
275
284
```golang
276
285
client.List()
277
286
```
287
+
278
288
#### List clusters by resource group
289
+
279
290
```golang
280
291
client.ListByResourceGroup("<Resource Group Name>")
281
292
```
@@ -284,6 +295,7 @@ client.ListByResourceGroup("<Resource Group Name>")
284
295
> Both `List()` and `ListByResourceGroup()` return a `ClusterListResultPage` struct. To get the next page, you can call `Next()`. This can be repeated until `ClusterListResultPage.NotDone()` returns `false`, as shown in the example below.
285
296
286
297
#### Example
298
+
287
299
The following example prints the properties of all clusters for the current subscription:
288
300
289
301
```golang
@@ -317,6 +329,7 @@ You can update the tags of a given cluster like so:
317
329
```golang
318
330
client.Update(context.Background(), "<Resource Group Name>", "<Cluster Name>", hdi.ClusterPatchParameters{<map[string]*string} of Tags>)
319
331
```
332
+
320
333
#### Example
321
334
322
335
```golang
@@ -335,7 +348,7 @@ client.Resize(context.Background(), "<Resource Group Name>", "<Cluster Name>", h
335
348
336
349
The HDInsight Management SDK can also be used to manage monitoring on your clusters via the Operations Management Suite (OMS).
337
350
338
-
Similarly to how you created `ClusterClient` to use for management operations, you need to create an `ExtensionClient` to use for monitoring operations. Once you have completed the Authentication section above, you can create an `ExtensionClient` like so:
351
+
Similarly to how you created `ClusterClient` to use for management operations, you need to create an `ExtensionClient` to use for monitoring operations. Once you've completed the Authentication section above, you can create an `ExtensionClient` like so:
@@ -388,7 +401,7 @@ var scriptAction1 = hdi.RuntimeScriptAction{Name: to.StringPtr("<Script Name>"),
388
401
client.ExecuteScriptActions(context.Background(), "<Resource Group Name>", "<Cluster Name>", hdi.ExecuteScriptActionParameters{PersistOnSuccess: to.BoolPtr(true), ScriptActions: &[]hdi.RuntimeScriptAction{scriptAction1}}) //add more RuntimeScriptActions to the list to execute multiple scripts
389
402
```
390
403
391
-
For the 'Delete Script Action' and 'List Persisted Script Actions' operations, you need to create a `ScriptActionsClient`, similarly to how you created `ClusterClient` to use for management operations. Once you have completed the Authentication section above, you can create a `ScriptActionsClient` like so:
404
+
For the 'Delete Script Action' and 'List Persisted Script Actions' operations, you need to create a `ScriptActionsClient`, similarly to how you created `ClusterClient` to use for management operations. Once you've completed the Authentication section above, you can create a `ScriptActionsClient` like so:
For this operation, you need to create a `ScriptExecutionHistoryClient`, similarly to how you created `ClusterClient` to use for management operations. Once you have completed the Authentication section above, you can create a `ScriptActionsClient` like so:
452
+
For this operation, you need to create a `ScriptExecutionHistoryClient`, similarly to how you created `ClusterClient` to use for management operations. Once you've completed the Authentication section above, you can create a `ScriptActionsClient` like so:
* Explore the [GoDoc reference material](https://godoc.org/github.com/Azure/azure-sdk-for-go/services/preview/hdinsight/mgmt/2018-06-01-preview/hdinsight). The GoDocs provide reference documentation for all functions in the SDK.
490
+
Explore the [GoDoc reference material](https://godoc.org/github.com/Azure/azure-sdk-for-go/services/preview/hdinsight/mgmt/2018-06-01-preview/hdinsight). The GoDocs provide reference documentation for all functions in the SDK.
0 commit comments