You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/quick-create-azure-cli.md
+33-33Lines changed: 33 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ ms.date: 02/28/2023
13
13
---
14
14
15
15
# Quickstart: Create an Azure Stream Analytics job using the Azure CLI
16
-
In this quickstart, you will use Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. The Stream Analytics job reads data from IoT Hub, transforms the data, and writes the data back to a container in a blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
16
+
In this quickstart, you will use Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. The Stream Analytics job reads data from IoT Hub, transforms the data, and writes the output data to a container in a blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
17
17
18
18
## Before you begin
19
19
@@ -23,40 +23,39 @@ In this quickstart, you will use Azure CLI to define a Stream Analytics job that
23
23
24
24
- Create a resource group. All Azure resources must be deployed into a resource group. Resource groups allow you to organize and manage related Azure resources.
25
25
26
-
For this quickstart, create a resource group named *streamanalyticsrg* in the *eastus* location with the following [az group create](/cli/azure/group#az-group-create) command:
26
+
For this quickstart, create a resource group named **streamanalyticsrg** in the **eastus** location with the following [az group create](/cli/azure/group#az-group-create) command:
27
27
28
28
```azurecli
29
29
az group create --name streamanalyticsrg --location eastus
30
30
```
31
31
32
32
## Prepare the input data
33
33
34
-
Before you define the Stream Analytics job, prepare the data that's used for the job's input.
34
+
Before you define the Stream Analytics job, prepare the data that's used for the job's input. The following Azure CLI commands prepare the **input** data required by the job.
35
35
36
-
The following Azure CLI code blocks are commands that prepare the input data required by the job. Review the sections to understand the code.
37
-
38
-
1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. Because IoT Hub names are unique, you need to come up with your own IoT Hub name. Set the SKU to F1 to use the free tier if it's available with your subscription. If not, choose the next lowest tier.
36
+
1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. As IoT Hub names must be globally unique, you may have to change the name if it's already taken. Set the SKU to F1 to use the free tier if it's available with your subscription. If not, choose the next lowest tier.
39
37
40
38
```azurecli
41
-
az iot hub create --name "MyASAIoTHub" --resource-group streamanalyticsrg --sku S1
39
+
iotHubName=MyASAIoTHub
40
+
az iot hub create --name $iotHubName --resource-group streamanalyticsrg --sku S1
42
41
```
43
42
44
-
Once the IoT hub has been created, get the IoT Hub connection string using the [az iot hub show-connection-string](/cli/azure/iot/hub) command. Copy the entire connection string and save it for when you add the IoT Hub as input to your Stream Analytics job.
43
+
Once the IoT hub has been created, get the IoT Hub connection string using the [az iot hub connection-string show](/cli/azure/iot/hub/connection-string#az-iot-hub-connection-string-show) command. Copy the entire connection string and save it. You use it while adding the IoT Hub as an input to your Stream Analytics job.
45
44
46
45
```azurecli
47
-
az iot hub connection-string show --hub-name "MyASAIoTHub"
46
+
az iot hub connection-string show --hub-name $iotHubName
48
47
```
49
48
50
-
2. Add a device to IoT Hub using the [az iothub device-identity create](/cli/azure/iot/hub/device-identity#az-iot-hub-device-identity-create) command. This example creates a device called **MyASAIoTDevice**.
49
+
2. Add a device to IoT Hub using the [az iothub device-identity create](/cli/azure/iot/hub/device-identity#az-iot-hub-device-identity-create) command. This example creates a device called **MyASAIoTDevice**.
51
50
52
51
```azurecli
53
-
az iot hub device-identity create --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice"
52
+
az iot hub device-identity create --hub-name $iotHubName --device-id "MyASAIoTDevice"
54
53
```
55
54
56
55
3. Get the device connection string using the [az iot hub device-identity connection-string show](/cli/azure/iot/hub/device-identity/connection-string#az-iot-hub-device-identity-connection-string-show) command. Copy the entire connection string and save it for when you create the Raspberry Pi simulator.
57
56
58
57
```azurecli
59
-
az iot hub device-identity connection-string show --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice" --output table
58
+
az iot hub device-identity connection-string show --hub-name $iotHubName --device-id "MyASAIoTDevice" --output table
60
59
```
61
60
62
61
**Output example:**
@@ -67,16 +66,12 @@ The following Azure CLI code blocks are commands that prepare the input data req
67
66
68
67
## Create a blob storage account
69
68
70
-
The following Azure CLI code blocks create a blob storage account that's used for job output. Review the sections to understand the code.
71
-
72
-
1. Define a variable to hold the name for storage account. If this command gives an error, switch to the Bash shell in the Azure Cloud Shell.
69
+
The following Azure CLI commands create a blob **storage account** that's used for job **output**.
73
70
74
-
```azurecli
75
-
storageAccountName="asatutorialstorage$RANDOM"
76
-
```
77
71
1. Create a general-purpose storage account with the [az storage account create](/cli/azure/storage/account) command. The general-purpose storage account can be used for all four services: blobs, files, tables, and queues.
78
72
79
73
```azurecli
74
+
storageAccountName="asatutorialstorage$RANDOM"
80
75
az storage account create \
81
76
--name $storageAccountName \
82
77
--resource-group streamanalyticsrg \
@@ -92,7 +87,7 @@ The following Azure CLI code blocks create a blob storage account that's used fo
92
87
echo $key
93
88
```
94
89
95
-
> [!NOTE]
90
+
> [!IMPORTANT]
96
91
> Note down the access key for the Azure storage account. You will use this key later in this quickstart.
97
92
3. Create a container named `state` for storing blobs with the [az storage container create](/cli/azure/storage/container) command. You use the storage account key to authorize the operation to create the container. For more information about authorizing data operations with Azure CLI, see [Authorize access to blob or queue data with Azure CLI](../storage/blobs/authorize-data-operations-cli.md).
98
93
@@ -106,7 +101,7 @@ The following Azure CLI code blocks create a blob storage account that's used fo
106
101
107
102
## Create a Stream Analytics job
108
103
109
-
Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command. Review the next sections to understand the code
104
+
Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command.
110
105
111
106
```azurecli
112
107
az stream-analytics job create \
@@ -122,22 +117,26 @@ az stream-analytics job create \
122
117
123
118
## Configure input to the job
124
119
125
-
Add an input to your job by using the [az stream-analytics input](/cli/azure/stream-analytics/input#az-stream-analytics-input-create) cmdlet. This cmdlet takes the job name, job input name, resource group name, and the job input definition as parameters. The job input definition is a JSON file that contains the properties required to configure the job's input. In this example, you'll create an IoT Hub as an input.
120
+
Add an input to your job by using the [az stream-analytics input](/cli/azure/stream-analytics/input#az-stream-analytics-input-create) cmdlet. This cmdlet takes the job name, job input name, resource group name, and the input properties in JSON format as parameters. In this example, you'll create an IoT Hub as an input.
126
121
127
122
> [!IMPORTANT]
128
-
> Replace `IOT HUB ACCESS KEY` with the value of Shared Access Key in the IOT Hub connection string you saved. For example, if the IOT Hub connection string is: `HostName=MyASAIoTHub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxxx=`, the Shared Access Key value is `xxxxxxxxxxxxxx=`.
123
+
> - Replace `IOT HUB ACCESS KEY` with the value of Shared Access Key in the IOT Hub connection string you saved. For example, if the IOT Hub connection string is: `HostName=MyASAIoTHub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxxx=`, the Shared Access Key value is `xxxxxxxxxxxxxx=`. While replacing the value, make sure that you don't delete `\` (escape) character for `"` (double quotes).
124
+
> - Update the value of `iotHubNamespace` in the following command if you used a name other than `MyASAIoTHub`. Run `echo $iotHubName` to see the name of your IoT Hub.
Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, and the job output definition as parameters.
136
+
Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, data source in JSON format, and serialization type as parameters.
138
137
139
138
> [!IMPORTANT]
140
-
> Replace `STORAGEACCOUNTNAME>` with the name of your Azure Storage account and `STORAGEACCESSKEY>` with the access key for your storage account. If you didn't note down these values, run the following commands to get them: `echo $storageAccountName` and `echo $key`.
139
+
> Replace `STORAGEACCOUNTNAME>` with the name of your Azure Storage account and `STORAGEACCESSKEY>` with the access key for your storage account. If you didn't note down these values, run the following commands to get them: `echo $storageAccountName` and `echo $key`. While replacing the values, make sure that you don't delete `\` (escape) character for `"` (double quotes).
141
140
142
141
```azurecli
143
142
az stream-analytics output create \
@@ -150,9 +149,7 @@ az stream-analytics output create \
150
149
151
150
## Define the transformation query
152
151
153
-
Add a transformation your job by using the [az stream-analytics transformation create](/cli/azure/stream-analytics/transformation#az-stream-analytics-transformation-create) cmdlet. This cmdlet takes the job name, job transformation name, resource group name, and the job transformation definition as parameters.
154
-
155
-
Run the `az stream-analytics transformation create` cmdlet.
152
+
Add a transformation your job by using the [az stream-analytics transformation create](/cli/azure/stream-analytics/transformation#az-stream-analytics-transformation-create) cmdlet.
156
153
157
154
```azurecli
158
155
az stream-analytics transformation create \
@@ -166,7 +163,7 @@ az stream-analytics transformation create \
166
163
167
164
1. Open the [Raspberry Pi Azure IoT Online Simulator](https://azure-samples.github.io/raspberry-pi-web-simulator/).
168
165
169
-
2. Replace the placeholder in Line 15 with the entire Azure IoT Hub Device connection stringyou saved in a previous section.
166
+
2. Replace the placeholder in line 15 with the entire Azure IoT Hub **Device connection string** (not IoT Hub connection string) you saved at the beginning of the quickstart.
170
167
171
168
3. Select **Run**. The output should show the sensor data and messages that are being sent to your IoT Hub.
172
169
@@ -176,7 +173,7 @@ az stream-analytics transformation create \
176
173
177
174
Start the job by using the [az stream-analytics job start](/cli/azure/stream-analytics/job#az-stream-analytics-job-start) cmdlet. This cmdlet takes the job name, resource group name, output start mode, and start time as parameters. `OutputStartMode` accepts values of `JobStartTime`, `CustomTime`, or `LastOutputEventTime`.
178
175
179
-
After you run the following cmdlet, it returns `True` as output if the job starts. In the storage container, an output folder is created with the transformed data.
176
+
After you run the following cmdlet, it returns `True` as output if the job starts.
180
177
181
178
```azurecli
182
179
az stream-analytics job start \
@@ -185,7 +182,11 @@ az stream-analytics job start \
185
182
--output-start-mode JobStartTime
186
183
```
187
184
188
-
You see entries similar to the following one in the blob container.
185
+
Give it a few minutes and then verify that an output file is created in the `state` blob container.
186
+
187
+
:::image type="content" source="./media/stream-analytics-quick-create-powershell/output-file-container.png" alt-text="Screenshot showing the output file in the State blob container.":::
188
+
189
+
Download and open the file to see several entries similar to the following one:
189
190
190
191
```json
191
192
{
@@ -208,8 +209,7 @@ You see entries similar to the following one in the blob container.
208
209
```
209
210
210
211
## Clean up resources
211
-
212
-
When no longer needed, delete the resource group, the streaming job, and all related resources. Deleting the job avoids billing the streaming units consumed by the job. If you're planning to use the job in future, you can skip deleting it, and stop the job for now. If you aren't going to continue to use this job, delete all resources created by this quickstart by running the following cmdlet:
212
+
Delete the resource group, which will delete all the resources in the resource group including Stream Analytics job, IoT Hub, and Azure Storage account.
0 commit comments