Skip to content

Commit 9c8457e

Browse files
committed
final version
1 parent 094cc8d commit 9c8457e

File tree

3 files changed

+33
-33
lines changed

3 files changed

+33
-33
lines changed
41.8 KB
Loading
41.8 KB
Loading

articles/stream-analytics/quick-create-azure-cli.md

Lines changed: 33 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ ms.date: 02/28/2023
1313
---
1414

1515
# Quickstart: Create an Azure Stream Analytics job using the Azure CLI
16-
In this quickstart, you will use Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. The Stream Analytics job reads data from IoT Hub, transforms the data, and writes the data back to a container in a blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
16+
In this quickstart, you will use Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. The Stream Analytics job reads data from IoT Hub, transforms the data, and writes the output data to a container in a blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
1717

1818
## Before you begin
1919

@@ -23,40 +23,39 @@ In this quickstart, you will use Azure CLI to define a Stream Analytics job that
2323

2424
- Create a resource group. All Azure resources must be deployed into a resource group. Resource groups allow you to organize and manage related Azure resources.
2525

26-
For this quickstart, create a resource group named *streamanalyticsrg* in the *eastus* location with the following [az group create](/cli/azure/group#az-group-create) command:
26+
For this quickstart, create a resource group named **streamanalyticsrg** in the **eastus** location with the following [az group create](/cli/azure/group#az-group-create) command:
2727

2828
```azurecli
2929
az group create --name streamanalyticsrg --location eastus
3030
```
3131

3232
## Prepare the input data
3333

34-
Before you define the Stream Analytics job, prepare the data that's used for the job's input.
34+
Before you define the Stream Analytics job, prepare the data that's used for the job's input. The following Azure CLI commands prepare the **input** data required by the job.
3535

36-
The following Azure CLI code blocks are commands that prepare the input data required by the job. Review the sections to understand the code.
37-
38-
1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. Because IoT Hub names are unique, you need to come up with your own IoT Hub name. Set the SKU to F1 to use the free tier if it's available with your subscription. If not, choose the next lowest tier.
36+
1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. As IoT Hub names must be globally unique, you may have to change the name if it's already taken. Set the SKU to F1 to use the free tier if it's available with your subscription. If not, choose the next lowest tier.
3937

4038
```azurecli
41-
az iot hub create --name "MyASAIoTHub" --resource-group streamanalyticsrg --sku S1
39+
iotHubName=MyASAIoTHub
40+
az iot hub create --name $iotHubName --resource-group streamanalyticsrg --sku S1
4241
```
4342
44-
Once the IoT hub has been created, get the IoT Hub connection string using the [az iot hub show-connection-string](/cli/azure/iot/hub) command. Copy the entire connection string and save it for when you add the IoT Hub as input to your Stream Analytics job.
43+
Once the IoT hub has been created, get the IoT Hub connection string using the [az iot hub connection-string show](/cli/azure/iot/hub/connection-string#az-iot-hub-connection-string-show) command. Copy the entire connection string and save it. You use it while adding the IoT Hub as an input to your Stream Analytics job.
4544
4645
```azurecli
47-
az iot hub connection-string show --hub-name "MyASAIoTHub"
46+
az iot hub connection-string show --hub-name $iotHubName
4847
```
4948
50-
2. Add a device to IoT Hub using the [az iothub device-identity create](/cli/azure/iot/hub/device-identity#az-iot-hub-device-identity-create) command. This example creates a device called **MyASAIoTDevice**.
49+
2. Add a device to IoT Hub using the [az iothub device-identity create](/cli/azure/iot/hub/device-identity#az-iot-hub-device-identity-create) command. This example creates a device called **MyASAIoTDevice**.
5150
5251
```azurecli
53-
az iot hub device-identity create --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice"
52+
az iot hub device-identity create --hub-name $iotHubName --device-id "MyASAIoTDevice"
5453
```
5554
5655
3. Get the device connection string using the [az iot hub device-identity connection-string show](/cli/azure/iot/hub/device-identity/connection-string#az-iot-hub-device-identity-connection-string-show) command. Copy the entire connection string and save it for when you create the Raspberry Pi simulator.
5756
5857
```azurecli
59-
az iot hub device-identity connection-string show --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice" --output table
58+
az iot hub device-identity connection-string show --hub-name $iotHubName --device-id "MyASAIoTDevice" --output table
6059
```
6160
6261
**Output example:**
@@ -67,16 +66,12 @@ The following Azure CLI code blocks are commands that prepare the input data req
6766
6867
## Create a blob storage account
6968
70-
The following Azure CLI code blocks create a blob storage account that's used for job output. Review the sections to understand the code.
71-
72-
1. Define a variable to hold the name for storage account. If this command gives an error, switch to the Bash shell in the Azure Cloud Shell.
69+
The following Azure CLI commands create a blob **storage account** that's used for job **output**.
7370
74-
```azurecli
75-
storageAccountName="asatutorialstorage$RANDOM"
76-
```
7771
1. Create a general-purpose storage account with the [az storage account create](/cli/azure/storage/account) command. The general-purpose storage account can be used for all four services: blobs, files, tables, and queues.
7872
7973
```azurecli
74+
storageAccountName="asatutorialstorage$RANDOM"
8075
az storage account create \
8176
--name $storageAccountName \
8277
--resource-group streamanalyticsrg \
@@ -92,7 +87,7 @@ The following Azure CLI code blocks create a blob storage account that's used fo
9287
echo $key
9388
```
9489
95-
> [!NOTE]
90+
> [!IMPORTANT]
9691
> Note down the access key for the Azure storage account. You will use this key later in this quickstart.
9792
3. Create a container named `state` for storing blobs with the [az storage container create](/cli/azure/storage/container) command. You use the storage account key to authorize the operation to create the container. For more information about authorizing data operations with Azure CLI, see [Authorize access to blob or queue data with Azure CLI](../storage/blobs/authorize-data-operations-cli.md).
9893
@@ -106,7 +101,7 @@ The following Azure CLI code blocks create a blob storage account that's used fo
106101

107102
## Create a Stream Analytics job
108103

109-
Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command. Review the next sections to understand the code
104+
Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command.
110105

111106
```azurecli
112107
az stream-analytics job create \
@@ -122,22 +117,26 @@ az stream-analytics job create \
122117

123118
## Configure input to the job
124119

125-
Add an input to your job by using the [az stream-analytics input](/cli/azure/stream-analytics/input#az-stream-analytics-input-create) cmdlet. This cmdlet takes the job name, job input name, resource group name, and the job input definition as parameters. The job input definition is a JSON file that contains the properties required to configure the job's input. In this example, you'll create an IoT Hub as an input.
120+
Add an input to your job by using the [az stream-analytics input](/cli/azure/stream-analytics/input#az-stream-analytics-input-create) cmdlet. This cmdlet takes the job name, job input name, resource group name, and the input properties in JSON format as parameters. In this example, you'll create an IoT Hub as an input.
126121

127122
> [!IMPORTANT]
128-
> Replace `IOT HUB ACCESS KEY` with the value of Shared Access Key in the IOT Hub connection string you saved. For example, if the IOT Hub connection string is: `HostName=MyASAIoTHub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxxx=`, the Shared Access Key value is `xxxxxxxxxxxxxx=`.
123+
> - Replace `IOT HUB ACCESS KEY` with the value of Shared Access Key in the IOT Hub connection string you saved. For example, if the IOT Hub connection string is: `HostName=MyASAIoTHub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxxx=`, the Shared Access Key value is `xxxxxxxxxxxxxx=`. While replacing the value, make sure that you don't delete `\` (escape) character for `"` (double quotes).
124+
> - Update the value of `iotHubNamespace` in the following command if you used a name other than `MyASAIoTHub`. Run `echo $iotHubName` to see the name of your IoT Hub.
129125
130126
```azurecli
131127
az stream-analytics input create \
132-
--properties "{\"type\":\"Stream\",\"datasource\":{\"type\":\"Microsoft.Devices/IotHubs\",\"properties\":{\"consumerGroupName\":\"\sdkconsumergroup\\",\"endpoint\":\"messages/events\",\"iotHubNamespace\":\"MyASAIoTHub\",\"sharedAccessPolicyKey\":\"IOT HUB ACCESS KEY\",\"sharedAccessPolicyName\":\"iothubowner\"}},\"serialization\":{\"type\":\"Json\",\"encoding\":\"UTF8\"}}" --input-name "asaiotinput" --job-name "streamanalyticsjob" --resource-group "streamanalyticsrg"
128+
--properties "{\"type\":\"Stream\",\"datasource\":{\"type\":\"Microsoft.Devices/IotHubs\",\"properties\":{\"consumerGroupName\":\"\$Default\",\"endpoint\":\"messages/events\",\"iotHubNamespace\":\"MyASAIoTHub\",\"sharedAccessPolicyKey\":\"IOT HUB ACCESS KEY\",\"sharedAccessPolicyName\":\"iothubowner\"}},\"serialization\":{\"type\":\"Json\",\"encoding\":\"UTF8\"}}" \
129+
--input-name "asaiotinput" \
130+
--job-name "streamanalyticsjob" \
131+
--resource-group "streamanalyticsrg"
133132
```
134133

135134
## Configure output to the job
136135

137-
Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, and the job output definition as parameters.
136+
Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, data source in JSON format, and serialization type as parameters.
138137

139138
> [!IMPORTANT]
140-
> Replace `STORAGEACCOUNTNAME>` with the name of your Azure Storage account and `STORAGEACCESSKEY>` with the access key for your storage account. If you didn't note down these values, run the following commands to get them: `echo $storageAccountName` and `echo $key`.
139+
> Replace `STORAGEACCOUNTNAME>` with the name of your Azure Storage account and `STORAGEACCESSKEY>` with the access key for your storage account. If you didn't note down these values, run the following commands to get them: `echo $storageAccountName` and `echo $key`. While replacing the values, make sure that you don't delete `\` (escape) character for `"` (double quotes).
141140
142141
```azurecli
143142
az stream-analytics output create \
@@ -150,9 +149,7 @@ az stream-analytics output create \
150149

151150
## Define the transformation query
152151

153-
Add a transformation your job by using the [az stream-analytics transformation create](/cli/azure/stream-analytics/transformation#az-stream-analytics-transformation-create) cmdlet. This cmdlet takes the job name, job transformation name, resource group name, and the job transformation definition as parameters.
154-
155-
Run the `az stream-analytics transformation create` cmdlet.
152+
Add a transformation your job by using the [az stream-analytics transformation create](/cli/azure/stream-analytics/transformation#az-stream-analytics-transformation-create) cmdlet.
156153

157154
```azurecli
158155
az stream-analytics transformation create \
@@ -166,7 +163,7 @@ az stream-analytics transformation create \
166163

167164
1. Open the [Raspberry Pi Azure IoT Online Simulator](https://azure-samples.github.io/raspberry-pi-web-simulator/).
168165

169-
2. Replace the placeholder in Line 15 with the entire Azure IoT Hub Device connection string you saved in a previous section.
166+
2. Replace the placeholder in line 15 with the entire Azure IoT Hub **Device connection string** (not IoT Hub connection string) you saved at the beginning of the quickstart.
170167

171168
3. Select **Run**. The output should show the sensor data and messages that are being sent to your IoT Hub.
172169

@@ -176,7 +173,7 @@ az stream-analytics transformation create \
176173

177174
Start the job by using the [az stream-analytics job start](/cli/azure/stream-analytics/job#az-stream-analytics-job-start) cmdlet. This cmdlet takes the job name, resource group name, output start mode, and start time as parameters. `OutputStartMode` accepts values of `JobStartTime`, `CustomTime`, or `LastOutputEventTime`.
178175

179-
After you run the following cmdlet, it returns `True` as output if the job starts. In the storage container, an output folder is created with the transformed data.
176+
After you run the following cmdlet, it returns `True` as output if the job starts.
180177

181178
```azurecli
182179
az stream-analytics job start \
@@ -185,7 +182,11 @@ az stream-analytics job start \
185182
--output-start-mode JobStartTime
186183
```
187184

188-
You see entries similar to the following one in the blob container.
185+
Give it a few minutes and then verify that an output file is created in the `state` blob container.
186+
187+
:::image type="content" source="./media/stream-analytics-quick-create-powershell/output-file-container.png" alt-text="Screenshot showing the output file in the State blob container.":::
188+
189+
Download and open the file to see several entries similar to the following one:
189190

190191
```json
191192
{
@@ -208,8 +209,7 @@ You see entries similar to the following one in the blob container.
208209
```
209210

210211
## Clean up resources
211-
212-
When no longer needed, delete the resource group, the streaming job, and all related resources. Deleting the job avoids billing the streaming units consumed by the job. If you're planning to use the job in future, you can skip deleting it, and stop the job for now. If you aren't going to continue to use this job, delete all resources created by this quickstart by running the following cmdlet:
212+
Delete the resource group, which will delete all the resources in the resource group including Stream Analytics job, IoT Hub, and Azure Storage account.
213213

214214
```azurecli
215215
az group delete \

0 commit comments

Comments
 (0)