ASA - Inputs & CLI QS

spelluru · spelluru · commit ab6bf69b9f8a · 2023-02-28T15:25:37.000-05:00
diff --git a/articles/stream-analytics/quick-create-azure-cli.md b/articles/stream-analytics/quick-create-azure-cli.md
@@ -9,12 +9,11 @@ ms.reviewer: jasonh
 ms.workload: big-data
 ms.topic: quickstart
 ms.custom: mvc, devx-track-azurecli, mode-api
-ms.date: 07/01/2020
+ms.date: 02/28/2023
 ---
 
 # Quickstart: Create an Azure Stream Analytics job using the Azure CLI
-
-In this quickstart, you use the Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. Your Stream Analytics job will read data from IoT Hub, transform the data, and write the data back to a container in blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
+In this quickstart, you will use Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. The Stream Analytics job reads data from IoT Hub, transforms the data, and writes the data back to a container in a blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.
 
 ## Before you begin
 
@@ -36,7 +35,7 @@ Before you define the Stream Analytics job, prepare the data that's used for the
 
 The following Azure CLI code blocks are commands that prepare the input data required by the job. Review the sections to understand the code.
 
-1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. Because IoT Hub names are unique, you need to come up with your own IoT Hub name. Set the SKU to F1 to use the free tier if it is available with your subscription. If not, choose the next lowest tier.
+1. Create an IoT Hub using the [az iot hub create](/cli/azure/iot/hub#az-iot-hub-create) command. This example creates an IoT Hub called **MyASAIoTHub**. Because IoT Hub names are unique, you need to come up with your own IoT Hub name. Set the SKU to F1 to use the free tier if it's available with your subscription. If not, choose the next lowest tier.
 
     ```azurecli
     az iot hub create --name "MyASAIoTHub" --resource-group streamanalyticsrg --sku S1
@@ -45,7 +44,7 @@ The following Azure CLI code blocks are commands that prepare the input data req
     Once the IoT hub has been created, get the IoT Hub connection string using the [az iot hub show-connection-string](/cli/azure/iot/hub) command. Copy the entire connection string and save it for when you add the IoT Hub as input to your Stream Analytics job.
 
     ```azurecli
-    az iot hub show-connection-string --hub-name "MyASAIoTHub"
+    az iot hub connection-string show --hub-name "MyASAIoTHub"
     ```
 
 2. Add a device to IoT Hub using the [az iothub device-identity create](/cli/azure/iot/hub/device-identity#az-iot-hub-device-identity-create) command. This example creates a device called **MyASAIoTDevice**.
@@ -70,128 +69,74 @@ The following Azure CLI code blocks are commands that prepare the input data req
 
 The following Azure CLI code blocks create a blob storage account that's used for job output. Review the sections to understand the code.
 
-1. Create a general-purpose storage account with the [az storage account create](/cli/azure/storage/account) command. The general-purpose storage account can be used for all four services: blobs, files, tables, and queues.
+1. Define a variable to hold the name for storage account.
 
-   Remember to replace placeholder values in angle brackets with your own values:
+    ```azurecli
+    storageAccountName="asatutorialstorage$RANDOM"
+    ```
+1. Create a general-purpose storage account with the [az storage account create](/cli/azure/storage/account) command. The general-purpose storage account can be used for all four services: blobs, files, tables, and queues.
 
    ```azurecli
    az storage account create \
-       --name <storage-account> \
+       --name $storageAccountName \
        --resource-group streamanalyticsrg \
        --location eastus \
        --sku Standard_ZRS \
        --encryption-services blob
    ```
 
-2. Get the key for your storage account by running the [az storage account keys list](/cli/azure/storage/account/keys) command. Save this key to use in the next step.
+2. Get the key for your storage account by running the [az storage account keys list](/cli/azure/storage/account/keys) command. 
 
    ```azurecli
-   az storage account keys list -g streamanalyticsrg -n <storage-account>
+   key=$(az storage account keys list -g streamanalyticsrg -n $storageAccountName --query "[0].value" -o tsv)
    ```
 
 3. Create a container for storing blobs with the [az storage container create](/cli/azure/storage/container) command. You use the storage account key to authorize the operation to create the container. For more information about authorizing data operations with Azure CLI, see [Authorize access to blob or queue data with Azure CLI](../storage/blobs/authorize-data-operations-cli.md).
 
    ```azurecli
    az storage container create \
-       --account-name <storage-account> \
+       --account-name $storageAccountName \
        --name sample-container \
-       --account-key <key>
+       --account-key $key \
        --auth-mode key
    ```
 
 ## Create a Stream Analytics job
 
-The following Azure CLI code blocks create a Stream Analytics job. Review the sections to understand the code
-
-1. Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command.
+Create a Stream Analytics job with the [az stream-analytics job create](/cli/azure/stream-analytics/job#az-stream-analytics-job-create) command. Review the next sections to understand the code
 
 ```azurecli
 az stream-analytics job create \
-    --resource-group streamanalyticsrg 
-    --name streamanalyticsjob \
-    --location eastus \
+    --job-name "streamanalyticsjob" \
+    --resource-group "streamanalyticsrg" \
+    --location "eastus" \
     --output-error-policy "Drop" \
-    --events-outoforder-policy "Drop" \
-    --events-outoforder-max-delay 5 \
-    --events-late-arrival-max-delay 16 \     
+    --out-of-order-policy "Drop" \
+    --order-max-delay 5 \
+    --arrival-max-delay 16 \
     --data-locale "en-US"
 ```
 
 ## Configure input to the job
 
 Add an input to your job by using the [az stream-analytics input](/cli/azure/stream-analytics/input#az-stream-analytics-input-create) cmdlet. This cmdlet takes the job name, job input name, resource group name, and the job input definition as parameters. The job input definition is a JSON file that contains the properties required to configure the job's input. In this example, you'll create an IoT Hub as an input.
 
-On your local machine, create a file named `datasource.json` and add the following JSON data to it. Make sure to replace the value for `sharedAccessPolicyKey` with the `SharedAccessKey` portion of the IoT Hub connection string you saved in a previous section.
-
-```json
-{
-    "type": "Microsoft.Devices/IotHubs",
-    "properties": {
-        "iotHubNamespace": "iothub",
-        "sharedAccessPolicyName": "iothubowner",
-        "sharedAccessPolicyKey": "sharedAccessPolicyKey=",
-        "consumerGroupName": "sdkconsumergroup",
-        "endpoint": "messages/events"
-    }
-}
-```
-
-On your local machine, create a file named `serialization.json` and add the following JSON data to it.
-
-```json
-{
-     "type": "Json",
-     "properties": {
-         "encoding": "UTF8"
-     }
-}
-```
-
-Next, run the `az stream-analytics input create` cmdlet. Be sure to replace the value of `datasource` variable with the path where you've stored the job input definition JSON file, and the value of `serialization` variable with the path where you've stored the serialization JSON file.
+> [!IMPORTANT]
+> Replace `<IOT HUB ACCESS KEY>` with the value of Shared Access Key in the IOT Hub connection string you saved. For example, if the IOT Hub connection string is: `HostName=MyASAIoTHub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxxx=`, the Shared Access Key value is `xxxxxxxxxxxxxx=`.
 
 ```azurecli
-az stream-analytics input create \
-    --resource-group streamanalyticsrg 
-    --job-name streamanalyticsjob \
-    --name asaiotinput \
-    --type Stream \
-    --datasource datasource.json \
-    --serialization serialization.json
+az stream-analytics input create --properties "{\"type\":\"Stream\",\"datasource\":{\"type\":\"Microsoft.Devices/IotHubs\",\"properties\":{\"consumerGroupName\":\"$Default\",\"endpoint\":\"messages/events\",\"iotHubNamespace\":\"iothub\",\"sharedAccessPolicyKey\":\"<IOT HUB ACCESS KEY>",\"sharedAccessPolicyName\":\"iothubowner\"}},\"serialization\":{\"type\":\"Json\",\"encoding\":\"UTF8\"}}" --input-name "asaiotinput" --job-name "streamanalyticsjob" --resource-group "streamanalyticsrg"
 ```
 
 ## Configure output to the job
 
-Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, and the job output definition as parameters. The job output definition is a JSON file that contains the properties required to configure job's output. This example uses blob storage as output.
-
-On your local machine, create a file named `datasink.json`, and add the following JSON data to it. Make sure to replace the value for `accountKey` with your storage account's access key that is the value stored in $storageAccountKey value.
-
-```json
-{
-    "type": "Microsoft.Storage/Blob",
-    "properties": {
-        "storageAccounts": [
-            {
-                "accountName": "<storage-account>",
-                "accountKey": "accountKey=="
-            }
-        ],
-        "container": "state",
-        "pathPattern": "{date}/{time}",
-        "dateFormat": "yyyy/MM/dd",
-        "timeFormat": "HH"
-    }
-}
-```
+Add an output to your job by using the [az stream-analytics output create](/cli/azure/stream-analytics/output#az-stream-analytics-output-create) cmdlet. This cmdlet takes the job name, job output name, resource group name, and the job output definition as parameters.
 
-Next, run the `az stream-analytics output` cmdlet. Be sure to replace the value of `datasource` variable with the path where you've stored the job output definition JSON file, and the value of `serialization` variable with the path where you've stored the serialization JSON file.
+> [!IMPORTANT]
+> Replace `<STORAGE ACCOUNT NAME>` with the name of your Azure Storage account and `<ACCESS KEY>` with the access key for your storage account. 
 
 ```azurecli
-az stream-analytics output create \
-    --resource-group streamanalyticsrg \
-    --job-name streamanalyticsjob \
-    --name asabloboutput \
-    --datasource datasink.json \
-    --serialization serialization.json
+az stream-analytics output create --job-name streamanalyticsjob --datasource "{\"type\":\"Microsoft.Storage/Blob\",\"properties\":{\"container\":\"state\",\"dateFormat\":\"yyyy/MM/dd\",\"pathPattern\":\"{date}/{time}\",\"storageAccounts\":[{\"accountKey\":\"<ACCESS KEY>",\"accountName\":\"<STORAGE ACCOUNT NAME>\"}],\"timeFormat\":\"HH\"}}" --serialization "{\"type\":\"Json\",\"properties\":{\"format\":\"Array\",\"encoding\":\"UTF8\"}}" --output-name asabloboutput --resource-group streamanalyticsrg
 ```
 
 ## Define the transformation query
@@ -206,15 +151,15 @@ az stream-analytics transformation create \
     --job-name streamanalyticsjob \
     --name Transformation \
     --streaming-units "6" \
-    --transformation-query "SELECT * INTO asabloboutput FROM asaiotinput WHERE Temperature > 27"
+    --saql "SELECT * INTO asabloboutput FROM asaiotinput WHERE Temperature > 27"
 ```
 ## Run the IoT simulator
 
 1. Open the [Raspberry Pi Azure IoT Online Simulator](https://azure-samples.github.io/raspberry-pi-web-simulator/).
 
 2. Replace the placeholder in Line 15 with the entire Azure IoT Hub Device connection string you saved in a previous section.
 
-3. Click **Run**. The output should show the sensor data and messages that are being sent to your IoT Hub.
+3. Select **Run**. The output should show the sensor data and messages that are being sent to your IoT Hub.
 
     ![Raspberry Pi Azure IoT Online Simulator](./media/stream-analytics-quick-create-powershell/ras-pi-connection-string.png)
 
diff --git a/articles/stream-analytics/stream-analytics-add-inputs.md b/articles/stream-analytics/stream-analytics-add-inputs.md
@@ -1,11 +1,11 @@
 ---
 title: Understand inputs for Azure Stream Analytics
-description: This article describe the concept of inputs in an Azure Stream Analytics job, comparing streaming input to reference data input.
+description: This article describes the concept of inputs in an Azure Stream Analytics job, comparing streaming input to reference data input.
 ms.service: stream-analytics
 author: enkrumah
 ms.author: ebnkruma
 ms.topic: conceptual
-ms.date: 10/29/2020
+ms.date: 02/28/2023
 ---
 # Understand inputs for Azure Stream Analytics
 
@@ -25,15 +25,18 @@ You can use the [Azure portal](stream-analytics-quick-create-portal.md#configure
 > We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
 
 ## Stream and reference inputs
-As data is pushed to a data source, it's consumed by the Stream Analytics job and processed in real time. Inputs are divided into two types: data stream inputs and reference data inputs.
+As data is pushed to a data source, it's consumed by the Stream Analytics job and processed in real time. Inputs are divided into two types:
+
+- Data stream inputs
+- Reference data inputs.
 
 ### Data stream input
-A data stream is an unbounded sequence of events over time. Stream Analytics jobs must include at least one data stream input. Event Hubs, IoT Hub, Azure Data Lake Storage Gen2 and Blob storage are supported as data stream input sources. Event Hubs are used to collect event streams from multiple devices and services. These streams might include social media activity feeds, stock trade information, or data from sensors. IoT Hubs are optimized to collect data from connected devices in Internet of Things (IoT) scenarios.  Blob storage can be used as an input source for ingesting bulk data as a stream, such as log files.  
+A data stream is an unbounded sequence of events over time. Stream Analytics jobs must include at least one data stream input. Event Hubs, IoT Hub, Azure Data Lake Storage Gen2 and Blob storage are supported as data stream input sources. Event Hubs is used to collect event streams from multiple devices and services. These streams might include social media activity feeds, stock trade information, or data from sensors. IoT Hubs are optimized to collect data from connected devices in Internet of Things (IoT) scenarios.  Blob storage can be used as an input source for ingesting bulk data as a stream, such as log files.  
 
 For more information about streaming data inputs, see [Stream data as input into Stream Analytics](stream-analytics-define-inputs.md)
 
 ### Reference data input
-Stream Analytics also supports input known as *reference data*. Reference data is either completely static or changes slowly. It is typically used to perform correlation and lookups. For example, you might join data in the data stream input to data in the reference data, much as you would perform a SQL join to look up static values. Azure Blob storage, Azure Data Lake Storage Gen2, and Azure SQL Database are currently supported as input sources for reference data. Reference data source blobs have a limit of up to 300 MB in size, depending on the query complexity and allocated Streaming Units (see the [Size limitation](stream-analytics-use-reference-data.md#size-limitation) section of the reference data documentation for more details).
+Stream Analytics also supports input known as *reference data*. Reference data is either completely static or changes slowly. It's typically used to perform correlation and lookups. For example, you might join data in the data stream input to data in the reference data, much as you would perform a SQL join to look up static values. Azure Blob storage, Azure Data Lake Storage Gen2, and Azure SQL Database are currently supported as input sources for reference data. Reference data source blobs have a limit of up to 300 MB in size, depending on the query complexity and allocated Streaming Units. For more information, see the [Size limitation](stream-analytics-use-reference-data.md#size-limitation) section of the reference data documentation.
 
 For more information about reference data inputs, see [Using reference data for lookups in Stream Analytics](stream-analytics-use-reference-data.md)
 
