Skip to content

Commit 24c83d7

Browse files
authored
Merge pull request #193525 from xujxu/update-asa-doc-fo-tooling
update docs for asa tooling
2 parents 2be1e0b + faed892 commit 24c83d7

10 files changed

+42
-28
lines changed

articles/stream-analytics/TOC.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
- name: Run a C# UDF
3838
href: stream-analytics-edge-csharp-udf.md
3939
- name: Create a custom .NET deserializer
40-
href: custom-deserializer.md
40+
href: visual-studio-code-custom-deserializer.md
4141
- name: Create an IoT Edge job
4242
href: ../iot-edge/tutorial-deploy-stream-analytics.md
4343
maintainContext: true
@@ -288,6 +288,8 @@
288288
href: stream-analytics-vs-tools.md
289289
- name: Debug queries using job diagram
290290
href: debug-locally-using-job-diagram.md
291+
- name: Create a custom .NET deserializer
292+
href: custom-deserializer.md
291293
- name: Develop an edge job
292294
href: stream-analytics-tools-for-visual-studio-edge-jobs.md
293295
- name: Set up CI/CD pipeline

articles/stream-analytics/cicd-overview.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ Follow the steps in this guide to create a CI/CD pipeline for Stream Analytics.
2323

2424
Use Azure Stream Analytics tools for [Visual Studio Code](./quick-create-visual-studio-code.md) or [Visual Studio](stream-analytics-quick-create-vs.md) to [develop and test queries locally](develop-locally.md). You can also [export an existing job](visual-studio-code-explore-jobs.md#export-a-job-to-a-local-project) to a local project.
2525

26+
> [!NOTE]
27+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
28+
2629
2. Commit your Azure Stream Analytics projects to your source control system, like a Git repository.
2730

2831
3. Use [Azure Stream Analytics CI/CD tools](cicd-tools.md) to build the projects and generate Azure Resource Manager templates for the deployment.

articles/stream-analytics/custom-deserializer.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
2-
title: Tutorial - Custom .NET deserializers for Azure Stream Analytics cloud jobs
3-
description: This tutorial demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio.
2+
title: Custom .NET deserializers for Azure Stream Analytics cloud jobs
3+
description: This doc demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio.
44
author: sidramadoss
55
ms.author: sidram
66
ms.service: stream-analytics
77
ms.topic: tutorial
88
ms.date: 12/17/2020
99
---
1010

11-
# Tutorial: Custom .NET deserializers for Azure Stream Analytics
11+
# Custom .NET deserializers for Azure Stream Analytics in Visual Studio
1212

1313
Azure Stream Analytics has [built-in support for three data formats](stream-analytics-parsing-json.md): JSON, CSV, and Avro. With custom .NET deserializers, you can read data from other formats such as [Protocol Buffer](https://developers.google.com/protocol-buffers/), [Bond](https://github.com/Microsoft/bond) and other user defined formats for both cloud and edge jobs.
1414

articles/stream-analytics/develop-locally.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ The environments in the following table support local development:
2424
|[Visual Studio 2019](stream-analytics-tools-for-visual-studio-install.md) |Stream Analytics Tools is part of the Azure development and Data storage and processing workloads in Visual Studio. You can use Visual Studio to write custom C# user-defined functions and deserializers. To learn more, see [Create an Azure Stream Analytics job by using Visual Studio](stream-analytics-quick-create-vs.md).|
2525
|[Command prompt or terminal](stream-analytics-tools-for-visual-studio-cicd.md)|The Azure Stream Analytics CI/CD NuGet package provides tools for Visual studio project build, local testing on an arbitrary machine. The Azure Stream Analytics CI/CD npm package provides tools for Visual Studio Code project builds (which generates an Azure Resource Manager template) on an arbitrary machine.|
2626

27+
> [!NOTE]
28+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
29+
2730
## Next steps
2831

2932
* [Test Stream Analytics queries locally with sample data using Visual Studio Code](visual-studio-code-local-run.md)

articles/stream-analytics/stream-analytics-add-inputs.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,10 @@ Stream Analytics has first-class integration with four kinds of resources as inp
1919

2020
These input resources can live in the same Azure subscription as your Stream Analytics job, or from a different subscription.
2121

22-
You can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-input), [Azure PowerShell](/powershell/module/az.streamanalytics/New-azStreamAnalyticsInput), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.inputsoperationsextensions), [REST API](/rest/api/streamanalytics/2020-03-01/inputs), and [Visual Studio](stream-analytics-tools-for-visual-studio-install.md) to create, edit, and test Stream Analytics job inputs.
22+
You can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-input), [Azure PowerShell](/powershell/module/az.streamanalytics/New-azStreamAnalyticsInput), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.inputsoperationsextensions), [REST API](/rest/api/streamanalytics/2020-03-01/inputs), [Visual Studio](stream-analytics-tools-for-visual-studio-install.md), and [Visual Studio Code](./quick-create-visual-studio-code.md) to create, edit, and test Stream Analytics job inputs.
23+
24+
> [!NOTE]
25+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
2326
2427
## Stream and reference inputs
2528
As data is pushed to a data source, it's consumed by the Stream Analytics job and processed in real time. Inputs are divided into two types: data stream inputs and reference data inputs.

articles/stream-analytics/stream-analytics-define-inputs.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ Stream Analytics supports compression across all data stream input sources. Supp
2626

2727
You can use the [Azure portal](stream-analytics-quick-create-portal.md), [Visual Studio](stream-analytics-quick-create-vs.md), and [Visual Studio Code](quick-create-visual-studio-code.md) to add and view or edit existing inputs on your streaming job. You can also test input connections and test queries from sample data from the Azure portal, [Visual Studio](stream-analytics-vs-tools-local-run.md), and [Visual Studio Code](visual-studio-code-local-run.md). When you write a query, you list the input in the FROM clause. You can get the list of available inputs from the **Query** page in the portal. If you wish to use multiple inputs, you can `JOIN` them or write multiple `SELECT` queries.
2828

29+
> [!NOTE]
30+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
2931
3032
## Stream data from Event Hubs
3133

articles/stream-analytics/stream-analytics-define-outputs.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,10 @@ ms.date: 01/14/2022
1313

1414
An Azure Stream Analytics job consists of an input, query, and an output. There are several output types to which you can send transformed data. This article lists the supported Stream Analytics outputs. When you design your Stream Analytics query, refer to the name of the output by using the [INTO clause](/stream-analytics-query/into-azure-stream-analytics). You can use a single output per job, or multiple outputs per streaming job (if you need them) by adding multiple INTO clauses to the query.
1515

16-
To create, edit, and test Stream Analytics job outputs, you can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-output), [Azure PowerShell](stream-analytics-quick-create-powershell.md#configure-output-to-the-job), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.ioutputsoperations), [REST API](/rest/api/streamanalytics/), and [Visual Studio](stream-analytics-quick-create-vs.md).
16+
To create, edit, and test Stream Analytics job outputs, you can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-output), [Azure PowerShell](stream-analytics-quick-create-powershell.md#configure-output-to-the-job), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.ioutputsoperations), [REST API](/rest/api/streamanalytics/), [Visual Studio](stream-analytics-quick-create-vs.md), and [Visual Studio Code](./quick-create-visual-studio-code.md).
17+
18+
> [!NOTE]
19+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
1720
1821
Some outputs types support [partitioning](#partitioning), and [output batch sizes](#output-batch-size) vary to optimize throughput. The following table shows features that are supported for each output type:
1922

articles/stream-analytics/stream-analytics-previews.md

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,3 @@ You can use the job diagram while testing your query locally to examine the inte
3030
## Explore jobs in Visual Studio Code
3131

3232
Stream Analytics Explorer in Visual Studio Code Extension gives developers a lightweight experience for managing their Stream Analytics jobs. In the Stream Analytics Explorer, you can easily manage your jobs, view job diagram, and debug in Job Monitor.
33-
34-
## Debug query steps in Visual Studio
35-
36-
You can easily preview the intermediate row set on a data diagram when doing local testing in Azure Stream Analytics tools for Visual Studio.
37-
38-
39-
## Live data testing in Visual Studio
40-
41-
Visual Studio tools for Azure Stream Analytics enhance the local testing feature that allows you to test you queries against live event streams from cloud sources such as Event Hub or IoT hub. Learn how to [Test live data locally using Azure Stream Analytics tools for Visual Studio](stream-analytics-live-data-local-testing.md).
42-
43-

articles/stream-analytics/stream-analytics-tools-for-visual-studio-install.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ Visual Studio 2019 and Visual Studio 2017 support Azure Data Lake and Stream Ana
1414

1515
For more information on using the tools, see [Quickstart: Create an Azure Stream Analytics job by using Visual Studio](stream-analytics-quick-create-vs.md).
1616

17+
> [!NOTE]
18+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
19+
1720
## Install
1821

1922
Visual Studio Enterprise (Ultimate/Premium), Professional, and Community editions support the tools. Express edition and Visual Studio for Mac don't support them.

articles/stream-analytics/visual-studio-code-custom-deserializer.md

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Create custom .NET deserializers for Azure Stream Analytics cloud jobs using Visual Studio Code
2+
title: Tutorial - Create custom .NET deserializers for Azure Stream Analytics cloud jobs using Visual Studio Code
33
description: This tutorial demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code.
44
author: su-jie
55
ms.author: sujie
@@ -9,21 +9,27 @@ ms.date: 12/22/2020
99
---
1010

1111

12-
# Create custom .NET deserializers for Azure Stream Analytics in Visual Studio Code
12+
# Tutorial: Custom .NET deserializers for Azure Stream Analytics in Visual Studio Code
1313

1414
Azure Stream Analytics has [built-in support for three data formats](stream-analytics-parsing-json.md): JSON, CSV, and Avro. With custom .NET deserializers, you can read data from other formats such as [Protocol Buffer](https://developers.google.com/protocol-buffers/), [Bond](https://github.com/Microsoft/bond) and other user defined formats for cloud jobs.
1515

16-
## Custom .NET deserializers in Visual Studio Code
16+
This tutorial demonstrates how to create, test, and debug a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code. To learn how to create .NET deserializers in Visual Studio, see [Create .NET deserializers for Azure Stream Analytics jobs in Visual Studio](custom-deserializer.md).
1717

18-
You can create, test and debug a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code.
18+
In this tutorial, you learn how to:
1919

20-
### Prerequisites
20+
> [!div class="checklist"]
21+
> * Create a custom deserializer for protocol buffer.
22+
> * Create an Azure Stream Analytics job in Visual Studio.
23+
> * Configure your Stream Analytics job to use the custom deserializer.
24+
> * Run your Stream Analytics job locally to test and debug the custom deserializer.
25+
26+
## Prerequisites
2127

2228
* Install [.NET core SDK](https://dotnet.microsoft.com/download) and restart Visual Studio Code.
2329

2430
* Use this [quickstart](quick-create-visual-studio-code.md) to learn how to create a Stream Analytics job using Visual Studio Code.
2531

26-
### Create a custom deserializer
32+
## Create a custom deserializer
2733

2834
1. Open a terminal and run following command to create a .NET class library in Visual Studio Code for your custom deserializer called **ProtobufDeserializer**.
2935

@@ -45,11 +51,11 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
4551

4652
4. Build the **ProtobufDeserializer** project.
4753

48-
### Add an Azure Stream Analytics project
54+
## Add an Azure Stream Analytics project
4955

50-
1. Open Visual Studio Code and select **Ctrl+Shift+P** to open the command palette. Then enter ASA and select **ASA: Create New Project**. Name it **ProtobufCloudDeserializer**.
56+
Open Visual Studio Code and select **Ctrl+Shift+P** to open the command palette. Then enter ASA and select **ASA: Create New Project**. Name it **ProtobufCloudDeserializer**.
5157

52-
### Configure a Stream Analytics job
58+
## Configure a Stream Analytics job
5359

5460
1. Double-click **JobConfig.json**. Use the default configurations, except for the following settings:
5561

@@ -88,7 +94,7 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
8894
|-------|---------------|
8995
|Select local file path|Click CodeLens to select < The file path for the downloaded sample protobuf input file>|
9096

91-
### Execute the Stream Analytics job
97+
## Execute the Stream Analytics job
9298

9399
1. Open **ProtobufCloudDeserializer.asaql** and select **Run Locally** from CodeLens then choose **Use Local Input** from the dropdown list.
94100

@@ -98,7 +104,7 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
98104

99105
You have successfully implemented a custom deserializer for your Stream Analytics job! In this tutorial, you tested the custom deserializer locally with local input data. You can also test it [using live data input in the cloud](visual-studio-code-local-run-live-input.md). For running the job in the cloud, you would properly configure the input and output. Then submit the job to Azure from Visual Studio Code to run your job in the cloud using the custom deserializer you just implemented.
100106

101-
### Debug your deserializer
107+
## Debug your deserializer
102108

103109
You can debug your .NET deserializer locally the same way you debug standard .NET code.
104110

0 commit comments

Comments
 (0)