You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/cicd-overview.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,6 +23,9 @@ Follow the steps in this guide to create a CI/CD pipeline for Stream Analytics.
23
23
24
24
Use Azure Stream Analytics tools for [Visual Studio Code](./quick-create-visual-studio-code.md) or [Visual Studio](stream-analytics-quick-create-vs.md) to [develop and test queries locally](develop-locally.md). You can also [export an existing job](visual-studio-code-explore-jobs.md#export-a-job-to-a-local-project) to a local project.
25
25
26
+
> [!NOTE]
27
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
28
+
26
29
2. Commit your Azure Stream Analytics projects to your source control system, like a Git repository.
27
30
28
31
3. Use [Azure Stream Analytics CI/CD tools](cicd-tools.md) to build the projects and generate Azure Resource Manager templates for the deployment.
Copy file name to clipboardExpand all lines: articles/stream-analytics/custom-deserializer.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,14 @@
1
1
---
2
-
title: Tutorial - Custom .NET deserializers for Azure Stream Analytics cloud jobs
3
-
description: This tutorial demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio.
2
+
title: Custom .NET deserializers for Azure Stream Analytics cloud jobs
3
+
description: This doc demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio.
4
4
author: sidramadoss
5
5
ms.author: sidram
6
6
ms.service: stream-analytics
7
7
ms.topic: tutorial
8
8
ms.date: 12/17/2020
9
9
---
10
10
11
-
# Tutorial: Custom .NET deserializers for Azure Stream Analytics
11
+
# Custom .NET deserializers for Azure Stream Analytics in Visual Studio
12
12
13
13
Azure Stream Analytics has [built-in support for three data formats](stream-analytics-parsing-json.md): JSON, CSV, and Avro. With custom .NET deserializers, you can read data from other formats such as [Protocol Buffer](https://developers.google.com/protocol-buffers/), [Bond](https://github.com/Microsoft/bond) and other user defined formats for both cloud and edge jobs.
Copy file name to clipboardExpand all lines: articles/stream-analytics/develop-locally.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,6 +24,9 @@ The environments in the following table support local development:
24
24
|[Visual Studio 2019](stream-analytics-tools-for-visual-studio-install.md)|Stream Analytics Tools is part of the Azure development and Data storage and processing workloads in Visual Studio. You can use Visual Studio to write custom C# user-defined functions and deserializers. To learn more, see [Create an Azure Stream Analytics job by using Visual Studio](stream-analytics-quick-create-vs.md).|
25
25
|[Command prompt or terminal](stream-analytics-tools-for-visual-studio-cicd.md)|The Azure Stream Analytics CI/CD NuGet package provides tools for Visual studio project build, local testing on an arbitrary machine. The Azure Stream Analytics CI/CD npm package provides tools for Visual Studio Code project builds (which generates an Azure Resource Manager template) on an arbitrary machine.|
26
26
27
+
> [!NOTE]
28
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
29
+
27
30
## Next steps
28
31
29
32
*[Test Stream Analytics queries locally with sample data using Visual Studio Code](visual-studio-code-local-run.md)
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-add-inputs.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,10 @@ Stream Analytics has first-class integration with four kinds of resources as inp
19
19
20
20
These input resources can live in the same Azure subscription as your Stream Analytics job, or from a different subscription.
21
21
22
-
You can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-input), [Azure PowerShell](/powershell/module/az.streamanalytics/New-azStreamAnalyticsInput), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.inputsoperationsextensions), [REST API](/rest/api/streamanalytics/2020-03-01/inputs), and [Visual Studio](stream-analytics-tools-for-visual-studio-install.md) to create, edit, and test Stream Analytics job inputs.
22
+
You can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-input), [Azure PowerShell](/powershell/module/az.streamanalytics/New-azStreamAnalyticsInput), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.inputsoperationsextensions), [REST API](/rest/api/streamanalytics/2020-03-01/inputs), [Visual Studio](stream-analytics-tools-for-visual-studio-install.md), and [Visual Studio Code](./quick-create-visual-studio-code.md) to create, edit, and test Stream Analytics job inputs.
23
+
24
+
> [!NOTE]
25
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
23
26
24
27
## Stream and reference inputs
25
28
As data is pushed to a data source, it's consumed by the Stream Analytics job and processed in real time. Inputs are divided into two types: data stream inputs and reference data inputs.
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-define-inputs.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,6 +26,8 @@ Stream Analytics supports compression across all data stream input sources. Supp
26
26
27
27
You can use the [Azure portal](stream-analytics-quick-create-portal.md), [Visual Studio](stream-analytics-quick-create-vs.md), and [Visual Studio Code](quick-create-visual-studio-code.md) to add and view or edit existing inputs on your streaming job. You can also test input connections and test queries from sample data from the Azure portal, [Visual Studio](stream-analytics-vs-tools-local-run.md), and [Visual Studio Code](visual-studio-code-local-run.md). When you write a query, you list the input in the FROM clause. You can get the list of available inputs from the **Query** page in the portal. If you wish to use multiple inputs, you can `JOIN` them or write multiple `SELECT` queries.
28
28
29
+
> [!NOTE]
30
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-define-outputs.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,10 @@ ms.date: 01/14/2022
13
13
14
14
An Azure Stream Analytics job consists of an input, query, and an output. There are several output types to which you can send transformed data. This article lists the supported Stream Analytics outputs. When you design your Stream Analytics query, refer to the name of the output by using the [INTO clause](/stream-analytics-query/into-azure-stream-analytics). You can use a single output per job, or multiple outputs per streaming job (if you need them) by adding multiple INTO clauses to the query.
15
15
16
-
To create, edit, and test Stream Analytics job outputs, you can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-output), [Azure PowerShell](stream-analytics-quick-create-powershell.md#configure-output-to-the-job), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.ioutputsoperations), [REST API](/rest/api/streamanalytics/), and [Visual Studio](stream-analytics-quick-create-vs.md).
16
+
To create, edit, and test Stream Analytics job outputs, you can use the [Azure portal](stream-analytics-quick-create-portal.md#configure-job-output), [Azure PowerShell](stream-analytics-quick-create-powershell.md#configure-output-to-the-job), [.NET API](/dotnet/api/microsoft.azure.management.streamanalytics.ioutputsoperations), [REST API](/rest/api/streamanalytics/), [Visual Studio](stream-analytics-quick-create-vs.md), and [Visual Studio Code](./quick-create-visual-studio-code.md).
17
+
18
+
> [!NOTE]
19
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
17
20
18
21
Some outputs types support [partitioning](#partitioning), and [output batch sizes](#output-batch-size) vary to optimize throughput. The following table shows features that are supported for each output type:
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-previews.md
-11Lines changed: 0 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,14 +30,3 @@ You can use the job diagram while testing your query locally to examine the inte
30
30
## Explore jobs in Visual Studio Code
31
31
32
32
Stream Analytics Explorer in Visual Studio Code Extension gives developers a lightweight experience for managing their Stream Analytics jobs. In the Stream Analytics Explorer, you can easily manage your jobs, view job diagram, and debug in Job Monitor.
33
-
34
-
## Debug query steps in Visual Studio
35
-
36
-
You can easily preview the intermediate row set on a data diagram when doing local testing in Azure Stream Analytics tools for Visual Studio.
37
-
38
-
39
-
## Live data testing in Visual Studio
40
-
41
-
Visual Studio tools for Azure Stream Analytics enhance the local testing feature that allows you to test you queries against live event streams from cloud sources such as Event Hub or IoT hub. Learn how to [Test live data locally using Azure Stream Analytics tools for Visual Studio](stream-analytics-live-data-local-testing.md).
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-tools-for-visual-studio-install.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,9 @@ Visual Studio 2019 and Visual Studio 2017 support Azure Data Lake and Stream Ana
14
14
15
15
For more information on using the tools, see [Quickstart: Create an Azure Stream Analytics job by using Visual Studio](stream-analytics-quick-create-vs.md).
16
16
17
+
> [!NOTE]
18
+
> We strongly recommend using [**Stream Analytics tools for Visual Studio Code**](./quick-create-visual-studio-code.md) for best local development experience. There are known feature gaps in Stream Analytics tools for Visual Studio 2019 (version 2.6.3000.0) and it won't be improved going forward.
19
+
17
20
## Install
18
21
19
22
Visual Studio Enterprise (Ultimate/Premium), Professional, and Community editions support the tools. Express edition and Visual Studio for Mac don't support them.
Copy file name to clipboardExpand all lines: articles/stream-analytics/visual-studio-code-custom-deserializer.md
+17-11Lines changed: 17 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: Create custom .NET deserializers for Azure Stream Analytics cloud jobs using Visual Studio Code
2
+
title: Tutorial - Create custom .NET deserializers for Azure Stream Analytics cloud jobs using Visual Studio Code
3
3
description: This tutorial demonstrates how to create a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code.
4
4
author: su-jie
5
5
ms.author: sujie
@@ -9,21 +9,27 @@ ms.date: 12/22/2020
9
9
---
10
10
11
11
12
-
# Create custom .NET deserializers for Azure Stream Analytics in Visual Studio Code
12
+
# Tutorial: Custom .NET deserializers for Azure Stream Analytics in Visual Studio Code
13
13
14
14
Azure Stream Analytics has [built-in support for three data formats](stream-analytics-parsing-json.md): JSON, CSV, and Avro. With custom .NET deserializers, you can read data from other formats such as [Protocol Buffer](https://developers.google.com/protocol-buffers/), [Bond](https://github.com/Microsoft/bond) and other user defined formats for cloud jobs.
15
15
16
-
## Custom .NET deserializers in Visual Studio Code
16
+
This tutorial demonstrates how to create, test, and debug a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code. To learn how to create .NET deserializers in Visual Studio, see [Create .NET deserializers for Azure Stream Analytics jobs in Visual Studio](custom-deserializer.md).
17
17
18
-
You can create, test and debug a custom .NET deserializer for an Azure Stream Analytics cloud job using Visual Studio Code.
18
+
In this tutorial, you learn how to:
19
19
20
-
### Prerequisites
20
+
> [!div class="checklist"]
21
+
> * Create a custom deserializer for protocol buffer.
22
+
> * Create an Azure Stream Analytics job in Visual Studio.
23
+
> * Configure your Stream Analytics job to use the custom deserializer.
24
+
> * Run your Stream Analytics job locally to test and debug the custom deserializer.
25
+
26
+
## Prerequisites
21
27
22
28
* Install [.NET core SDK](https://dotnet.microsoft.com/download) and restart Visual Studio Code.
23
29
24
30
* Use this [quickstart](quick-create-visual-studio-code.md) to learn how to create a Stream Analytics job using Visual Studio Code.
25
31
26
-
###Create a custom deserializer
32
+
## Create a custom deserializer
27
33
28
34
1. Open a terminal and run following command to create a .NET class library in Visual Studio Code for your custom deserializer called **ProtobufDeserializer**.
29
35
@@ -45,11 +51,11 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
45
51
46
52
4. Build the **ProtobufDeserializer** project.
47
53
48
-
###Add an Azure Stream Analytics project
54
+
## Add an Azure Stream Analytics project
49
55
50
-
1.Open Visual Studio Code and select **Ctrl+Shift+P** to open the command palette. Then enter ASA and select **ASA: Create New Project**. Name it **ProtobufCloudDeserializer**.
56
+
Open Visual Studio Code and select **Ctrl+Shift+P** to open the command palette. Then enter ASA and select **ASA: Create New Project**. Name it **ProtobufCloudDeserializer**.
51
57
52
-
###Configure a Stream Analytics job
58
+
## Configure a Stream Analytics job
53
59
54
60
1. Double-click **JobConfig.json**. Use the default configurations, except for the following settings:
55
61
@@ -88,7 +94,7 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
88
94
|-------|---------------|
89
95
|Select local file path|Click CodeLens to select < The file path for the downloaded sample protobuf input file>|
90
96
91
-
###Execute the Stream Analytics job
97
+
## Execute the Stream Analytics job
92
98
93
99
1. Open **ProtobufCloudDeserializer.asaql** and select **Run Locally** from CodeLens then choose **Use Local Input** from the dropdown list.
94
100
@@ -98,7 +104,7 @@ You can create, test and debug a custom .NET deserializer for an Azure Stream An
98
104
99
105
You have successfully implemented a custom deserializer for your Stream Analytics job! In this tutorial, you tested the custom deserializer locally with local input data. You can also test it [using live data input in the cloud](visual-studio-code-local-run-live-input.md). For running the job in the cloud, you would properly configure the input and output. Then submit the job to Azure from Visual Studio Code to run your job in the cloud using the custom deserializer you just implemented.
100
106
101
-
###Debug your deserializer
107
+
## Debug your deserializer
102
108
103
109
You can debug your .NET deserializer locally the same way you debug standard .NET code.
0 commit comments