You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# JavaScript user-defined functions in Azure Stream Analytics
16
16
17
17
Azure Stream Analytics supports user-defined functions written in JavaScript. With the rich set of **String**, **RegExp**, **Math**, **Array**, and **Date** methods that JavaScript provides, complex data transformations with Stream Analytics jobs become easier to create.
18
18
19
-
In this tutorial, you learn how to:
19
+
## Overview
20
20
21
-
> [!div class="checklist"]
22
-
> * Define a JavaScript user-defined function
23
-
> * Add the function to the portal
24
-
> * Define a query that runs the function
25
-
26
-
If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
27
-
28
-
## JavaScript user-defined functions
29
-
JavaScript user-defined functions support stateless, compute-only scalar functions that do not require external connectivity. The return value of a function can only be a scalar (single) value. After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function.
21
+
JavaScript user-defined functions support stateless, compute-only scalar functions that don't require external connectivity. The return value of a function can only be a scalar (single) value. After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function.
30
22
31
23
Here are some scenarios where you might find JavaScript user-defined functions useful:
32
24
* Parsing and manipulating strings that have regular expression functions, for example, **Regexp_Replace()** and **Regexp_Extract()**
33
25
* Decoding and encoding data, for example, binary-to-hex conversion
34
-
*Performing mathematic computations with JavaScript **Math** functions
35
-
*Performing array operations like sort, join, find, and fill
26
+
*Doing mathematic computations with JavaScript **Math** functions
27
+
*Doing array operations like sort, join, find, and fill
36
28
37
-
Here are some things that you cannot do with a JavaScript user-defined function in Stream Analytics:
38
-
* Call out external REST endpoints, for example, performing reverse IP lookup or pulling reference data from an external source
29
+
Here are some things that you can't do with a JavaScript user-defined function in Stream Analytics:
30
+
* Call out external REST endpoints, for example, doing reverse IP lookup or pulling reference data from an external source
39
31
* Perform custom event format serialization or deserialization on inputs/outputs
40
32
* Create custom aggregates
41
33
42
-
Although functions like **Date.GetDate()** or **Math.random()**are not blocked in the functions definition, you should avoid using them. These functions **do not** return the same result every time you call them, and the Azure Stream Analytics service does not keep a journal of function invocations and returned results. If a function returns different result on the same events, repeatability is not guaranteed when a job is restarted by you or by the Stream Analytics service.
34
+
Although functions like **Date.GetDate()** or **Math.random()**aren't blocked in the functions definition, you should avoid using them. These functions **don't** return the same result every time you call them, and the Azure Stream Analytics service doesn't keep a journal of function invocations and returned results. If a function returns different result on the same events, repeatability isn't guaranteed when a job is restarted by you or by the Stream Analytics service.
43
35
44
-
## Add a JavaScript user-defined function in the Azure portal
45
-
To create a simple JavaScript user-defined function under an existing Stream Analytics job, follow these steps:
36
+
## Add a JavaScript user-defined function to your job
46
37
47
38
> [!NOTE]
48
39
> These steps work on the Stream Analytics jobs configured to run in the cloud. If your Stream Analytics job is configured to run on Azure IoT Edge, instead use Visual Studio and [write the user-defined function using C#](stream-analytics-edge-csharp-udf.md).
49
40
50
-
1. In the Azure portal, find your Stream Analytics job.
51
-
52
-
2. Under the **Job topology** heading, select **Functions**. An empty list of functions appears.
41
+
To create a JavaScript user-defined function in your Stream Analytics job, select **Functions** under **Job Topology**. Then, select **JavaScript UDF** from the **+Add** dropdown menu.
53
42
54
-
3. To create a new user-defined function, select **+ Add**.
4. On the **New Function** blade, for **Function Type**, select **JavaScript**. A default function template appears in the editor.
45
+
You must then provide the following properties and select **Save**.
57
46
58
-
5. For the **UDF alias**, enter **hex2Int**, and change the function implementation as follows:
47
+
|Property|Description|
48
+
|--------|-----------|
49
+
|Function alias|Enter a name to invoke the function in your query.|
50
+
|Output type|Type that will be returned by your JavaScript user-defined function to your Stream Analytics query.|
51
+
|Function definition|Implementation of your JavaScript function that will be executed each time your UDF gets invoked from your query.|
59
52
60
-
```javascript
61
-
// Convert Hex value to integer.
62
-
function hex2Int(hexValue) {
63
-
return parseInt(hexValue, 16);
64
-
}
65
-
```
53
+
## Test and troubleshoot JavaScript UDFs
66
54
67
-
6. Select **Save**. Your function appears in the list of functions.
68
-
7. Select the new **hex2Int** function, and check the function definition. All functions have a **UDF** prefix added to the function alias. You need to *include the prefix* when you call the function in your Stream Analytics query. In this case, you call **UDF.hex2Int**.
55
+
You can test and debug your JavaScript UDF logic in any browser. Debugging and testing the logic of these user-defined functions is currently not supported in the Stream Analytics portal. Once the function works as expected, you can add it to the Stream Analytics job as mentioned above and then invoke it directly from your query. You can test your query logic with JavaScript UDF using [Stream Analytics tools for Visual Studio](https://docs.microsoft.com/azure/stream-analytics/stream-analytics-tools-for-visual-studio-install).
69
56
70
-
## Testing JavaScript UDFs
71
-
You can test and debug your JavaScript UDF logic in any browser. Debugging and testing the logic of these user-defined functions is currently not supported in the Stream Analytics portal. Once the function works as expected, you can add it to the Stream Analytics job as mentioned above and then invoke it directly from your query.
57
+
JavaScript runtime errors are considered fatal, and are surfaced through the Activity log. To retrieve the log, in the Azure portal, go to your job and select **Activity log**.
72
58
73
59
## Call a JavaScript user-defined function in a query
74
60
75
-
1. In the query editor, under the **Job topology** heading, select **Query**.
76
-
2. Edit your query, and then call the user-defined function, like this:
61
+
You can easily invoke your JavaScript function in your query using the function alias prefixed with **udf**. Here is an example of a JavaScript UDF that converts hexadecimal values to integer being invoked in a Stream Analytics query.
77
62
78
-
```SQL
63
+
```SQL
79
64
SELECT
80
65
time,
81
66
UDF.hex2Int(offset) AS IntOffset
82
67
INTO
83
68
output
84
69
FROM
85
70
InputStream
86
-
```
87
-
88
-
3. To upload the sample data file, right-click the job input.
89
-
4. To test your query, select **Test**.
90
-
71
+
```
91
72
92
73
## Supported JavaScript objects
74
+
93
75
Azure Stream Analytics JavaScript user-defined functions support standard, built-in JavaScript objects. For a list of these objects, see [Global Objects](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects).
94
76
95
77
### Stream Analytics and JavaScript type conversion
@@ -106,10 +88,8 @@ Record | Object
106
88
Array | Array
107
89
NULL | Null
108
90
109
-
110
91
Here are JavaScript-to-Stream Analytics conversions:
111
92
112
-
113
93
JavaScript | Stream Analytics
114
94
--- | ---
115
95
Number | Bigint (if the number is round and between long.MinValue and long.MaxValue; otherwise, it's double)
@@ -120,14 +100,12 @@ Array | Array
120
100
Null, Undefined | NULL
121
101
Any other type (for example, a function or error) | Not supported (results in runtime error)
122
102
123
-
JavaScript language is case sensitive and casing of the object fields in JavaScript code must match the casing of the fields in the incoming data. Please note that jobs with compatibility level 1.0 will convert fields from SQL SELECT statement to be lowercase. Under compatibility level 1.1 and higher, fields from SELECT statement will have the same casing as specified in the SQL query.
124
-
125
-
## Troubleshooting
126
-
JavaScript runtime errors are considered fatal, and are surfaced through the Activity log. To retrieve the log, in the Azure portal, go to your job and select **Activity log**.
103
+
JavaScript language is case-sensitive and casing of the object fields in JavaScript code must match the casing of the fields in the incoming data. Jobs with compatibility level 1.0 will convert fields from SQL SELECT statement to be lowercase. Under compatibility level 1.1 and higher, fields from SELECT statement will have the same casing as specified in the SQL query.
127
104
128
105
## Other JavaScript user-defined function patterns
129
106
130
107
### Write nested JSON to output
108
+
131
109
If you have a follow-up processing step that uses a Stream Analytics job output as input, and it requires a JSON format, you can write a JSON string to output. The next example calls the **JSON.stringify()** function to pack all name/value pairs of the input, and then write them as a single string value in output.
132
110
133
111
**JavaScript user-defined function definition:**
@@ -151,19 +129,7 @@ FROM
151
129
input PARTITION BY PARTITIONID
152
130
```
153
131
154
-
## Clean up resources
155
-
156
-
When no longer needed, delete the resource group, the streaming job, and all related resources. Deleting the job avoids billing the streaming units consumed by the job. If you're planning to use the job in future, you can stop it and re-start it later when you need. If you are not going to continue to use this job, delete all resources created by this quickstart by using the following steps:
157
-
158
-
1. From the left-hand menu in the Azure portal, click **Resource groups** and then click the name of the resource you created.
159
-
2. On your resource group page, click **Delete**, type the name of the resource to delete in the text box, and then click **Delete**.
160
-
161
-
## Get help
162
-
For additional help, try our [Azure Stream Analytics forum](https://social.msdn.microsoft.com/Forums/azure/home?forum=AzureStreamAnalytics).
163
-
164
132
## Next steps
165
133
166
-
In this tutorial, you have created a Stream Analytics job that runs a simple JavaScript user-defined function. To learn more about Stream Analytics, continue to the real-time scenario articles:
167
-
168
-
> [!div class="nextstepaction"]
169
-
> [Real-time Twitter sentiment analysis in Azure Stream Analytics](stream-analytics-twitter-sentiment-analysis-trends.md)
0 commit comments