You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/concept-dataflow-conversions.md
+96-2Lines changed: 96 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: Dataflow conversions
2
+
title: Convert data using dataflow conversions
3
3
description: Learn about dataflow conversions for transforming data in Azure IoT Operations.
4
4
author: PatAltimore
5
5
ms.author: patricka
@@ -10,7 +10,7 @@ ms.date: 08/01/2024
10
10
#CustomerIntent: As an operator, I want to understand how to use dataflow conversions to transform data.
11
11
---
12
12
13
-
# Dataflow conversions
13
+
# Convert data using dataflow conversions
14
14
15
15
You can use dataflow conversions to transform data in Azure IoT Operations. The *conversion* element in a dataflow is used to compute values for output fields. You can use input fields, available operations, data types, and type conversions in dataflow conversions.
16
16
@@ -214,3 +214,97 @@ Functions can be used in the conversion formula to perform various operations.
214
214
* string manipulation (for example, `uppercase()`)
215
215
* explicit conversion (for example, `ISO8601_datetime`)
216
216
* aggregation (for example, `avg()`)
217
+
218
+
## Available operations
219
+
220
+
Dataflow offers a wide range of out-of-the-box (OOTB) conversion functions that allow users to easily perform unit conversions without the need for complex calculations. These predefined functions cover common conversions such as temperature, pressure, length, weight, and volume. The following is a list of the available conversion functions, along with their corresponding formulas and function names:
221
+
222
+
| Conversion | Formula | Function Name |
223
+
| --- | --- | --- |
224
+
| Celsius to Fahrenheit | F = (C * 9/5) + 32 | cToF |
These functions are designed to simplify the conversion process, allowing users to input values in one unit and receive the corresponding value in another unit effortlessly.
243
+
244
+
Additionally, we provide a scaling function to scale the range of value to the user-defined range. Example-`scale($1,0,10,0,100)`the input value is scaled from the range 0 to 10 to the range 0 to 100.
245
+
246
+
Moreover, users have the flexibility to define their own conversion functions using simple mathematical formulas. Our system supports basic operators such as addition (`+`), subtraction (`-`), multiplication (`*`), and division (`/`). These operators follow standard rules of precedence (for example, multiplication and division are performed before addition and subtraction), which can be adjusted using parentheses to ensure the correct order of operations. This capability empowers users to customize their unit conversions to meet specific needs or preferences, enhancing the overall utility and versatility of the system.
247
+
248
+
249
+
For more complex calculations, functions like `sqrt` (which finds the square root of a number) are also available.
250
+
251
+
### Available arithmetic, comparison, and boolean operators grouped by precedence
252
+
253
+
| Operator | Description |
254
+
|----------|-------------|
255
+
| ^ | Exponentiation: $1 ^ 3 |
256
+
257
+
Since `Exponentiation` has the highest precedence, it's executed first unless parentheses override this order:
258
+
259
+
* `$1 * 2 ^ 3` is interpreted as `$1 * 8` because the `2 ^ 3` part is executed first, before multiplication.
260
+
* `($1 * 2) ^ 3` processes the multiplication before exponentiation.
261
+
262
+
| Operator | Description |
263
+
|----------|-------------|
264
+
| - | Negation |
265
+
| ! | Logical not |
266
+
267
+
`Negation` and `Logical not` have high precedence, so they always stick to their immediate neighbor, except when exponentiation is involved:
268
+
269
+
* `-$1 * 2` negates $1 first, then multiplies.
270
+
* `-($1 * 2)` multiplies, then negates the result
271
+
272
+
| Operator | Description |
273
+
|----------|-------------|
274
+
| * | Multiplication: $1 * 10 |
275
+
| / | Division: $1 / 25 (Result is an integer if both arguments are integers, otherwise float) |
276
+
| % | Modulo: $1 % 25 |
277
+
278
+
`Multiplication`, `Division`, and `Modulo`, having the same precedence, are executed from left to right, unless the order is altered by parentheses.
279
+
280
+
| Operator | Description |
281
+
|----------|-------------|
282
+
| + | Addition for numeric values, concatenation for strings |
283
+
| - | Subtraction |
284
+
285
+
`Addition` and `Subtraction` are considered weaker operations compared to those in the previous group:
286
+
287
+
* `$1 + 2 * 3` results in `$1 + 6`, as `2 * 3` is executed first due to the higher precedence of `Multiplication`.
288
+
* `($1 + 2) * 3` prioritizes the `addition` before `multiplication`.
289
+
290
+
| Operator | Description |
291
+
|----------|-------------|
292
+
| < | Less than |
293
+
| > | Greater than |
294
+
| <= | Less than or equal to |
295
+
| >= | Greater than or equal to |
296
+
| == | Equal to |
297
+
| != | Not equal to |
298
+
299
+
`Comparisons` operate on numeric, boolean, and string values. Since they have lower precedence than arithmetic operators, no parentheses are needed to compare results effectively:
300
+
301
+
* `$1 * 2 <= $2` is equivalent to `($1 * 2) <= $2`.
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-create-dataflow.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -116,7 +116,7 @@ spec:
116
116
117
117
## Configure transformation
118
118
119
-
The transformation operation is where you can transform the data from the source before sending it to the destination. Transformations are optional. If you don't need to make changes to the data, don't include the transformation operation in the dataflow configuration. Multiple transformations are chained together in stages regardless of the order they are specified in the configuration.
119
+
The transformation operation is where you can transform the data from the source before sending it to the destination. Transformations are optional. If you don't need to make changes to the data, don't include the transformation operation in the dataflow configuration. Multiple transformations are chained together in stages regardless of the order they're specified in the configuration.
120
120
121
121
```yaml
122
122
spec:
@@ -178,13 +178,15 @@ If the dataset has a record with the `asset` field, similar to:
178
178
179
179
The data from the source with the `deviceId` field matching `thermostat1` has the `location` and `manufacturer` fields available `filter` and `map` stages.
180
180
181
+
For more information, see [Enrich data using dataflows](concept-dataflow-enrich.md) and [Convert data using dataflows](concept-dataflow-conversions.md).
182
+
181
183
### Filter: Filter data based on a condition
182
184
183
185
To filter the data on a condition, you can use the `filter` stage. The condition is specified as a field in the source data that matches a value.
0 commit comments