You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/concept-dataflow-language.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: PatAltimore
5
5
ms.author: patricka
6
6
ms.subservice: azure-mqtt-broker
7
7
ms.topic: concept-article
8
-
ms.date: 07/23/2024
8
+
ms.date: 07/25/2024
9
9
10
10
#CustomerIntent: As an operator, I want to understand how to use the dataflow mapping language to transform data.
11
11
---
@@ -430,8 +430,8 @@ Now, consider a scenario where a specific field needs a different calculation:
430
430
431
431
In this case, the `Opacity` field has a unique calculation. Two options to handle this overlapping scenario are:
432
432
433
-
1. Include both mappings for `Opacity`. Since the output fields are different in this example, they wouldn't override each other.
434
-
2. Use the more specific rule for `Opacity` and remove the more generic one.
433
+
-Include both mappings for `Opacity`. Since the output fields are different in this example, they wouldn't override each other.
434
+
-Use the more specific rule for `Opacity` and remove the more generic one.
435
435
436
436
Consider a special case for the same fields to help deciding the right action:
437
437
@@ -462,7 +462,7 @@ For example, the mapping that directs `Opacity.Max` and `Opacity.Min` to an empt
462
462
463
463
### Wildcards in contextualization datasets
464
464
465
-
While a detailed explanation of Contextualization Datasets will follow later, let's see now how they can be used with wildcards through an example. Consider a dataset named `position` that contains the following record:
465
+
While a detailed explanation of contextualization datasets is explained later, let's see now how they can be used with wildcards through an example. Consider a dataset named `position` that contains the following record:
466
466
467
467
```json
468
468
{
@@ -516,10 +516,10 @@ When discussing conversions, several aspects need to be clarified:
516
516
517
517
* **Reference to Input Fields**: Understanding how to correctly reference values from input fields in the conversion formula.
518
518
* **Available Operations**: Identifying which operations (such as addition, subtraction, multiplication, and division) can be utilized in conversions.
519
-
* **Data Types**: The types of data (e.g., integer, floating-point, string) that the formula can process and manipulate.
519
+
* **Data Types**: The types of data (for example, integer, floating-point, string) that the formula can process and manipulate.
520
520
* **Type Conversions**: How data types are converted between the input field values, the formula evaluation, and the output fields.
521
521
522
-
This section will explore each of these aspects.
522
+
This section explores each of these aspects.
523
523
524
524
## Reference to input fields
525
525
@@ -535,11 +535,11 @@ In conversions, formulas can operate on static values (For example, a number lik
535
535
conversion: ($1, $2, $3, $4)
536
536
```
537
537
538
-
In this example, the conversion results in an array containing the values of `[Max, Min, Mid.Avg, Mid.Mean]`. Comments in the YAML file (e.g., `# - $1`, `# - $2`) are optional but help clarify the connection between each field property and its role in the conversion formula.
538
+
In this example, the conversion results in an array containing the values of `[Max, Min, Mid.Avg, Mid.Mean]`. Comments in the YAML file (for example, `# - $1`, `# - $2`) are optional but help clarify the connection between each field property and its role in the conversion formula.
539
539
540
540
## Available operations
541
541
542
-
Conversions use simple math formulas similar to those learned in middle school. Basic operators such as addition (`+`) and multiplication (`*`) are included, each following specific rules of precedence (e.g., `*` is performed before `+`), which can be modified using parentheses.
542
+
Conversions use simple math formulas similar to those learned in middle school. Basic operators such as addition (`+`) and multiplication (`*`) are included, each following specific rules of precedence (for example, `*` is performed before `+`), which can be modified using parentheses.
543
543
544
544
For more complex calculations, functions like `sqrt` (which finds the square root of a number) are also available.
545
545
@@ -652,7 +652,7 @@ The mapper is designed to be flexible, converting internal types into output typ
652
652
653
653
### Explicit type conversions
654
654
655
-
While the automatic conversions generally operate as one might expect, based on common implementation practices, there are instances where the right conversion cannot be determined automatically, resulting in an `unsupported` error. To address these situations, several conversion functions are available to explicitly define how data should be transformed. These functions provide more control over how data is converted and ensure that data integrity is maintained even when automatic methods fall short.
655
+
While the automatic conversions generally operate as one might expect, based on common implementation practices, there are instances where the right conversion can't be determined automatically, resulting in an `unsupported` error. To address these situations, several conversion functions are available to explicitly define how data should be transformed. These functions provide more control over how data is converted and ensure that data integrity is maintained even when automatic methods fall short.
656
656
657
657
<!-- We don't have functions, need a list of what they usually want -->
658
658
@@ -669,9 +669,9 @@ If a formula is specified, the data types available for use in formulas are limi
669
669
* Arrays of the above types
670
670
* Missing value
671
671
672
-
`Map`and `Byte` cannot participate in formulas in any way.
672
+
`Map`and `Byte` can't participate in formulas in any way.
673
673
674
-
Types related to time (`date time`, `time`, and `duration`) are converted into integer values representing time in seconds. After formula evaluation, results are stored in the internal representation and not converted back; for example, a datetime converted to seconds remains an integer. If the value is to be used in date-time fields, an explicit conversion method must applied, for example, to convert the value into an ISO8601 string, which will then automatically converted to the date-time type of the output serialization format. **[FIXME as of today, no such functions are available]**
674
+
Types related to time (`date time`, `time`, and `duration`) are converted into integer values representing time in seconds. After formula evaluation, results are stored in the internal representation and not converted back. For example, a *datetime* converted to seconds remains an integer. If the value is to be used in date-time fields, an explicit conversion method must be applied. For example, converting the value into an ISO8601 string that is automatically converted to the date-time type of the output serialization format.
675
675
676
676
### Using irregular types
677
677
@@ -698,7 +698,7 @@ With the mapping:
698
698
699
699
This configuration selects the smallest value from the `Measurements` array for the output field.
700
700
701
-
It is also possible to use functions that result a new array:**[FIXME: this is not available today]**
701
+
It's also possible to use functions that result a new array:
702
702
703
703
```yaml
704
704
- inputs:
@@ -826,9 +826,9 @@ This is how the dataset is used, when a new record is being processed:
826
826
827
827
In this example the `WorkingHours` field is added to the output record, while the `BaseSalary` is used conditionally: only when the incoming record doesn't contain `BaseSalary` field (or the value is `null` in case of nullable a field)
828
828
829
-
Note, that the request for the contextualization data does not happen with every incoming record: the mapper requests the dataset at the beginning and then it receives notifications from DSS about the changes, while it uses a cached version of the dataset.
829
+
Note, that the request for the contextualization data doesn't happen with every incoming record: the mapper requests the dataset at the beginning and then it receives notifications from DSS about the changes, while it uses a cached version of the dataset.
Copy file name to clipboardExpand all lines: articles/iot-operations/connect-to-cloud/howto-create-dataflow.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: PatAltimore
5
5
ms.author: patricka
6
6
ms.subservice: azure-mqtt-broker
7
7
ms.topic: conceptual
8
-
ms.date: 07/22/2024
8
+
ms.date: 07/25/2024
9
9
10
10
#CustomerIntent: As an operator, I want to understand how to create a dataflow to connect data sources.
11
11
---
@@ -384,7 +384,7 @@ You can also use user properties.
384
384
385
385
If the user property isn't present in the source data, the part of the path referencing the user property is empty. For example, if the user property `customer` isn't present in the source data, the output path is `factory/` .
386
386
387
-
If you specified a *enrich* stage during transformation, you can use the enriched data in the output path.
387
+
If you specified an *enrich* stage during transformation, you can use the enriched data in the output path.
0 commit comments