|
1 | 1 | ---
|
2 |
| -title: Map data by using dataflows |
3 |
| -description: Learn about the dataflow mapping language for transforming data in Azure IoT Operations. |
| 2 | +title: Map data by using data flows |
| 3 | +description: Learn about the data flow mapping language for transforming data in Azure IoT Operations. |
4 | 4 | author: PatAltimore
|
5 | 5 | ms.author: patricka
|
6 | 6 | ms.subservice: azure-data-flows
|
7 | 7 | ms.topic: concept-article
|
8 | 8 | ms.date: 11/11/2024
|
9 | 9 | ai-usage: ai-assisted
|
10 | 10 |
|
11 |
| -#CustomerIntent: As an operator, I want to understand how to use the dataflow mapping language to transform data. |
| 11 | +#CustomerIntent: As an operator, I want to understand how to use the data flow mapping language to transform data. |
12 | 12 | ms.service: azure-iot-operations
|
13 | 13 | ---
|
14 | 14 |
|
15 |
| -# Map data by using dataflows |
| 15 | +# Map data by using data flows |
16 | 16 |
|
17 | 17 | [!INCLUDE [kubernetes-management-preview-note](../includes/kubernetes-management-preview-note.md)]
|
18 | 18 |
|
19 |
| -Use the dataflow mapping language to transform data in Azure IoT Operations. The syntax is a simple, yet powerful, way to define mappings that transform data from one format to another. This article provides an overview of the dataflow mapping language and key concepts. |
| 19 | +Use the data flow mapping language to transform data in Azure IoT Operations. The syntax is a simple, yet powerful, way to define mappings that transform data from one format to another. This article provides an overview of the data flow mapping language and key concepts. |
20 | 20 |
|
21 | 21 | Mapping allows you to transform data from one format to another. Consider the following input record:
|
22 | 22 |
|
@@ -127,7 +127,7 @@ When you use MQTT or Kafka as a source or destination, you can access various me
|
127 | 127 | * **Topic**: Works for both MQTT and Kafka. It contains the string where the message was published. Example: `$metadata.topic`.
|
128 | 128 | * **User property**: In MQTT, this refers to the free-form key/value pairs an MQTT message can carry. For example, if the MQTT message was published with a user property with key "priority" and value "high", then the `$metadata.user_property.priority` reference hold the value "high". User property keys can be arbitrary strings and may require escaping: `$metadata.user_property."weird key"` uses the key "weird key" (with a space).
|
129 | 129 | * **System property**: This term is used for every property that is not a user property. Currently, only a single system property is supported: `$metadata.system_property.content_type`, which reads the content type property of the MQTT message (if set).
|
130 |
| -* **Header**: This is the Kafka equivalent of the MQTT user property. Kafka can use any binary value for a key, but dataflow supports only UTF-8 string keys. Example: `$metadata.header.priority`. This functionality is similar to user properties. |
| 130 | +* **Header**: This is the Kafka equivalent of the MQTT user property. Kafka can use any binary value for a key, but data flows support only UTF-8 string keys. Example: `$metadata.header.priority`. This functionality is similar to user properties. |
131 | 131 |
|
132 | 132 | #### Mapping metadata properties
|
133 | 133 |
|
@@ -269,7 +269,7 @@ inputs: [
|
269 | 269 |
|
270 | 270 | ---
|
271 | 271 |
|
272 |
| -In a dataflow, a path described by dot notation might include strings and some special characters without needing escaping: |
| 272 | +In a data flow, a path described by dot notation might include strings and some special characters without needing escaping: |
273 | 273 |
|
274 | 274 | # [Bicep](#tab/bicep)
|
275 | 275 |
|
@@ -309,7 +309,7 @@ inputs: [
|
309 | 309 |
|
310 | 310 | The previous example, among other special characters, contains dots within the field name. Without escaping, the field name would serve as a separator in the dot notation itself.
|
311 | 311 |
|
312 |
| -While a dataflow parses a path, it treats only two characters as special: |
| 312 | +While a data flow parses a path, it treats only two characters as special: |
313 | 313 |
|
314 | 314 | * Dots (`.`) act as field separators.
|
315 | 315 | * Single quotation marks, when placed at the beginning or the end of a segment, start an escaped section where dots aren't treated as field separators.
|
@@ -850,13 +850,13 @@ Consider a special case for the same fields to help decide the right action:
|
850 | 850 |
|
851 | 851 | An empty `output` field in the second definition implies not writing the fields in the output record (effectively removing `Opacity`). This setup is more of a `Specialization` than a `Second Rule`.
|
852 | 852 |
|
853 |
| -Resolution of overlapping mappings by dataflows: |
| 853 | +Resolution of overlapping mappings by data flows: |
854 | 854 |
|
855 | 855 | * The evaluation progresses from the top rule in the mapping definition.
|
856 | 856 | * If a new mapping resolves to the same fields as a previous rule, the following conditions apply:
|
857 | 857 | * A `Rank` is calculated for each resolved input based on the number of segments the wildcard captures. For instance, if the `Captured Segments` are `Properties.Opacity`, the `Rank` is 2. If it's only `Opacity`, the `Rank` is 1. A mapping without wildcards has a `Rank` of 0.
|
858 |
| - * If the `Rank` of the latter rule is equal to or higher than the previous rule, a dataflow treats it as a `Second Rule`. |
859 |
| - * Otherwise, the dataflow treats the configuration as a `Specialization`. |
| 858 | + * If the `Rank` of the latter rule is equal to or higher than the previous rule, a data flow treats it as a `Second Rule`. |
| 859 | + * Otherwise, the data flow treats the configuration as a `Specialization`. |
860 | 860 |
|
861 | 861 | For example, the mapping that directs `Opacity.Max` and `Opacity.Min` to an empty output has a `Rank` of 0. Because the second rule has a lower `Rank` than the previous one, it's considered a specialization and overrides the previous rule, which would calculate a value for `Opacity`.
|
862 | 862 |
|
|
0 commit comments