Merge pull request #284247 from paulth1/processing-and-routing-data-batch1

[AQ] edit pass: processing-and-routing-data-batch1

Author: ShawnJackson

articles/iot-operations/connect-to-cloud/howto-configure-dataflow-endpoint.md

@@ -14,9 +14,10 @@ ms.date: 08/03/2024
 
 [!INCLUDE [public-preview-note](../includes/public-preview-note.md)]
 
-To get started with dataflows, you need to configure endpoints. An endpoint is the connection point for the dataflow. You can use an endpoint as a source or destination for the dataflow. Some endpoint types can be used as [both sources and destinations](#endpoint-types-for-use-as-sources-and-destinations), while others are for [destinations only](#endpoint-types-for-destinations-only). A dataflow needs at least one source endpoint and one destination endpoint.
+To get started with dataflows, you need to configure endpoints. An endpoint is the connection point for the dataflow. You can use an endpoint as a source or destination for the dataflow. Some endpoint types can be used as [both sources and destinations](#endpoint-types-for-use-as-sources-and-destinations), while others are for [destinations only](#endpoint-type-for-destinations-only). A dataflow needs at least one source endpoint and one destination endpoint.
+
+The following example shows a custom resource definition with all of the configuration options. The required fields are dependent on the endpoint type. Review the sections for each endpoint type for configuration guidance.
 
-The following example shows a custom resource definition with all of the configuration options. The required fields are dependent on the endpoint type. Review the sections for each endpoint type for configuration guidance. 
 ```yaml
 apiVersion: connectivity.iotoperations.azure.com/v1beta1
 kind: DataflowEndpoint
@@ -57,33 +58,35 @@ spec:
 
 | Name                                            | Description                                                                 |
 |-------------------------------------------------|-----------------------------------------------------------------------------|
-| endpointType                             | Type of the endpoint. Values: mqtt, kafka, dataExplorer, dataLakeStorage, fabricOneLake, or localStorage. |
-| authentication.method                    | Method of authentication. Values: *systemAssignedManagedIdentity*, *x509Credentials*, *userAssignedManagedIdentity*, or *serviceAccountToken*. |
-| authentication.systemAssignedManagedIdentitySettings.audience | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |
-| authentication.x509CredentialsSettings.certificateSecretName | Secret name of the X.509 certificate. |
-| authentication.userAssignedManagedIdentitySettings.clientId | Client ID for the user-assigned managed identity. |
-| authentication.userAssignedManagedIdentitySettings.tenantId | Tenant ID. |
-| authentication.userAssignedManagedIdentitySettings.audience | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |
-| authentication.serviceAccountTokenSettings.audience | Audience of the service account. Optional, defaults to the broker internal service account audience. |
-| mqttSettings.host                        | Host of the MQTT broker in the form of \<hostname\>:\<port\>. connects to MQTT broker if omitted.|
-| mqttSettings.tls                        | TLS configuration. Omit for no TLS or MQTT broker. |
-| mqttSettings.tls.mode                    | Enable or disable TLS. Values: *enabled* or *disabled*. Defaults to *disabled*. |
-| mqttSettings.tls.trustedCaCertificateConfigMap | Trusted CA certificate config map. No CA certificate if omitted. No CA certificate works for public endpoints like Azure Event Grid.|
-| mqttSettings.sharedSubscription | Shared subscription settings. No shared subscription if omitted. |
-| mqttSettings.sharedSubscription.groupMinimumShareNumber | Number of clients to use for shared subscription. |
-| mqttSettings.sharedSubscription.groupName | Shared subscription group name. |
-| mqttSettings.clientIdPrefix              | Client ID prefix. Client ID generated by the dataflow is \<prefix\>-id. No prefix if omitted.|
-| mqttSettings.retain                      | Whether or not to keep the retain setting. Values: *keep* or *never*. Defaults to *keep*. |
-| mqttSettings.sessionExpirySeconds        | Session expiry in seconds. Defaults to *3600*.|
-| mqttSettings.qos                         | Quality of service. Values: *0* or *1*. Defaults to 1.|
-| mqttSettings.protocol                    | Use MQTT or websockets. Values: *mqtt* or *websockets*. Defaults to mqtt.|
-| mqttSettings.maxInflightMessages         | The max number of messages to keep in flight. For subscribe, it's the receive maximum. For publish, it's the maximum number of messages to send before waiting for an acknowledgment. Default is *100*. |
+| `endpointType`                             | Type of the endpoint. Values: `mqtt`, `kafka`, `dataExplorer`, `dataLakeStorage`, `fabricOneLake`, or `localStorage`. |
+| `authentication.method`                    | Method of authentication. Values: `systemAssignedManagedIdentity`, `x509Credentials`, `userAssignedManagedIdentity`, or `serviceAccountToken`. |
+| `authentication.systemAssignedManagedIdentitySettings.audience` | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |
+| `authentication.x509CredentialsSettings.certificateSecretName` | Secret name of the X.509 certificate. |
+| `authentication.userAssignedManagedIdentitySettings.clientId` | Client ID for the user-assigned managed identity. |
+| `authentication.userAssignedManagedIdentitySettings.tenantId` | Tenant ID. |
+| `authentication.userAssignedManagedIdentitySettings.audience` | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |
+| `authentication.serviceAccountTokenSettings.audience` | Audience of the service account. Optional, defaults to the broker internal service account audience. |
+| `mqttSettings.host`                        | Host of the MQTT broker in the form of \<hostname\>:\<port\>. Connects to MQTT broker if omitted.|
+| `mqttSettings.tls`                        | TLS configuration. Omit for no TLS or MQTT broker. |
+| `mqttSettings.tls.mode`                    | Enable or disable TLS. Values: `enabled` or `disabled`. Defaults to `disabled`. |
+| `mqttSettings.tls.trustedCaCertificateConfigMap` | Trusted certificate authority (CA) certificate config map. No CA certificate if omitted. No CA certificate works for public endpoints like Azure Event Grid.|
+| `mqttSettings.sharedSubscription` | Shared subscription settings. No shared subscription if omitted. |
+| `mqttSettings.sharedSubscription.groupMinimumShareNumber` | Number of clients to use for shared subscription. |
+| `mqttSettings.sharedSubscription.groupName` | Shared subscription group name. |
+| `mqttSettings.clientIdPrefix`              | Client ID prefix. Client ID generated by the dataflow is \<prefix\>-id. No prefix if omitted.|
+| `mqttSettings.retain`                      | Whether or not to keep the retain setting. Values: `keep` or `never`. Defaults to `keep`. |
+| `mqttSettings.sessionExpirySeconds`        | Session expiry in seconds. Defaults to `3600`.|
+| `mqttSettings.qos`                         | Quality of service. Values: `0` or `1`. Defaults to `1`.|
+| `mqttSettings.protocol`                    | Use MQTT or web sockets. Values: `mqtt` or `websockets`. Defaults to `mqtt`.|
+| `mqttSettings.maxInflightMessages`         | The maximum number of messages to keep in flight. For subscribe, it's the receive maximum. For publish, it's the maximum number of messages to send before waiting for an acknowledgment. Default is `100`. |
 
 ## Endpoint types for use as sources and destinations
 
+The following endpoint types are used as sources and destinations.
+
 ### MQTT
 
-MQTT endpoints are used for MQTT sources and destinations. You can configure the endpoint, TLS, authentication, and other settings.
+MQTT endpoints are used for MQTT sources and destinations. You can configure the endpoint, Transport Layer Security (TLS), authentication, and other settings.
 
 #### MQTT broker
 
@@ -106,7 +109,7 @@ spec:
 
 #### Event Grid
 
-To configure an Event Grid MQTT broker endpoint, use managed identity for authentication.
+To configure an Azure Event Grid MQTT broker endpoint, use managed identity for authentication.
 
 ```yaml
 apiVersion: connectivity.iotoperations.azure.com/v1beta1
@@ -141,7 +144,7 @@ spec:
       trustedCaCertificateConfigMap: <your CA certificate config map>
 ```
 
-Under `authentication` , you can configure the authentication method for the MQTT broker. Supported methods include X.509:
+Under `authentication`, you can configure the authentication method for the MQTT broker. Supported methods include X.509:
 
 ```yaml
 authentication:
@@ -151,14 +154,14 @@ authentication:
 ```
 
 > [!IMPORTANT]
-> When using X.509 authentication with Event Grid MQTT broker, go to the Event Grid namespace > **Configuration** and check these settings:
+> When you use X.509 authentication with an Event Grid MQTT broker, go to the Event Grid namespace > **Configuration** and check these settings:
 > 
-> - **Enable MQTT** checked
-> - **Enable alternative client authentication name sources** checked
->   - Select **Certificate Subject Name** the dropdown.
-> - Set **Maximum client sessions per authentication name** to **3** or more
+> - **Enable MQTT**: Select the checkbox.
+> - **Enable alternative client authentication name sources**: Select the checkbox.
+> - **Certificate Subject Name**: Select this option in the dropdown list.
+> - **Maximum client sessions per authentication name**: Set to **3** or more.
 > 
-> The alternative client authentication and max client sessions options allows dataflows to use client certificate subject name for authentication instead of MQTT CONNECT Username. This is important so that dataflows can spawn multiple instances and still be able to connect. To learn more, see [Event Grid MQTT client certificate authentication](../../event-grid/mqtt-client-certificate-authentication.md) [multi-session support](../../event-grid/mqtt-establishing-multiple-sessions-per-client.md).
+> The alternative client authentication and maximum client sessions options allow dataflows to use client certificate subject name for authentication instead of `MQTT CONNECT Username`. This capability is important so that dataflows can spawn multiple instances and still be able to connect. To learn more, see [Event Grid MQTT client certificate authentication](../../event-grid/mqtt-client-certificate-authentication.md) and [Multi-session support](../../event-grid/mqtt-establishing-multiple-sessions-per-client.md).
 
 System-assigned managed identity:
 
@@ -211,7 +214,7 @@ Kafka endpoints are used for Kafka sources and destinations. You can configure t
 
 #### Azure Event Hubs
 
-To configure an Azure Event Hubs Kafka, the recommended way is to use managed identity for authentication.
+To configure an Azure Event Hubs Kafka, we recommend that you use managed identity for authentication.
 
 ```yaml
 apiVersion: connectivity.iotoperations.azure.com/v1beta1
@@ -232,7 +235,7 @@ spec:
 
 #### Other Kafka brokers
 
-For example, to configure a Kafka endpoint set the host, TLS, authentication, and other settings as needed.
+For example, to configure a Kafka endpoint, set the host, TLS, authentication, and other settings as needed.
 
 ```yaml
 apiVersion: connectivity.iotoperations.azure.com/v1beta1
@@ -250,7 +253,7 @@ spec:
     consumerGroupId: mqConnector
 ```
 
-Under `authentication` , you can configure the authentication method for the Kafka broker. Supported methods include SASL, X.509, system-assigned managed identity, and user-assigned managed identity.
+Under `authentication`, you can configure the authentication method for the Kafka broker. Supported methods include SASL, X.509, system-assigned managed identity, and user-assigned managed identity.
 
 ```yaml
 authentication:
@@ -303,13 +306,15 @@ spec:
 ```
 
 > [!IMPORTANT]
-> By default, data flows don't send MQTT message user properties to Kafka destinations. These user properties include values such as `subject` that stores the name of the asset sending the message. To include user properties in the Kafka message, you must update the `DataflowEndpoint` configuration to include: `copyMqttProperties: enabled`.
+> By default, data flows don't send MQTT message user properties to Kafka destinations. These user properties include values such as `subject` that stores the name of the asset sending the message. To include user properties in the Kafka message, you must update the `DataflowEndpoint` configuration to include `copyMqttProperties: enabled`.
+
+## Endpoint type for destinations only
 
-## Endpoint types for destinations only
+The following endpoint type is used for destinations only.
 
 ### Local storage and Edge Storage Accelerator
 
-Use the local storage option to send data to a locally available persistent volume, through which you can upload data via Edge Storage Accelerator (ESA) edge volumes. In this case, the format must be parquet.
+Use the local storage option to send data to a locally available persistent volume, through which you can upload data via Edge Storage Accelerator edge volumes. In this case, the format must be Parquet.
 
 ```yaml
 apiVersion: connectivity.iotoperations.azure.com/v1beta1