You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/stream-analytics-define-kafka-input.md
+24-21Lines changed: 24 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,16 @@
1
1
---
2
2
title: Stream data from Kafka into Azure Stream Analytics
3
-
description: Learn about setting up Azure Stream Analytics as a consumer from kafka
3
+
description: Learn about setting up Azure Stream Analytics as a consumer from Kafka
4
4
author: enkrumah
5
5
ms.author: ebnkruma
6
6
ms.service: stream-analytics
7
7
ms.topic: conceptual
8
-
ms.date: 09/25/2023
8
+
ms.date: 09/26/2023
9
9
---
10
10
11
11
# Stream data from Kafka into Azure Stream Analytics (Preview)
12
12
13
-
Kafka is a distributed streaming platform used to publish and subscribe to streams of records. Kafka is designed to allow your apps to process records as they occur. It is an open-source system developed by the Apache Software Foundation written in Java and Scala.
13
+
Kafka is a distributed streaming platform used to publish and subscribe to streams of records. Kafka is designed to allow your apps to process records as they occur. It is an open-source system developed by the Apache Software Foundation and written in Java and Scala.
14
14
15
15
The following are the major use cases:
16
16
* Messaging
@@ -19,41 +19,42 @@ The following are the major use cases:
19
19
* Log Aggregation
20
20
* Stream Processing
21
21
22
-
Azure Stream Analytics allows you to connect directly to kafka clusters to ingest data.The solution is low code and entirely managed by the Azure Stream Analytics team at Microsoft, allowing it to meet business compliance standards. The Kafka Adapters are backward compatible and support all Kafka versions with the latest client release starting from version 0.10. Users can connect to Kafka clusters inside a VNET and Kafka clusters with a public endpoint depending on the configurations. The configuration relies on existing Kafka configuration conventions.
22
+
Azure Stream Analytics lets you connect directly to Kafka clusters to ingest data.The solution is low code and entirely managed by the Azure Stream Analytics team at Microsoft, allowing it to meet business compliance standards. The Kafka Adapters are backward compatible and support all versions with the latest client release starting from version 0.10. Users can connect to Kafka clusters inside a VNET and Kafka clusters with a public endpoint, depending on the configurations. The configuration relies on existing Kafka configuration conventions.
| SASL_SSL | It combines two different security mechanisms - SASL (Simple Authentication and Security Layer) and SSL (Secure Sockets Layer) - to ensure both authentication and encryption are in place for data transmission. |
35
-
| Kafka topic | A unit of your Kafka cluster you want to write events to. |
36
-
| SASL_PLAINTEXT | standard authentication with username and password without encryption |
37
-
| None | The serialization format (JSON, CSV, Avro, Parquet) of the incoming data stream. |
31
+
32
+
|Property name |Description |
33
+
|----------|-----------|
34
+
|mTLS |encryption and authentication |
35
+
|SASL_SSL |It combines two different security mechanisms - SASL (Simple Authentication and Security Layer) and SSL (Secure Sockets Layer) - to ensure both authentication and encryption are in place for data transmission. |
36
+
|Kafka topic |A unit of your Kafka cluster you want to write events to. |
37
+
|SASL_PLAINTEXT |standard authentication with username and password without encryption |
38
+
|None |The serialization format (JSON, CSV, Avro, Parquet) of the incoming data stream. |
38
39
39
40
40
41
> [!IMPORTANT]
41
-
> Confluent Cloud supports authenticating using API Keys, OAuth, or SAML single sign-on (SSO). Azure Stream Analytics do not currently support these authentication options. There will be a update in the near future to supporting authenticating to confluent cloud using API Keys.
42
+
> Confluent Cloud supports authenticating using API Keys, OAuth, or SAML single sign-on (SSO). Azure Stream Analytics does not currently support these authentication options. There will be an update shortly to support authenticating to the confluent cloud using API Keys.
42
43
>
43
44
44
45
45
46
### Key Vault Integration
46
47
47
48
> [!NOTE]
48
-
> When using truststore certificates with mTLS or SASL_SSL security protocols, you must have Azure KeyVault and managed identity configured for your Azure Stream Analytics job.
49
+
> When using trust store certificates with mTLS or SASL_SSL security protocols, you must have Azure KeyVault and managed identity configured for your Azure Stream Analytics job.
49
50
>
50
51
51
-
Azure Stream Analytics integrates seamlessly with Azure Key Vault to access stored secrets needed for authentication and encryption when using mTLS or SASL_SSL security protocols. Your Azure Stream Analytics job connects to Azure Key Vault using managed identity to ensure secure connection and avoid exfiltration of secrets.
52
+
Azure Stream Analytics integrates seamlessly with Azure Key Vault to access stored secrets needed for authentication and encryption when using mTLS or SASL_SSL security protocols. Your Azure Stream Analytics job connects to Azure Key Vault using managed identity to ensure a secure connection and avoid the exfiltration of secrets.
52
53
53
-
You can store the certificates as Key Vault certificate or Key Vault secret. Private keys are in PEM format.
54
+
You can store the certificates as Key Vault certificates or Key Vault secrets. Private keys are in PEM format.
54
55
55
56
### Key Vault Integration
56
-
When configuring your Azure Stream Analytics job to connect to your Kafka clusters, depending on your configuration, you may have to configure your job to be able to access your Kafka clusters which are behind a firewall or inside a virtual network. Please visit the Azure Stream Analytics VNET documentation to learn more about configuring private endpoints to access resources which are inside a virtual network or behind a firewall.
57
+
When configuring your Azure Stream Analytics job to connect to your Kafka clusters, depending on your configuration, you may have to configure your job to access your Kafka clusters, which are behind a firewall or inside a virtual network. You can visit the Azure Stream Analytics VNET documentation to learn more about configuring private endpoints to access resources inside a virtual network or behind a firewall.
57
58
58
59
59
60
### Configuration
@@ -62,18 +63,20 @@ The following table lists the property names and their description for creating
0 commit comments