Skip to content

Commit 3630ad4

Browse files
authored
Update stream-analytics-define-kafka-input.md
1 parent 564bf80 commit 3630ad4

File tree

1 file changed

+24
-21
lines changed

1 file changed

+24
-21
lines changed

articles/stream-analytics/stream-analytics-define-kafka-input.md

Lines changed: 24 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
---
22
title: Stream data from Kafka into Azure Stream Analytics
3-
description: Learn about setting up Azure Stream Analytics as a consumer from kafka
3+
description: Learn about setting up Azure Stream Analytics as a consumer from Kafka
44
author: enkrumah
55
ms.author: ebnkruma
66
ms.service: stream-analytics
77
ms.topic: conceptual
8-
ms.date: 09/25/2023
8+
ms.date: 09/26/2023
99
---
1010

1111
# Stream data from Kafka into Azure Stream Analytics (Preview)
1212

13-
Kafka is a distributed streaming platform used to publish and subscribe to streams of records. Kafka is designed to allow your apps to process records as they occur. It is an open-source system developed by the Apache Software Foundation written in Java and Scala.
13+
Kafka is a distributed streaming platform used to publish and subscribe to streams of records. Kafka is designed to allow your apps to process records as they occur. It is an open-source system developed by the Apache Software Foundation and written in Java and Scala.
1414

1515
The following are the major use cases:
1616
* Messaging
@@ -19,41 +19,42 @@ The following are the major use cases:
1919
* Log Aggregation
2020
* Stream Processing
2121

22-
Azure Stream Analytics allows you to connect directly to kafka clusters to ingest data.The solution is low code and entirely managed by the Azure Stream Analytics team at Microsoft, allowing it to meet business compliance standards. The Kafka Adapters are backward compatible and support all Kafka versions with the latest client release starting from version 0.10. Users can connect to Kafka clusters inside a VNET and Kafka clusters with a public endpoint depending on the configurations. The configuration relies on existing Kafka configuration conventions.
22+
Azure Stream Analytics lets you connect directly to Kafka clusters to ingest data. The solution is low code and entirely managed by the Azure Stream Analytics team at Microsoft, allowing it to meet business compliance standards. The Kafka Adapters are backward compatible and support all versions with the latest client release starting from version 0.10. Users can connect to Kafka clusters inside a VNET and Kafka clusters with a public endpoint, depending on the configurations. The configuration relies on existing Kafka configuration conventions.
2323

2424
### Compression
2525

26-
Supported compression types are: None, Gzip, Snappy, LZ4 and Zstd compression.
26+
Supported compression types are None, Gzip, Snappy, LZ4, and Zstd.
2727

2828
## Authentication and Encryption
2929

3030
You can use four types of security protocols to connect to your Kafka clusters:
31-
| Property name | Description |
32-
|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
33-
| mTLS | encryption and authentication |
34-
| SASL_SSL | It combines two different security mechanisms - SASL (Simple Authentication and Security Layer) and SSL (Secure Sockets Layer) - to ensure both authentication and encryption are in place for data transmission. |
35-
| Kafka topic | A unit of your Kafka cluster you want to write events to. |
36-
| SASL_PLAINTEXT | standard authentication with username and password without encryption |
37-
| None | The serialization format (JSON, CSV, Avro, Parquet) of the incoming data stream. |
31+
32+
|Property name |Description |
33+
|----------|-----------|
34+
|mTLS |encryption and authentication |
35+
|SASL_SSL |It combines two different security mechanisms - SASL (Simple Authentication and Security Layer) and SSL (Secure Sockets Layer) - to ensure both authentication and encryption are in place for data transmission. |
36+
|Kafka topic |A unit of your Kafka cluster you want to write events to. |
37+
|SASL_PLAINTEXT |standard authentication with username and password without encryption |
38+
|None |The serialization format (JSON, CSV, Avro, Parquet) of the incoming data stream. |
3839

3940

4041
> [!IMPORTANT]
41-
> Confluent Cloud supports authenticating using API Keys, OAuth, or SAML single sign-on (SSO). Azure Stream Analytics do not currently support these authentication options. There will be a update in the near future to supporting authenticating to confluent cloud using API Keys.
42+
> Confluent Cloud supports authenticating using API Keys, OAuth, or SAML single sign-on (SSO). Azure Stream Analytics does not currently support these authentication options. There will be an update shortly to support authenticating to the confluent cloud using API Keys.
4243
>
4344
4445

4546
### Key Vault Integration
4647

4748
> [!NOTE]
48-
> When using truststore certificates with mTLS or SASL_SSL security protocols, you must have Azure KeyVault and managed identity configured for your Azure Stream Analytics job.
49+
> When using trust store certificates with mTLS or SASL_SSL security protocols, you must have Azure KeyVault and managed identity configured for your Azure Stream Analytics job.
4950
>
5051
51-
Azure Stream Analytics integrates seamlessly with Azure Key Vault to access stored secrets needed for authentication and encryption when using mTLS or SASL_SSL security protocols. Your Azure Stream Analytics job connects to Azure Key Vault using managed identity to ensure secure connection and avoid exfiltration of secrets.
52+
Azure Stream Analytics integrates seamlessly with Azure Key Vault to access stored secrets needed for authentication and encryption when using mTLS or SASL_SSL security protocols. Your Azure Stream Analytics job connects to Azure Key Vault using managed identity to ensure a secure connection and avoid the exfiltration of secrets.
5253

53-
You can store the certificates as Key Vault certificate or Key Vault secret. Private keys are in PEM format.
54+
You can store the certificates as Key Vault certificates or Key Vault secrets. Private keys are in PEM format.
5455

5556
### Key Vault Integration
56-
When configuring your Azure Stream Analytics job to connect to your Kafka clusters, depending on your configuration, you may have to configure your job to be able to access your Kafka clusters which are behind a firewall or inside a virtual network. Please visit the Azure Stream Analytics VNET documentation to learn more about configuring private endpoints to access resources which are inside a virtual network or behind a firewall.
57+
When configuring your Azure Stream Analytics job to connect to your Kafka clusters, depending on your configuration, you may have to configure your job to access your Kafka clusters, which are behind a firewall or inside a virtual network. You can visit the Azure Stream Analytics VNET documentation to learn more about configuring private endpoints to access resources inside a virtual network or behind a firewall.
5758

5859

5960
### Configuration
@@ -62,18 +63,20 @@ The following table lists the property names and their description for creating
6263
| Property name | Description |
6364
|------------------------------|-------------------------------------------------------------------------------------------------------------------------|
6465
| Input/Output Alias | A friendly name used in queries to reference your input or output |
65-
| Bootstrap server addresses | A list of host/port pairs to use for establishing the connection to the Kafka cluster. |
66+
| Bootstrap server addresses | A list of host/port pairs to establish the connection to the Kafka cluster. |
6667
| Kafka topic | A unit of your Kafka cluster you want to write events to. |
67-
| Security Protocol | How you want to connect to your Kafka cluster. Azure Stream Analytics supports: mTLS, SASL_SSL, SASL_PLAINTEXT or None. |
68+
| Security Protocol | How you want to connect to your Kafka cluster. Azure Stream Analytics supports mTLS, SASL_SSL, SASL_PLAINTEXT or None. |
6869
| Event Serialization format | The serialization format (JSON, CSV, Avro, Parquet, Protobuf) of the incoming data stream. |
6970

71+
72+
7073
### Limitations
71-
* When configuring your Azure Stream Analytics jobs to use VNET/SWIFT, your job must be configured with a minimum of six (6) streaming units.
74+
* When configuring your Azure Stream Analytics jobs to use VNET/SWIFT, your job must be configured with at least six (6) streaming units.
7275
* When using mTLS or SASL_SSL with Azure Key Vault, you must convert your Java Key Store to PEM format.
7376
* The minimum version of Kafka you can configure Azure Stream Analytics to connect to is version 0.10.
7477

7578
> [!NOTE]
76-
> For direct on using the Azure Stream Analytics kafka adapter, please reach out to [[email protected]](mailto:[email protected]).
79+
> For direct help with using the Azure Stream Analytics Kafka adapter, please reach out to [[email protected]](mailto:[email protected]).
7780
>
7881
7982

0 commit comments

Comments
 (0)