Skip to content

Commit 7c479a6

Browse files
committed
Kafka - OAuth updates
1 parent 57f1a83 commit 7c479a6

5 files changed

+42
-8
lines changed

articles/event-hubs/authenticate-managed-identity.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,13 +133,17 @@ var ehClient = EventHubClient.CreateWithManagedIdentity(new Uri($"sb://{EventHub
133133
```
134134
---
135135

136+
## Event Hubs for Kafka
137+
You can use Apache Kafka applications to send messages to and receive messages from Azure Event Hubs using managed identity OAuth. For a sample, see [Event Hubs for Kafka - send and receive messages using managed identity OAuth](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth/java/managedidentity).
138+
136139
## Samples
137140
- [Microsoft.Azure.EventHubs samples](https://github.com/Azure/azure-event-hubs/tree/master/samples/DotNet/Microsoft.Azure.EventHubs/Rbac).
138141

139142
These samples use the old **Microsoft.Azure.EventHubs** library, but you can easily update it to using the latest **Azure.Messaging.EventHubs** library. To move the sample from using the old library to new one, see the [Guide to migrate from Microsoft.Azure.EventHubs to Azure.Messaging.EventHubs](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/migration-guide-from-v4.md).
140143
- [Azure.Messaging.EventHubs samples](https://github.com/Azure/azure-event-hubs/tree/master/samples/DotNet/Azure.Messaging.EventHubs/ManagedIdentityWebApp)
141144

142145
This sample has been updated to use the latest **Azure.Messaging.EventHubs** library.
146+
- [Event Hubs for Kafka - send and receive messages using managed identity OAuth](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth/java/managedidentity)
143147

144148

145149
## Next steps

articles/event-hubs/authorize-access-azure-active-directory.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ For more information about how built-in roles are defined, see [Understand role
6262
- [Azure.Messaging.EventHubs samples](https://github.com/Azure/azure-event-hubs/tree/master/samples/DotNet/Azure.Messaging.EventHubs/ManagedIdentityWebApp)
6363

6464
This sample has been updated to use the latest **Azure.Messaging.EventHubs** library.
65+
- [Event Hubs for Kafka samples](https://github.com/Azure/azure-event-hubs-for-kafka/). You can find samples in the quickstart and tutorials folders.
66+
6567

6668
## Next steps
6769
- Learn how to assign a built-in-RBAC role to a security principal, see [Authenticate access to Event Hubs resources using Azure Active Directory](authenticate-application.md).

articles/event-hubs/authorize-access-event-hubs.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ documentationcenter: ''
77
author: spelluru
88

99
ms.topic: conceptual
10-
ms.date: 08/22/2019
10+
ms.date: 02/12/2020
1111
ms.author: spelluru
1212

1313
---
@@ -16,6 +16,12 @@ Every time you publish or consume events/data from an event hub, your client is
1616

1717
Azure Event Hubs offers the following options for authorizing access to secure resources:
1818

19+
- Azure Active Directory
20+
- Shared access signature
21+
22+
> [!NOTE]
23+
> This article applies to both Event Hubs and [Apache Kafka](event-hubs-for-kafka-ecosystem-overview.md) scenarios.
24+
1925
## Azure Active Directory
2026
Azure Active Directory (Azure AD) integration for Event Hubs resources provides role-based access control (RBAC) for fine-grained control over a client’s access to resources. You can use role-based access control (RBAC) to grant permissions to security principal, which may be a user, a group, or an application service principal. The security principal is authenticated by Azure AD to return an OAuth 2.0 token. The token can be used to authorize a request to access an Event Hubs resource.
2127

articles/event-hubs/event-hubs-for-kafka-ecosystem-overview.md

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Event Hubs provides a Kafka endpoint that can be used by your existing Kafka bas
2020

2121
The Event Hubs for Kafka feature provides a protocol head on top of Azure Event Hubs that is binary compatible with Kafka versions 1.0 and later for both reading from and writing to Kafka topics. You may start using the Kafka endpoint from your applications with no code change but a minimal configuration change. You update the connection string in configurations to point to the Kafka endpoint exposed by your event hub instead of pointing to your Kafka cluster. Then, you can start streaming events from your applications that use the Kafka protocol into Event Hubs. This integration also supports frameworks like [Kafka Connect](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/connect), which is currently in preview.
2222

23-
Conceptually Kafka and Event Hubs are nearly identical: they are both partitioned logs built for streaming data. The following table maps concepts between Kafka and Event Hubs.
23+
Conceptually Kafka and Event Hubs are nearly identical: they're both partitioned logs built for streaming data. The following table maps concepts between Kafka and Event Hubs.
2424

2525
### Kafka and Event Hub conceptual mapping
2626

@@ -34,15 +34,33 @@ Conceptually Kafka and Event Hubs are nearly identical: they are both partitione
3434

3535
### Key differences between Kafka and Event Hubs
3636

37-
While [Apache Kafka](https://kafka.apache.org/) is software, which you can run wherever you choose, Event Hubs is a cloud service similar to Azure Blob Storage. There are no servers or networks to manage and no brokers to configure. You create a namespace, which is an FQDN in which your topics live, and then create Event Hubs or topics within that namespace. For more information about Event Hubs and namespaces, see [Event Hubs features](event-hubs-features.md#namespace). As a cloud service, Event Hubs uses a single stable virtual IP address as the endpoint, so clients do not need to know about the brokers or machines within a cluster.
37+
While [Apache Kafka](https://kafka.apache.org/) is software, which you can run wherever you choose, Event Hubs is a cloud service similar to Azure Blob Storage. There are no servers or networks to manage and no brokers to configure. You create a namespace, which is an FQDN in which your topics live, and then create Event Hubs or topics within that namespace. For more information about Event Hubs and namespaces, see [Event Hubs features](event-hubs-features.md#namespace). As a cloud service, Event Hubs uses a single stable virtual IP address as the endpoint, so clients don't need to know about the brokers or machines within a cluster.
3838

3939
Scale in Event Hubs is controlled by how many throughput units you purchase, with each throughput unit entitling you to 1 MB per second, or 1000 events per second of ingress. By default, Event Hubs scales up throughput units when you reach your limit with the [Auto-Inflate](event-hubs-auto-inflate.md) feature; this feature also works with the Event Hubs for Kafka feature.
4040

4141
### Security and authentication
42+
Azure Event Hubs provides multiple options to authorize access to your secure resources. Event Hubs integrates with Azure Active Directory (Azure AD), which provides a OAuth 2.0 compliant centralized authorization server. With Azure AD, you can use role-based access control (RBAC) to grant fine grained permissions to your client identities. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **OAUTHBEARER** for the mechanism. For details about RBAC roles and levels for scoping access, see [Authorize access with Azure AD](authorize-access-azure-active-directory.md).
4243

43-
Azure Event Hubs requires SSL or TLS for all communication and uses Open Authentication (OAuth) or Shared Access Signatures (SAS) for authentication. This requirement is also true for a Kafka endpoint within Event Hubs. For compatibility with Kafka, Event Hubs uses SASL PLAIN or OAUTHBEARER for authentication and SASL SSL for transport security. For more information about security in Event Hubs, see [Event Hubs authentication and security](event-hubs-authentication-and-security-model-overview.md).
44+
```xml
45+
bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093
46+
security.protocol=SASL_SSL
47+
sasl.mechanism=OAUTHBEARER
48+
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;
49+
sasl.login.callback.handler.class=CustomAuthenticateCallbackHandler;
50+
```
4451

45-
For samples that show how to use OAuth with Event Hubs for Kafka, see [samples on GitHub](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth).
52+
Event Hubs also provides the Shared Access Signatures (SAS) for delegated access to Event Hubs for Kafka resources. Authorizing access using OAuth 2.0 token-based mechanism provides superior security and ease of use over SAS. The built-in roles can also eliminate the need for ACL-based authorization, which has to be maintained and managed by the user. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **PLAIN** for the mechanism.
53+
54+
```xml
55+
bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093
56+
security.protocol=SASL_SSL
57+
sasl.mechanism=PLAIN
58+
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="{YOUR.EVENTHUBS.CONNECTION.STRING}";
59+
```
60+
61+
For a **tutorial** with step-by-step instructions to create a Kafka enabled event hub and access it using SAS or OAuth, see [Quickstart: Data streaming with Event Hubs using the Kafka protocol](event-hubs-quickstart-kafka-enabled-event-hubs.md).
62+
63+
For more **samples** that show how to use OAuth with Event Hubs for Kafka, see [samples on GitHub](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth).
4664

4765
## Other Event Hubs features available for Kafka
4866

articles/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,10 @@ When you create a standard tier Event Hubs namespace, the Kafka endpoint for the
5353
security.protocol=SASL_SSL
5454
sasl.mechanism=OAUTHBEARER
5555
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;
56-
sasl.login.callback.handler.class=class CustomAuthenticateCallbackHandler;
57-
```
56+
sasl.login.callback.handler.class=CustomAuthenticateCallbackHandler;
57+
```
58+
59+
You can find the source code for the sample handler class CustomAuthenticateCallbackHandler on [GitHub location](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth/java/appsecret/producer/src/main/java).
5860
4. Run the producer code and stream into Kafka-enabled Event Hubs:
5961

6062
```shell
@@ -82,8 +84,10 @@ When you create a standard tier Event Hubs namespace, the Kafka endpoint for the
8284
security.protocol=SASL_SSL
8385
sasl.mechanism=OAUTHBEARER
8486
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;
85-
sasl.login.callback.handler.class=class CustomAuthenticateCallbackHandler;
87+
sasl.login.callback.handler.class=CustomAuthenticateCallbackHandler;
8688
```
89+
90+
You can find the source code for the sample handler class CustomAuthenticateCallbackHandler on [GitHub location](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth/java/appsecret/consumer/src/main/java).
8791
7. Run the consumer code and process from Kafka enabled Event Hubs using your Kafka clients:
8892

8993
```java

0 commit comments

Comments
 (0)