Skip to content

Commit 005cc99

Browse files
authored
Merge pull request #311468 from spelluru/sbusfreshness0205
Azure messaging - Freshness
2 parents 2f508f4 + 34d4562 commit 005cc99

51 files changed

Lines changed: 189 additions & 173 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

articles/event-grid/index.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ metadata:
99
ms.topic: landing-page
1010
author: spelluru
1111
ms.author: spelluru
12-
ms.date: 12/12/2024
12+
ms.date: 02/05/2026
1313
# linkListType: architecture | concept | deploy | download | get-started | how-to-guide | learn | overview | quickstart | reference | tutorial | whats-new
1414

1515
landingContent:

articles/event-hubs/azure-event-hubs-apache-kafka-overview.md

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
---
2-
title: Azure Event Hubs for Apache Kafka
2+
title: Apache Kafka Protocol Support in Azure Event Hubs
33
description: Learn what Apache Kafka in Azure Event Hubs is and how to use it to stream data from Apache Kafka applications without setting up a Kafka cluster on your own.
4+
#customer intent: As a developer using Apache Kafka, I want to understand how Azure Event Hubs supports the Kafka protocol so that I can integrate my applications without modifying code.
45
ms.topic: overview
56
ms.subservice: kafka
6-
ms.date: 12/12/2024
7+
ms.date: 02/05/2026
78
# Customer intent: How does Azure Event Hubs support the Apache Kafka protocol?
89
---
910

@@ -39,7 +40,7 @@ Conceptually, Apache Kafka and Event Hubs are very similar. They're both partiti
3940
Kafka Streams is a client library for stream analytics that is part of the Apache Kafka open-source project, but is separate from the Apache Kafka event broker.
4041

4142
> [!NOTE]
42-
> Kafka Streams is currently in Public preview in Premium, and Dedicated tier.
43+
> Kafka Streams is currently in public preview in Premium and Dedicated tier.
4344
4445
Azure Event Hubs supports the Kafka Streams client library, with details and concepts available [here](apache-kafka-streams.md).
4546

@@ -60,28 +61,28 @@ Standalone and without ksqlDB, Kafka Streams has fewer capabilities than many al
6061
### Kafka Transactions
6162

6263
> [!NOTE]
63-
> Kafka Transactions is currently in Public preview in Premium, and Dedicated tier.
64+
> Kafka Transactions is currently in public preview in Premium and Dedicated tier.
6465
>
6566
66-
Azure Event Hubs supports Kafka transactions. More details regarding the support and concepts is available [here](apache-kafka-transactions.md)
67+
Azure Event Hubs supports Kafka transactions. For more details about the support and concepts, see [Apache Kafka transactions](apache-kafka-transactions.md).
6768

6869
### Compression
6970

7071
> [!NOTE]
71-
> The Kafka compression for Event Hubs is only supported in Premium and Dedicated tiers currently.
72+
> Kafka compression for Event Hubs is currently supported only in Premium and Dedicated tiers.
7273
>
7374
7475
The client-side [compression](https://cwiki.apache.org/confluence/display/KAFKA/Compression) feature in Apache Kafka clients conserves compute resources and bandwidth by compressing a batch of multiple messages into a single message on the producer side and decompressing the batch on the consumer side. The Apache Kafka broker treats the batch as a special message.
7576

76-
Kafka producer application developers can enable message compression by setting the compression.type property. Azure Event Hubs currently supports `gzip` compression.
77+
Kafka producer application developers can enable message compression by setting the `compression.type` property. Azure Event Hubs currently supports `gzip` compression.
7778

7879
```properties
7980

8081
Compression.type = none | gzip
8182

8283
```
8384

84-
While the feature is only supported for Apache Kafka traffic producer and consumer traffic, AMQP consumer can consume compressed Kafka traffic as decompressed messages.
85+
While the feature is only supported for Apache Kafka producer traffic and consumer traffic, AMQP consumer can consume compressed Kafka traffic as decompressed messages.
8586

8687

8788
## Key differences between Apache Kafka and Azure Event Hubs
@@ -94,24 +95,24 @@ Scale in Event Hubs is controlled by how many [throughput units (TUs)](event-hub
9495

9596
## Is Apache Kafka the right solution for your workload?
9697

97-
Coming from building applications using Apache Kafka, it's also useful to understand that Azure Event Hubs is part of a fleet of services, which also includes [Azure Service Bus](../service-bus-messaging/service-bus-messaging-overview.md), and [Azure Event Grid](../event-grid/overview.md).
98+
If you come from a background of building applications that use Apache Kafka, it's also useful to understand that Azure Event Hubs is part of a fleet of services that also includes [Azure Service Bus](../service-bus-messaging/service-bus-messaging-overview.md) and [Azure Event Grid](../event-grid/overview.md).
9899

99-
While some providers of commercial distributions of Apache Kafka might suggest that Apache Kafka is a one-stop-shop for all your messaging platform needs, the reality is that Apache Kafka doesn't implement, for instance, the [competing-consumer](/azure/architecture/patterns/competing-consumers) queue pattern, doesn't have support for [publish-subscribe](/azure/architecture/patterns/publisher-subscriber) at a level that allows subscribers access to the incoming messages based on server-evaluated rules other than plain offsets, and it has no facilities to track the lifecycle of a job initiated by a message or sidelining faulty messages into a dead-letter queue, all of which are foundational for many enterprise messaging scenarios.
100+
While some providers of commercial distributions of Apache Kafka might suggest that Apache Kafka is a one-stop shop for all your messaging platform needs, the reality is that Apache Kafka doesn't implement, for instance, the [competing-consumer](/azure/architecture/patterns/competing-consumers) queue pattern. It doesn't support [publish-subscribe](/azure/architecture/patterns/publisher-subscriber) at a level that allows subscribers access to the incoming messages based on server-evaluated rules other than plain offsets. It has no facilities to track the lifecycle of a job initiated by a message or sidelining faulty messages into a dead-letter queue. All of these features are foundational for many enterprise messaging scenarios.
100101

101-
To understand the differences between patterns and which pattern is best covered by which service, see the [Asynchronous messaging options in Azure](/azure/architecture/guide/technology-choices/messaging) guidance. As an Apache Kafka user, you can find that communication paths you have so far realized with Kafka, can be realized with far less basic complexity and yet more powerful capabilities using either Event Grid or Service Bus.
102+
To understand the differences between patterns and which pattern is best covered by which service, see the [Asynchronous messaging options in Azure](/azure/architecture/guide/technology-choices/messaging) guidance. As an Apache Kafka user, you can find that communication paths you realized with Kafka can be realized with far less basic complexity and yet more powerful capabilities by using either Event Grid or Service Bus.
102103

103104
If you need specific features of Apache Kafka that aren't available through the Event Hubs for Apache Kafka interface or if your implementation pattern exceeds the [Event Hubs quotas](event-hubs-quotas.md), you can also run a [native Apache Kafka cluster in Azure HDInsight](../hdinsight/kafka/apache-kafka-introduction.md).
104105

105106
## Security and authentication
106-
Every time you publish or consume events from an Event Hubs for Kafka, your client is trying to access the Event Hubs resources. You want to ensure that the resources are accessed using an authorized entity. When using Apache Kafka protocol with your clients, you can set your configuration for authentication and encryption using the SASL mechanisms. When using Event Hubs for Kafka requires the TLS-encryption (as all data in transit with Event Hubs is TLS encrypted), it can be done specifying the SASL_SSL option in your configuration file.
107+
Every time you publish or consume events from an Event Hubs for Kafka, your client tries to access the Event Hubs resources. You want to ensure that the resources are accessed using an authorized entity. When using Apache Kafka protocol with your clients, set your configuration for authentication and encryption using the SASL mechanisms. When using Event Hubs for Kafka, TLS-encryption is required (as all data in transit with Event Hubs is TLS encrypted). You can specify the SASL_SSL option in your configuration file.
107108

108-
Azure Event Hubs provides multiple options to authorize access to your secure resources.
109+
Azure Event Hubs provides multiple options to authorize access to your secure resources:
109110

110111
- OAuth 2.0
111112
- Shared access signature (SAS)
112113

113114
### OAuth 2.0
114-
Event Hubs integrates with Microsoft Entra ID, which provides an **OAuth 2.0** compliant centralized authorization server. With Microsoft Entra ID, you can use Azure role-based access control (Azure RBAC) to grant fine grained permissions to your client identities. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **OAUTHBEARER** for the mechanism. For details about Azure roles and levels for scoping access, see [Authorize access with Microsoft Entra ID](authorize-access-azure-active-directory.md).
115+
Event Hubs integrates with Microsoft Entra ID, which provides an **OAuth 2.0** compliant centralized authorization server. By using Microsoft Entra ID, you can use Azure role-based access control (Azure RBAC) to grant fine grained permissions to your client identities. Use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **OAUTHBEARER** for the mechanism. For details about Azure roles and levels for scoping access, see [Authorize access with Microsoft Entra ID](authorize-access-azure-active-directory.md).
115116

116117
```properties
117118
bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093
@@ -125,7 +126,7 @@ sasl.login.callback.handler.class=CustomAuthenticateCallbackHandler
125126
> These configuration properties are for the Java programming language. For **samples** that show how to use OAuth with Event Hubs for Kafka using different programming languages, see [samples on GitHub](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth).
126127
127128
### Shared Access Signature (SAS)
128-
Event Hubs also provides the **Shared Access Signatures (SAS)** for delegated access to Event Hubs for Kafka resources. Authorizing access using OAuth 2.0 token-based mechanism provides superior security and ease of use over SAS. The built-in roles can also eliminate the need for ACL-based authorization, which has to be maintained and managed by the user. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **PLAIN** for the mechanism.
129+
Event Hubs also provides **Shared Access Signatures (SAS)** for delegated access to Event Hubs for Kafka resources. Authorizing access using the OAuth 2.0 token-based mechanism provides superior security and ease of use over SAS. The built-in roles can also eliminate the need for ACL-based authorization, which you have to maintain and manage. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **PLAIN** for the mechanism.
129130

130131
```properties
131132
bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093
@@ -156,10 +157,10 @@ Additionally, Event Hubs features such as [Capture](event-hubs-capture-overview.
156157

157158
Azure Event Hubs for Apache Kafka supports both idempotent producers and idempotent consumers.
158159

159-
One of the core tenets of Azure Event Hubs is the concept of **at-least once** delivery. This approach ensures that events are always delivered. It also means that events can be received more than once, even repeatedly, by consumers such as a function. For this reason, it's important that the consumer supports the [idempotent consumer](https://microservices.io/patterns/communication-style/idempotent-consumer.html) pattern.
160+
One of the core tenets of Azure Event Hubs is the concept of **at-least once** delivery. This approach ensures that events are always delivered. It also means that consumers can receive events more than once, even repeatedly. For this reason, it's important that the consumer supports the [idempotent consumer](https://microservices.io/patterns/communication-style/idempotent-consumer.html) pattern.
160161

161162
## Related content
162-
This article provided an introduction to Event Hubs for Kafka. To learn more, see [Apache Kafka developer guide for Azure Event Hubs](apache-kafka-developer-guide.md).
163+
This article introduces Event Hubs for Kafka. To learn more, see [Apache Kafka developer guide for Azure Event Hubs](apache-kafka-developer-guide.md).
163164

164165
For a **tutorial** with step-by-step instructions to create an event hub and access it using SAS or OAuth, see [Quickstart: Data streaming with Event Hubs using the Kafka protocol](event-hubs-quickstart-kafka-enabled-event-hubs.md).
165166

articles/event-hubs/event-hubs-capture-enable-through-portal.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ To create an event hub within the namespace, follow these steps:
3333

3434
1. On the **Overview** page for your namespace, select **+ Event hub** on the command bar.
3535

36-
:::image type="content" source="./media/event-hubs-quickstart-portal/create-event-hub4.png" lightbox="./media/event-hubs-quickstart-portal/create-event-hub4.png" alt-text="Screenshot of the selection of Add event hub button on the command bar.":::
36+
:::image type="content" source="./media/event-hubs-quickstart-portal/create-event-hub-button.png" lightbox="./media/event-hubs-quickstart-portal/create-event-hub-button.png" alt-text="Screenshot of the selection of Add event hub button on the command bar.":::
3737
2. On the **Create event hub** page, type a name for your event hub, then select **Next: Capture** at the bottom of the page.
3838

3939
:::image type="content" source="./media/event-hubs-capture-enable-through-portal/create-event-hub-basics-page.png" alt-text="Screenshot of the Create event hub page.":::

0 commit comments

Comments
 (0)