azure-docs/articles/event-hubs/apache-kafka-frequently-asked-questions.yml at fe3d7e1aa0f8e37f19985ebfb10de416abdca504 · MicrosoftDocs/azure-docs · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
### YamlMime:FAQ
metadata:
  title: Frequently asked questions - Azure Event Hubs for Apache Kafka
  description: This article answers frequent questions asked about Azure Event Hubs' support for Apache Kafka clients not covered elsewhere.
  ms.topic: faq
  ms.subservice: kafka

  ms.date: 03/06/2025
title: Frequently asked questions - Event Hubs for Apache Kafka
summary: This article provides answers to some of the frequently asked questions on migrating to Event Hubs for Apache Kafka.


sections:
  - name: Ignored
    questions:
      - question: |
          Does Azure Event Hubs run on Apache Kafka?
        answer: |
          No. Azure Event Hubs is a cloud-native multi-tier broker with support for multiple protocols that is developed and maintained by Microsoft and doesn't use any Apache Kafka code. One of the supported protocols is the Kafka RPC protocol for the Kafka client's consumer and producer APIs. Event Hubs works with many of your existing Kafka applications. For more information, see [Event Hubs for Apache Kafka](azure-event-hubs-apache-kafka-overview.md). Because the concepts of Apache Kafka and Azure Event Hubs are very similar (but not identical), we're able to offer the unmatched reliability of Azure Event Hubs to customers with existing Apache Kafka investments.

      - question: |
          Event Hubs consumer group vs. Kafka consumer group
        answer: |
          What's the difference between an Event Hubs consumer group and a Kafka consumer group on Event Hubs? Kafka consumer groups on Event Hubs are fully distinct from standard Event Hubs consumer groups.

          **Event Hubs consumer groups**

          - They're Managed with create, retrieve, update, and delete (CRUD) operations via portal, SDK, or Azure Resource Manager templates. Event Hubs consumer groups can't be autocreated.
          - They're children entities of an event hub. It means that the same consumer group name can be reused between event hubs in the same namespace because they're separate entities.
          - They aren't used for storing offsets. Orchestrated AMQP consumption is done using external offset storage, for example, Azure Storage.

          **Kafka consumer groups**

          - They're autocreated.  Kafka groups can be managed via the Kafka consumer group APIs.
          - They can store offsets in the Event Hubs service.
          - They're used as keys in what is effectively an offset key-value store. For a unique pair of `group.id` and `topic-partition`, we store an offset in Azure Storage (3x replication). Event Hubs users don't incur extra storage costs from storing Kafka offsets. Offsets are manipulable via the Kafka consumer group APIs, but the offset storage *accounts* aren't directly visible or manipulable for Event Hubs users.
          - They span a namespace. Using the same Kafka group name for multiple applications on multiple topics means that all applications and their Kafka clients are rebalanced whenever only a single application needs rebalancing. Choose your group names wisely.
          - They fully distinct from Event Hubs consumer groups. You **don't** need to use `$Default`, nor do you need to worry about Kafka clients interfering with AMQP workloads.
          - They aren't viewable in the Azure portal. Consumer group info is accessible via Kafka APIs.

      - question: |
          Does Azure Event Hubs for Apache Kafka support shared access signature token authentication?
        answer: |
          Authenticating by using [OAuth 2.0 and shared access signature](azure-event-hubs-apache-kafka-overview.md#security-and-authentication) is supported.

          Shared access signature tokens are [generated](authenticate-shared-access-signature.md#generate-a-shared-access-signature-token) by using an authorization rule and one of its signing keys. It isn't supported when using the Event Hubs for Apache Kafka endpoint.

      - question: |
          Does Azure Event Hubs support Apache Kafka idempotent producers and consumers?
        answer: |
          Yes. Azure Event Hubs supports Apache Kafka idempotent producers and consumers.

additionalContent: |

  ## Next steps

  To learn more about Event Hubs and Event Hubs for Kafka, see the following articles:

  - [Apache Kafka developer guide for Event Hubs](apache-kafka-developer-guide.md)
  - [Apache Kafka migration guide for Event Hubs](apache-kafka-migration-guide.md)
  - [Apache Kafka troubleshooting guide for Event Hubs](apache-kafka-troubleshooting-guide.md)
  - [Recommended configurations](apache-kafka-configurations.md)