Merge pull request #253977 from kasun04/main

Revamp Event Hubs intro

Author: v-shils

articles/event-hubs/event-hubs-about.md

@@ -1,79 +1,88 @@
 ---
-title: What is Azure Event Hubs? - a Big Data ingestion service
-description: Learn about Azure Event Hubs, a Big Data streaming service that ingests millions of events per second.
+title: Azure Event Hubs – data streaming platform with Kafka support
+description: Learn about Azure Event Hubs, A real-time data streaming platform with native Apache Kafka support.
 ms.topic: overview
-ms.date: 03/07/2023
+ms.date: 10/09/2023
 ---
 
-# What is Azure Event Hubs? — A big data streaming platform and event ingestion service
-Event Hubs is a modern big data streaming platform and event ingestion service that can seamlessly integrate with other Azure and Microsoft services, such as Stream Analytics, Power BI, and Event Grid, along with outside services like Apache Spark. The service can process millions of events per second with low latency. The data sent to an event hub (Event Hubs instance) can be transformed and stored by using any real-time analytics providers or batching or storage adapters. 
+# Azure Event Hubs – A real-time data streaming platform with native Apache Kafka support
+Azure Event Hubs is a cloud native data streaming service that can stream millions of events per second, with low latency, from any source to any destination. Event Hubs is compatible with Apache Kafka, and it enables you to run existing Kafka workloads without any code changes. 
 
-## Why use Event Hubs?
-Data is valuable only when there's an easy way to process and get timely insights from data sources. Event Hubs provides a distributed stream processing platform with low latency and seamless integration, with data and analytics services inside and outside Azure to build your complete big data pipeline.
+Using Event Hubs to ingest and store streaming data, businesses can harness the power of streaming data to gain valuable insights, drive real-time analytics, and respond to events as they happen, enhancing overall efficiency and customer experience. 
+    :::image type="content" source="./media/event-hubs-about/event-streaming-platform.png" alt-text="Diagram that shows how Azure Event Hubs fits in an event streaming platform.":::
 
-Event Hubs represents the "front door" for an event pipeline, often called an **event ingestor** in solution architectures. An event ingestor is a component or service that sits between event publishers and event consumers to decouple the production of events from the consumption of those events. Event Hubs provides a unified streaming platform with time retention buffer, decoupling event producers from event consumers.
+Azure Event Hubs is the preferred event ingestion layer of any event streaming solution that you build on top of Azure. It seamlessly integrates with data and analytics services inside and outside Azure to build your complete data streaming pipeline to serve following use cases. 
 
-The following sections describe key features of the Azure Event Hubs service:
+- [Real-time analytics with Azure Stream Analytics](./process-data-azure-stream-analytics.md) to generate real-time insights from streaming data. 
+- Analyze and explore streaming data with Azure Data Explorer. 
+- Create your own cloud native applications, functions, or microservices that run on streaming data from Event Hubs.
+- Stream events with schema validation using a built-in schema registry to ensure quality and compatibility of streaming data. 
 
-## Fully managed PaaS
-Event Hubs is a fully managed Platform-as-a-Service (PaaS) with little configuration or management overhead, so you focus on your business solutions. [Event Hubs for Apache Kafka ecosystems](azure-event-hubs-kafka-overview.md) gives you the PaaS Kafka experience without having to manage, configure, or run your clusters.
 
-## Event Hubs for Apache Kafka
-Azure Event Hubs for Apache Kafka ecosystems enables [Apache Kafka (1.0 and later)](https://kafka.apache.org/) clients and applications to talk to Event Hubs. You don't need to set up, configure, and manage your own Kafka and Zookeeper clusters or use some Kafka-as-a-Service offering not native to Azure. For more information, see [Event Hubs for Apache Kafka ecosystems](azure-event-hubs-kafka-overview.md). 
+## Key capabilities?
+### Apache Kafka on Azure Event Hubs
+Azure Event Hubs is a multi-protocol event streaming engine that natively supports AMQP, Apache Kafka and HTTPs protocols. Since it supports Apache Kafka, you bring Kafka workloads to Azure Event Hubs without doing any code change. You don't need to set up, configure, and manage your own Kafka clusters or use some Kafka-as-a-Service offering not native to Azure.
 
-## Schema Registry in Azure Event Hubs
-Schema Registry in Event Hubs provides a centralized repository for managing schemas of events streaming applications. Azure Schema Registry comes free with every Event Hubs namespace, and it integrates seamlessly with your Kafka applications or Event Hubs SDK based applications. 
+Event Hubs is built from the ground up as a cloud native broker engine. Hence you can run Kafka workloads with better performance, better cost efficiency and with no operational overhead.  
 
-It ensures data compatibility and consistency across event producers and consumers, enabling seamless schema evolution, validation, and governance, and promoting efficient data exchange and interoperability. For more information, see [Schema Registry in Azure Event Hubs](schema-registry-overview.md). 
+### Schema Registry in Azure Event Hubs 
+Azure Schema Registry in Event Hubs provides a centralized repository for managing schemas of events streaming applications. Azure Schema Registry comes free with every Event Hubs namespace, and it integrates seamlessly with your Kafka applications or Event Hubs SDK based applications.
 
-## Support for real-time and batch processing
-Ingest, buffer, store, and process your stream in real time to get actionable insights. Event Hubs uses a [partitioned consumer model](event-hubs-scalability.md#partitions), enabling multiple applications to process the stream concurrently and letting you control the speed of processing. Azure Event Hubs also integrates with [Azure Functions](../azure-functions/index.yml) for a serverless architecture.
+:::image type="content" source="./media/event-hubs-about/schema-registry.png" alt-text="Diagram that shows Schema Registry and Azure Event Hubs integration.":::
 
-## Capture event data
-Capture your data in near-real time in an [Azure Blob storage](https://azure.microsoft.com/services/storage/blobs/) or [Azure Data Lake Storage](https://azure.microsoft.com/services/data-lake-store/) for long-term retention or micro-batch processing. You can achieve this behavior on the same stream you use for deriving real-time analytics. Setting up capture of event data is fast. There are no administrative costs to run it, and it scales automatically with Event Hubs [throughput units](event-hubs-scalability.md#throughput-units) or [processing units](event-hubs-scalability.md#processing-units). Event Hubs enables you to focus on data processing rather than on data capture. For more information, see [Event Hubs Capture](event-hubs-capture-overview.md).
 
-## Scalable
-With Event Hubs, you can start with data streams in megabytes, and grow to gigabytes or terabytes. The [Autoinflate](event-hubs-auto-inflate.md) feature is one of the many options available to scale the number of throughput units or processing units to meet your usage needs.
+It ensures data compatibility and consistency across event producers and consumers.  Schema Registry enables seamless schema evolution, validation, and governance, and promoting efficient data exchange and interoperability.
+Schema Registry seamlessly integrates with your existing Kafka applications and it supports multiple schema definitions formats including Avro and JSON Schemas.  
+
+### Real-time event stream processing with Azure Stream Analytics 
+Event Hubs integrates seamlessly with Azure Stream Analytics to enable real-time stream processing. With the built-in no-code editor, you can effortlessly develop a Stream Analytics job using drag-and-drop functionality, without writing any code. 
+
+:::image type="content" source="../stream-analytics/media/filter-ingest-data-lake-storage-gen2/filter-data-lake-gen2-card-start.png" alt-text="Diagram that shows Stream Analytics no code editor templates.":::
 
-## Rich ecosystem
-With a broad ecosystem available for the industry-standard AMQP 1.0 protocol and SDKs available in various languages: [.NET](https://github.com/Azure/azure-sdk-for-net/), [Java](https://github.com/Azure/azure-sdk-for-java/), [Python](https://github.com/Azure/azure-sdk-for-python/), [JavaScript](https://github.com/Azure/azure-sdk-for-js/), you can easily start processing your streams from Event Hubs. All supported client languages provide low-level integration. The ecosystem also provides you with seamless integration with Azure services like Azure Stream Analytics and Azure Functions and thus enables you to build serverless architectures.
+Alternatively, developers can use the SQL-based Stream Analytics query language to perform real-time stream processing and take advantage of a wide range of functions for analyzing streaming data.  
 
+### Exploring streaming data with Azure Data Explorer
+Azure Data Explorer is a fully managed platform for big data analytics that delivers high performance and allows for the analysis of large volumes of data in near real time. By integrating Event Hubs with Azure Data Explorer, you can easily perform near real-time analytics and exploration of streaming data. 
 
-## Event Hubs premium and dedicated 
-Event Hubs **premium** caters to high-end streaming needs that require superior performance, better isolation with predictable latency, and minimal interference in a managed multitenant PaaS environment. On top of all the features of the standard offering, the premium tier offers several extra features such as [dynamic partition scale up](dynamically-add-partitions.md), extended retention, and [customer-managed-keys](configure-customer-managed-key.md). For more information, see [Event Hubs Premium](event-hubs-premium-overview.md).
+:::image type="content" source="./media/event-hubs-about/data-explorer-integration.png" alt-text="Diagram that shows Azure Data explorer query and output.":::
 
-Event Hubs **dedicated** tier offers single-tenant deployments for customers with the most demanding streaming needs. This single-tenant offering has a guaranteed 99.99% SLA and is available only on our dedicated pricing tier. An Event Hubs cluster can ingress millions of events per second with guaranteed capacity and subsecond latency. Namespaces and event hubs created within the dedicated cluster include all features of the premium offering and more. For more information, see [Event Hubs Dedicated](event-hubs-dedicated-overview.md).
 
-For more information, see [comparison between Event Hubs tiers](event-hubs-quotas.md).
+### Rich ecosystem– Azure functions, SDKs and Kafka ecosystem
+Ingest, buffer, store, and process your stream in real time to get actionable insights. Event Hubs uses a partitioned consumer model, enabling multiple applications to process the stream concurrently and letting you control the speed of processing. Azure Event Hubs also integrates with Azure Functions for a serverless architecture.
+With a broad ecosystem available for the industry-standard AMQP 1.0 protocol and SDKs available in various languages: .NET, Java, Python, JavaScript, you can easily start processing your streams from Event Hubs. All supported client languages provide low-level integration. 
 
-## Event Hubs on Azure Stack Hub
-The Event Hubs service on Azure Stack Hub allows you to realize hybrid cloud scenarios. Streaming and event-based solutions are supported for both on-premises and Azure cloud processing. Whether your scenario is hybrid (connected), or disconnected, your solution can support processing of events/streams at large scale. Your scenario is only bound by the Event Hubs cluster size, which you can provision according to your needs. 
+The ecosystem also provides you with seamless integration Azure Functions, Azure Spring Apps, Kafka Connectors and other data analytics platforms and technologies such as Apache Spark and Apache Flink.
 
-The Event Hubs editions (on Azure Stack Hub and on Azure) offer a high degree of feature parity. This parity means SDKs, samples, PowerShell, CLI, and portals offer a similar experience, with few differences. 
 
-For more information, see [Event Hubs on Azure Stack Hub overview](/azure-stack/user/event-hubs-overview).
+### Flexible and cost-efficient event streaming
+You can experience flexible and cost-efficient event streaming through Event Hubs' diverse selection of tiers – including Standard, Premium, and Dedicated. These options cater to data streaming needs ranging from a few MB/s to several GB/s, allowing you to choose the perfect match for your requirements. 
 
-## Key architecture components
-Event Hubs contains the following key components. 
+### Scalable
+With Event Hubs, you can start with data streams in megabytes, and grow to gigabytes or terabytes. The [Autoinflate](event-hubs-auto-inflate.md) feature is one of the many options available to scale the number of throughput units or processing units to meet your usage needs.
+
+### Capture streaming data for long term retention and batch analytics
+Capture your data in near-real time in an Azure Blob storage or Azure Data Lake Storage for long-term retention or micro-batch processing. You can achieve this behavior on the same stream you use for deriving real-time analytics. Setting up capture of event data is fast.  
 
-| Component | Description | 
-| --------- | ----------- | 
-| Event producers | Any entity that sends data to an event hub. Event publishers can publish events using HTTPS or AMQP 1.0 or Apache Kafka (1.0 and higher). |
-| Partitions | Each consumer only reads a specific subset, or a partition, of the message stream. | 
-| Consumer groups | A view (state, position, or offset) of an entire event hub. Consumer groups enable consuming applications to each have a separate view of the event stream. They read the stream independently at their own pace and with their own offsets. | 
-| Event receivers | Any entity that reads event data from an event hub. All Event Hubs consumers connect via the AMQP 1.0 session. The Event Hubs service delivers events through a session as they become available. All Kafka consumers connect via the Kafka protocol 1.0 and later. | 
-| [Throughput units (standard tier)](event-hubs-scalability.md#throughput-units) or [processing units (premium tier)](event-hubs-scalability.md#processing-units) or [capacity units (dedicated)](event-hubs-dedicated-overview.md) | Prepurchased units of capacity that control the throughput capacity of Event Hubs. | 
+## How it works? 
+Event Hubs provides a unified event streaming platform with time retention buffer, decoupling event producers from event consumers. The producers and consumer applications can perform large scale data ingestion through multiple protocols. 
 
-The following figure shows the Event Hubs stream processing architecture:
-![Event Hubs](./media/event-hubs-about/event_hubs_architecture.png)
+The following figure shows the key components of Event Hubs architecture:
+:::image type="content" source="./media/event-hubs-about/components.png" alt-text="Diagram that shows the main components of Event Hubs.":::
+The key functional components of Event Hubs include: 
+- **Event Hub/Kafka topic**: In Event Hubs, you can organize events into event hubs or Kafka topic. It's an append only distributed log, which can comprise of one or more partitions. 
+- **Partitions** are used to scale an event hub. They are like lanes in a freeway. If you need more streaming throughput, you need to add more partitions.
+- **Producer applications** can ingest data to an event hub using Event Hubs SDKs or any Kafka producer client. 
+- **Consumer applications** consume data by seeking through the event log and maintaining consumer offset. Consumers can be based on Kafka consumer clients or Event Hubs SDK as well. 
+- **Consumer Group** is a logical group of consumer instances that reads data from an event hub/Kafka topic. It enables multiple consumers to read the same streaming data in an event hub independently at their own pace and with their own offsets.
+- **Namespace** is the management container for one or more event hubs or Kafka topics. The management tasks such as allocating streaming capacity, configuring network security, enabling Geo Disaster recovery etc. are handled at the namespace level. 
 
-> [!NOTE]
-> For more information, see [Event Hubs features or components](event-hubs-features.md).
 
 ## Next steps
 
-To get started using Event Hubs, see the **Send and receive events** tutorials:
+To get started using Event Hubs, see the following quick start guides. 
 
+### Stream data using Event Hubs SDK (AMQP) 
+You can use any of the following samples to stream data to Event Hubs using SDKs. 
 - [.NET Core](event-hubs-dotnet-standard-getstarted-send.md)
 - [Java](event-hubs-java-get-started-send.md)
 - [Spring](/azure/developer/java/spring-framework/configure-spring-cloud-stream-binder-java-app-azure-event-hub?toc=/azure/event-hubs/TOC.json)
@@ -83,8 +92,12 @@ To get started using Event Hubs, see the **Send and receive events** tutorials:
 - [C](event-hubs-c-getstarted-send.md) (send only)
 - [Apache Storm](event-hubs-storm-getstarted-receive.md) (receive only)
 
+### Stream data using Apache Kafka
+You can use following samples to stream data from your Kafka applications to Event Hubs. 
+- [Using Event Hubs with Kafka applications](event-hubs-java-get-started-send.md)
+
+### Schema validation with Schema Registry 
+You can use Event Hubs Schema Registry to perform schema validation for your event streaming applications. 
 
-To learn more about Event Hubs, see the following articles:
+- [Schema validation for Kafka applications](schema-registry-kafka-java-send-receive-quickstart.md)
 
-- [Event Hubs features overview](event-hubs-features.md)
-- [Frequently asked questions](event-hubs-faq.yml).