You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/kafka-azure/baseline.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,14 +27,14 @@ KRaft (Kafka Raft) mode replaces ZooKeeper by managing metadata directly within
27
27
Before you start Kafka in KRaft mode, you need to configure the broker and initialize the storage directory. You only need to do this once for each broker.
28
28
29
29
30
-
-**Edit the configuration file**
30
+
## Edit the configuration file
31
31
Open the Kafka configuration file in an editor:
32
32
33
33
```console
34
34
vi /opt/kafka/config/server.properties
35
35
```
36
36
37
-
- **Add or modify KRaft properties**
37
+
## Add or modify KRaft properties
38
38
Ensure the following configuration entries are present for a single-node KRaft setup:
39
39
40
40
```java
@@ -47,7 +47,7 @@ Before you start Kafka in KRaft mode, you need to configure the broker and initi
47
47
```
48
48
This configuration file sets up a single Kafka server to act as both a controller (managing cluster metadata) and a broker (handling data), running in KRaft mode. It defines the node's unique ID and specifies the local host as the sole participant in the controller quorum.
49
49
50
-
- **Format the storage directory**
50
+
## Format the storage directory
51
51
Format the metadata storage directory using the kafka-storage.sh tool. This initializes KRaft’s internal Raft logs with a unique cluster ID.
52
52
53
53
```console
@@ -60,7 +60,7 @@ Before you start Kafka in KRaft mode, you need to configure the broker and initi
60
60
```
61
61
This confirms that the Kafka storage directory has been successfully formatted and that the broker is ready to start in KRaft mode.
62
62
63
-
## Perform the Baseline Test
63
+
## Perform the baseline test
64
64
With Kafka 4.1.0 installed and configured in KRaft mode, you’re now ready to run a baseline test to verify that the Kafka broker starts correctly, topics can be created, and message flow works as expected.
Keep this terminal open and running. The broker process must stay active for all subsequent commands.
80
80
81
-
## Terminal 2 - create a Topic
81
+
## Terminal 2 - create a topic
82
82
Open a new terminal window. Create a topic named test-topic-kafka, which acts as a logical channel where producers send and consumers receive messages:
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/kafka-azure/benchmarking.md
+2-5Lines changed: 2 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -107,9 +107,6 @@ The producer sustained a throughput of ~257,500 records/sec (~24.5 MB/sec) with
107
107
The 95th percentile latency (1168 ms) and 99th percentile (1220 ms) show predictable network and I/O performance.
108
108
Kafka maintained consistent throughput, even under full-speed production, with no message loss or broker errors reported.
109
109
110
-
### Benchmark Comparison Insights
111
-
When analyzing performance on Azure Cobalt 100 Arm64 virtual machines:
112
-
**Producer efficiency**: The producer reached ~23–25 MB/sec throughput with average latencies below 900 ms, demonstrating stable delivery rates for high-volume workloads.
113
-
**Consumer scalability**: The consumer maintained ~262K messages/sec throughput with near-linear scaling of fetch performance — exceeding 1.85M messages/sec internally.
114
-
**Performance stability**: Both producer and consumer benchmarks showed low jitter and consistent latency distribution across iterations, confirming Kafka’s predictable behavior on Arm-based VMs.
110
+
### Benchmark comparison insights
111
+
When analyzing performance on Azure Cobalt 100 Arm64 virtual machines, you’ll notice that Kafka delivers stable and predictable results for both producers and consumers. The producer consistently achieves throughput between 23 MB/sec and 25 MB/sec, with average latencies below 900 ms. This means you can rely on efficient message delivery, even when handling high-volume workloads. On the consumer side, throughput remains strong at around 262,000 messages per second, and fetch performance scales nearly linearly, often exceeding 1.85 million messages per second internally. Throughout multiple benchmark runs, both producer and consumer tests demonstrate low jitter and consistent latency distribution, confirming that Kafka maintains reliable performance on Arm-based virtual machines.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/kafka-azure/create-instance.md
+14-14Lines changed: 14 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ These VMs are designed for a wide range of workloads and offer Arm-based perform
14
14
15
15
If you have never used the Microsoft Cloud Platform before, see the Microsoft guide on how to [Create a Linux virtual machine in the Azure portal](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal?tabs=ubuntu).
16
16
17
-
## Create an Arm-based Azure Virtual Machine
17
+
## Create an Arm-based Azure virtual machine
18
18
19
19
Creating a virtual machine based on Azure Cobalt 100 is no different from creating any other virtual machine in Azure. To create an Azure virtual machine, launch the Azure portal and navigate to **Virtual Machines**.
20
20
@@ -23,29 +23,29 @@ Creating a virtual machine based on Azure Cobalt 100 is no different from creati
23
23
- Choose the image for your virtual machine (for example, Ubuntu Pro 24.04 LTS) and select **Arm64** as the VM architecture.
24
24
- In the **Size** field, click on **See all sizes** and select the D-Series v6 family of virtual machines. Select **D4ps_v6** from the list.
25
25
26
-

26
+

27
27
28
-
5. Select "SSH public key" as an Authentication type. Azure will automatically generate an SSH key pair for you and allow you to store it for future use. It is a fast, simple, and secure way to connect to your virtual machine.
29
-
6. Fill in the Administrator username for your VM.
30
-
7. Select "Generate new key pair", and select "RSA SSH Format" as the SSH Key Type. RSA could offer better security with keys longer than 3072 bits. Give a Key pair name to your SSH key.
31
-
8. In the "Inbound port rules", select HTTP (80) and SSH (22) as the inbound ports.
28
+
- Select **SSH public key** as an authentication type. Azure automatically generates an SSH key pair for you and allows you to store it for future use. It is a fast, simple, and secure way to connect to your virtual machine.
29
+
- Fill in the administrator username for your VM.
30
+
- Select **Generate new key pair**, and select **RSA SSH Format** as the SSH key type. RSA can offer better security with keys longer than 3072 bits. Give a key pair name to your SSH key.
31
+
- In the **Inbound port rules**, select **HTTP (80)** and **SSH (22)** as the inbound ports.
32
32
33
-

33
+

34
34
35
-
9. Click on the "Review + Create" tab and review the configuration for your virtual machine. It should look like the following:
35
+
- Click on the **Review + Create** tab and review the configuration for your virtual machine. It should look like the following:
36
36
37
-

37
+

38
38
39
-
10. Finally, when you are confident about your selection, click on the "Create" button, and click on the "Download Private key and Create Resources" button.
39
+
- Finally, when you are confident about your selection, click on the "Create" button, and click on the "Download Private key and Create Resources" button.
40
40
41
-

41
+

42
42
43
-
11.Your virtual machine should be ready and running within no time. You can SSH into the virtual machine using the private key, along with the Public IP details.
43
+
Your virtual machine is now ready and running. To connect, use SSH with your private key and the VM's public IP address.
44
44
45
-

45
+

46
46
47
47
{{% notice Note %}}
48
48
49
-
To learn more about Arm-based virtual machine in Azure, refer to “Getting Started with Microsoft Azure” in [Get started with Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/azure).
49
+
To learn more about Arm-based virtual machine in Azure, refer to *Getting Started with Microsoft Azure* in [Get started with Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/azure).
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/kafka-azure/deploy.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ layout: learningpathall
10
10
11
11
This section guides you through installing the latest version of Apache Kafka on an Ubuntu Pro 24.04 (Arm64) virtual machine running on Azure Cobalt 100. Kafka is a high-throughput, distributed event streaming platform used for real-time data pipelines and messaging applications.
12
12
13
-
###Install Java
13
+
## Install Java
14
14
15
15
Apache Kafka runs on the Java Virtual Machine (JVM), so Java must be installed before setting up Kafka. Use the following commands to update your package index and install the default JDK:
16
16
```console
@@ -19,7 +19,7 @@ sudo apt install -y default-jdk
19
19
```
20
20
This installs the Java Development Kit (JDK), which includes the JVM, compiler, and standard libraries required for running Kafka services.
21
21
22
-
###Download and Install Kafka
22
+
## Download and install Kafka
23
23
24
24
Use the following commands to download and install Apache Kafka 4.1.0 in the /opt directory, extract the archive, and set appropriate permissions for your user. This prepares your system to run Kafka without requiring elevated privileges later.
The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends Apache Kafka version 3.5.0 as the minimum recommended on Arm platforms.
36
36
{{% /notice %}}
37
37
38
-
###Check installed Kafka version
38
+
## Check installed Kafka version
39
39
40
40
After extraction, verify that Kafka was installed successfully by checking the version:
0 commit comments