Skip to content

Commit 78639af

Browse files
rohit-ngvjdhama
authored andcommitted
fix: md lint issues
1 parent a6188fa commit 78639af

File tree

1 file changed

+18
-12
lines changed

1 file changed

+18
-12
lines changed

content/blog/clickhouse-benchmarking.md

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,11 @@
1-
# ClickHouse Deployment and Performance Benchmarking on ECS
1+
---
2+
title: "ClickHouse Deployment and Performance Benchmarking on ECS"
3+
authorId: "rohit"
4+
date: 2024-10-21
5+
draft: false
6+
featured: true
7+
weight: 1
8+
---
29

310
"Imagine being a Formula One driver, racing at breakneck speeds, but without any telemetry data to guide you. It’s a thrilling ride, but one wrong turn or overheating engine could lead to disaster. Just like a pit crew relies on performance metrics to optimize the car's speed and handling, we utilize observability in ClickHouse to monitor our data system's health. These metrics provide crucial insights, allowing us to identify bottlenecks, prevent outages, and fine-tune performance, ensuring our data engine runs as smoothly and efficiently as a championship-winning race car."
411

@@ -12,17 +19,17 @@ In this blog, we'll dive into the process of deploying ClickHouse on AWS Elastic
1219

1320
In this architecture, we utilize five servers to ensure data availability and reliability. Two of these servers are dedicated to hosting copies of the data, while the remaining three serve to coordinate the replication process. We will create a database and a table using the **ReplicatedMergeTree** engine, which allows for seamless data replication across the two data nodes.
1421

15-
#### Key Terms:
22+
### Key Terms
1623

1724
- **Replica:** In ClickHouse, a replica refers to a copy of your data. There is always at least one copy (the original), and adding a second replica enhances fault tolerance. This ensures that if one copy fails, the other remains accessible.
1825

1926
- **Shard:** A shard is a subset of your data. If you do not split the data across multiple servers, all data resides in a single shard. Sharding helps distribute the load when a single server's capacity is exceeded. The destination server for the data is determined by a sharding key, which can be random or derived from a hash function. In our examples, we will use a random key for simplicity.
2027

2128
This architecture not only protects your data but also allows for better handling of increased loads, making it a robust solution for data management in ClickHouse. For more detailed information, refer to the official documentation on [ClickHouse Replication Architecture](https://clickhouse.com/docs/en/architecture/replication).
2229

23-
### Configuration Changes for ClickHouse Deployment
30+
## Configuration Changes for ClickHouse Deployment
2431

25-
**Node Descriptions**
32+
### Node Descriptions
2633

2734
- **clickhouse-01**: Data node for storing data.
2835
- **clickhouse-02**: Another data node for data storage.
@@ -32,7 +39,7 @@ This architecture not only protects your data but also allows for better handlin
3239

3340
### Installation Steps
3441

35-
- **ClickHouse Server**: We deployed ClickHouse Server and Client on the data nodes, clickhouse-01 and clickhouse-02, using Docker images, specifically `clickhouse/clickhouse-server` for installation.
42+
- **ClickHouse Server**: We deployed ClickHouse Server and Client on the data nodes, clickhouse-01 and clickhouse-02, using Docker images, specifically `clickhouse/clickhouse-server` for installation.
3643

3744
- **ClickHouse Keeper**: Installed on the three servers (clickhouse-keeper-01, clickhouse-keeper-02, and clickhouse-keeper-03) using Docker image `clickhouse/clickhouse-keeper`.
3845

@@ -48,7 +55,7 @@ This architecture not only protects your data but also allows for better handlin
4855

4956
The configuration for clickhouse-01 includes five files for clarity, although they can be combined if desired. Here are key elements:
5057

51-
- **Network and Logging Configuration**:
58+
- **Network and Logging Configuration**:
5259
- Sets the display name to "cluster_1S_2R node 1."
5360
- Configures ports for HTTP (8123) and TCP (9000).
5461

@@ -68,7 +75,7 @@ The configuration for clickhouse-01 includes five files for clarity, although th
6875
</clickhouse>
6976
```
7077

71-
- **Macros Configuration**:
78+
- **Macros Configuration**:
7279
- Simplifies DDL by using macros for shard and replica numbers.
7380

7481
```xml
@@ -131,17 +138,17 @@ The configuration for clickhouse-01 includes five files for clarity, although th
131138

132139
The configuration is mostly similar to clickhouse-01, with key differences noted:
133140

134-
- **Network and Logging Configuration**:
141+
- **Network and Logging Configuration**:
135142
- Similar to clickhouse-01 but with a different display name.
136143

137-
- **Macros Configuration**:
144+
- **Macros Configuration**:
138145
- The replica is set to 02 on this node.
139146

140147
#### clickhouse-keeper Configuration
141148

142149
For ClickHouse Keeper, each node configuration includes:
143150

144-
- **General Configuration**:
151+
- **General Configuration**:
145152
- Ensure the server ID is unique across all instances.
146153

147154
```xml
@@ -173,7 +180,7 @@ All configuration changes were integrated into the Docker image using the base C
173180

174181
## ECS Cluster Setup and ClickHouse Deployment
175182

176-
#### ClickHouse Deployment Overview
183+
### ClickHouse Deployment Overview
177184

178185
- **AWS Partner Solution:** We leveraged the AWS Partner Solution Deployment Guide for ClickHouse to ensure a structured setup.
179186

@@ -205,7 +212,6 @@ All configuration changes were integrated into the Docker image using the base C
205212

206213
- **Auto Scaling Group:** An Auto Scaling Group was set up with `m5.large` instances, providing **3 GB of memory** for each container to ensure optimal performance under varying loads.
207214

208-
209215
## Performance Benchmarking Metrics
210216

211217
Now that the deployment architecture is established, let's move on to evaluating ClickHouse's performance through a series of benchmarking metrics.

0 commit comments

Comments
 (0)