Skip to content
This repository was archived by the owner on Aug 17, 2026. It is now read-only.

benchmarks.md

Latest commit

 

History

History
188 lines (127 loc) · 10.8 KB

benchmarks.md

File metadata and controls

188 lines (127 loc) · 10.8 KB
Raw
slug /cloud/managed-postgres/benchmarks
sidebar_label Benchmarks
title Performance benchmarks
description Performance benchmarks comparing Postgres managed by ClickHouse with AWS Aurora, RDS, and other managed PostgreSQL services
keywords
postgres benchmarks
performance
pgbench
aurora
rds
tps
latency
nvme performance
doc_type guide

import BetaBadge from '@theme/badges/BetaBadge'; import Image from '@theme/IdealImage'; import computeIntensive from '@site/static/images/managed-postgres/benchmarks/compute-intensive.png'; import ioReadOnly from '@site/static/images/managed-postgres/benchmarks/io-intensive-readonly.png'; import ioReadWrite from '@site/static/images/managed-postgres/benchmarks/io-intensive-readwrite.png';

:::info TL;DR

  • Benchmarked Postgres managed by ClickHouse against AWS RDS (16k provisioned IOPS) and Aurora IO Optimized using standard pgbench tests
  • Performance: ClickHouse's NVMe-backed Postgres delivers 4.3-9x faster performance for IO-intensive workloads and 12% faster for CPU-bound scenarios
  • Ideal for fast-growing AI-driven workloads that demand high transaction rates, low-latency data access, and predictable performance without IO bottlenecks :::

Benchmark overview {#overview}

We conducted comprehensive performance testing using pgbench, the standard PostgreSQL benchmarking tool, to evaluate workload performance under both moderate and high-concurrency scenarios.

Benchmarks {#benchmarks}

All performance tests were conducted using a client VM with the same compute capacity, collocated in the same region and availability zone as the PostgreSQL database to ensure fair comparison.

Test 1: IO Intensive - Read+Write (500 GB dataset) {#test1}

IO Intensive Read+Write benchmark results

Performance improvement over RDS (16k IOPS):

  • 326% higher TPS (4.3x faster)

Performance improvement over Aurora IO Optimized:

  • 345% higher TPS (4.5x faster)

Analysis: Mixed read/write workloads showcase the most dramatic performance advantages of NVMe storage and represent the most realistic scenario for fast-growing AI-driven workloads that require both high-throughput data ingestion and low-latency reads. Postgres managed by ClickHouse achieved 19.8K TPS with higher concurrency, demonstrating how NVMe storage scales effectively under load. This is 4.3-4.5x faster than RDS and Aurora. Network-attached storage solutions struggled with write-heavy operations, with RDS and Aurora maxing out at 4.4K-4.6K TPS despite provisioned capacity and even with Aurora's IO Optimized configuration.

Setup {#test1-setup}

This test evaluates mixed read/write performance with a large 500 GB dataset, stressing both read and write paths of the storage subsystem.

Instance configuration:

Configuration Postgres managed by ClickHouse RDS with 16k IOPS Aurora IO Optimized
PG Version 17 17 17
vCPUs 16 16 16
RAM 64 GB 64 GB 128 GB
Disk Size 1 TB 1 TB 1 TB
Disk Type NVMe (unlimited IOPS) Network-attached (16,000 IOPS) Network-attached (IO Optimized)

Test configuration:

# Initialize database (500 GB dataset)
pgbench -i -s 34247

# Read+Write benchmark
pgbench -c 256 -j 16 -T 600 -M prepared -P 30

Test 2: IO Intensive - Read-Only (500 GB dataset) {#test2}

IO Intensive Read-Only benchmark results

Performance improvement over RDS (16k IOPS):

  • 802% higher TPS (9.0x faster)

Analysis: The performance gap widens dramatically for read-intensive workloads that are IO Bound. Postgres managed by ClickHouse delivered 84.8K TPS, while RDS with 16,000 provisioned IOPS achieved only 9.4K TPS despite having equivalent compute resources. The key difference: ClickHouse's NVMe storage scales with higher concurrency, while network-attached storage remains constrained by provisioned IOPS limits. Even with provisioned IOPS, RDS was still 9x slower than ClickHouse, demonstrating the critical importance of storage architecture for IO-intensive workloads.

Setup {#test2-setup}

This test evaluates read performance with a large 500 GB dataset that doesn't fit in memory, stressing disk I/O capabilities.

Instance configuration:

Configuration Postgres managed by ClickHouse RDS with 16k IOPS
PG Version 17 17
vCPUs 16 16
RAM 64 GB 64 GB
Disk Size 1 TB 1 TB
Disk Type NVMe (unlimited IOPS) Network-attached (16,000 IOPS)

Test configuration:

# Initialize database (500 GB dataset)
pgbench -i -s 34247

# Read-only benchmark
pgbench -c 256 -j 16 -T 600 -M prepared -P 30 -S

Test 3: CPU Intensive (data fits in memory) {#test3}

CPU Intensive benchmark results

Performance improvement:

  • 12.3% higher TPS than RDS PostgreSQL

Analysis: Even in CPU-bound scenarios where disk I/O is minimal, Postgres managed by ClickHouse led the pack with 36.5K TPS. Despite both services hitting 100% CPU utilization, ClickHouse's NVMe storage delivered superior performance with better cache hit rates. The 12% advantage over RDS demonstrates the efficiency of the underlying infrastructure even when workloads are primarily CPU-bound.

Setup {#test3-setup}

This test evaluates CPU performance when the working set fits entirely in memory, minimizing disk I/O impact.

Instance configuration:

Configuration Postgres managed by ClickHouse RDS PostgreSQL
PG Version 17 17
vCPUs 2 2
RAM 8 GB 8 GB
Disk Type NVMe Network-attached (gp3)

Test configuration:

# Initialize database (2 GB dataset)
pgbench -i -s 136

# Warm-up run to load dataset into memory
pgbench -c 1 -T 60 -S -M prepared

# Run benchmark (read-only, prepared statements)
pgbench -c 32 -j 16 -T 300 -S -M prepared -P 30

Performance summary {#summary}

Key findings {#key-findings}

Across all three benchmark scenarios, Postgres managed by ClickHouse consistently delivered superior performance:

  1. IO-intensive read+write workloads: 4.3-4.5x higher TPS compared to RDS (16k IOPS) and Aurora IO Optimized
  2. IO-intensive read workloads: 9x higher TPS compared to RDS with 16k IOPS
  3. CPU-bound workloads: 12% higher TPS than RDS

When Postgres by ClickHouse excels {#when-it-excels}

Postgres by ClickHouse is ideal for applications that:

  • Power fast-growing AI-driven workloads requiring high-throughput data ingestion with frequent upserts, real-time feature updates, and analytics out of the box through seamless integration with ClickHouse for OLAP workloads
  • Perform frequent writes, updates, or mixed read/write operations
  • Need predictable, high-performance storage
  • Are currently constrained by IOPS limits on traditional managed Postgres services

If you expect analytics later and anticipate deeper ClickHouse integration—common in modern AI workloads where transactional data feeds real-time dashboards, feature stores, and ML pipelines—Postgres by ClickHouse should be your default go-to choice. The native integration eliminates complex ETL pipelines and enables seamless data flow between your operational database and analytical queries.

NVMe architecture advantage {#nvme-advantage}

The performance advantage comes from the fundamental architectural difference:

Aspect NVMe Storage (Managed Postgres) Network-Attached Storage (Provisioned IOPS)
IOPS 100k to virtually unlimited 16,000 provisioned
Network hops Zero (local device) Every disk operation requires network round trip
Performance scaling Scales linearly with concurrency Limited by provisioned IOPS

For more details on the performance benefits of NVMe storage, see NVMe-powered performance.

Cost-effectiveness {#cost-effectiveness}

Beyond raw performance, Postgres managed by ClickHouse offers superior price-performance:

  • Higher throughput per dollar: Achieve 4-9x more TPS compared to RDS with 16k provisioned IOPS and Aurora IO Optimized
  • Predictable costs: No need to provision additional IOPS capacity—unlimited local IOPS included
  • Lower compute requirements: Achieve target performance with smaller instance sizes due to efficient I/O
  • Reduced need for read replicas: Higher single-instance throughput reduces need for horizontal scaling

For workloads currently constrained by IOPS limits, switching to Managed Postgres can eliminate the need for expensive provisioned IOPS or IO-optimized configurations while delivering significantly better performance.

References {#references}

The complete benchmark data, configurations, and detailed metrics are available in our benchmark results spreadsheet.

Additional resources {#resources}