Skip to content

Commit 65ac26c

Browse files
committed
Add initial documentation for Shared Catalog
1 parent a743a5c commit 65ac26c

File tree

1 file changed

+85
-0
lines changed

1 file changed

+85
-0
lines changed
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
slug: /cloud/reference/shared-catalog
3+
sidebar_label: 'Shared Catalog'
4+
title: 'Shared Catalog and Shared Database Engine'
5+
keywords: ['SharedCatalog', 'SharedDatabaseEngine']
6+
description: 'Describes the Shared Catalog component and the Shared database engine in ClickHouse Cloud'
7+
---
8+
9+
# Shared Catalog and Shared Database Engine
10+
11+
**Available exclusively in ClickHouse Cloud (and first party partner cloud services)**
12+
13+
Shared Catalog is a cloud-native component responsible for replicating metadata and DDL operations of databases and tables that use stateless engines across replicas in ClickHouse Cloud. It enables consistent and centralized state management for these objects, ensuring metadata consistency even in dynamic or partially offline environments.
14+
15+
Shared Catalog does **not replicate tables themselves**, but ensures that all replicas have a consistent view of the database and table definitions by replicating DDL queries and metadata.
16+
17+
It supports replication of the following database engines:
18+
19+
- Shared
20+
- PostgreSQL
21+
- MySQL
22+
- DataLakeCatalog
23+
24+
## Architecture and Metadata Storage
25+
26+
All metadata and DDL query history in Shared Catalog is stored centrally in ZooKeeper. Nothing is persisted on local disk. This architecture ensures:
27+
28+
- Consistent state across all replicas
29+
- Statelessness of compute nodes
30+
- Fast, reliable replica bootstrapping
31+
32+
## Shared Database Engine
33+
34+
The **Shared database engine** works in conjunction with Shared Catalog to manage databases whose tables use **stateless table engines** such as `SharedMergeTree`. These table engines do not write persistent state to disk and are compatible with dynamic compute environments.
35+
36+
Shared database engine builds on and improves the behavior of the Replicated database engine while offering additional guarantees and operational benefits.
37+
38+
### Key Benefits
39+
40+
- **Atomic CREATE TABLE ... AS SELECT**
41+
Table creation and data insertion are executed atomically—either the entire operation completes, or the table is not created at all.
42+
43+
- **RENAME TABLE between databases**
44+
Enables atomic movement of tables across databases:
45+
```sql
46+
RENAME TABLE db1.table TO db2.table;
47+
```
48+
49+
- **Automatic table recovery with UNDROP TABLE**
50+
Dropped tables are retained for a default period of 8 hours and can be restored:
51+
```sql
52+
UNDROP TABLE my_table;
53+
```
54+
The retention window is configurable via server settings.
55+
56+
- **Improved compute-compute separation**
57+
Unlike the Replicated database engine, which requires all replicas to be online to process a DROP query, Shared Catalog performs centralized metadata deletion. This allows operations to succeed even when some replicas are offline.
58+
59+
- **Automatic metadata replication**
60+
Shared Catalog ensures that database definitions are automatically replicated to all servers on startup. Operators do not need to manually configure or synchronize metadata on new instances.
61+
62+
- **Centralized, versioned metadata state**
63+
Shared Catalog stores a single source of truth in ZooKeeper. When a replica starts, it fetches the latest state and applies the diff to reach consistency. During query execution, the system can wait for other replicas to reach at least the required version of metadata to ensure correctness.
64+
65+
## Usage in ClickHouse Cloud
66+
67+
For end users, using Shared Catalog and the Shared database engine requires no additional configuration. Database creation is the same as always:
68+
69+
```sql
70+
CREATE DATABASE my_database;
71+
```
72+
73+
ClickHouse Cloud automatically assigns the Shared database engine to databases. Any tables created within such a database using stateless engines will automatically benefit from Shared Catalog’s replication and coordination capabilities.
74+
75+
## Summary
76+
77+
Shared Catalog and the Shared database engine provide:
78+
79+
- Reliable and automatic metadata replication for stateless engines
80+
- Stateless compute with no local metadata persistence
81+
- Atomic operations for complex DDL
82+
- Improved support for elastic, ephemeral, or partially offline compute environments
83+
- Seamless usage for ClickHouse Cloud users
84+
85+
These capabilities make Shared Catalog the foundation for scalable, cloud-native metadata management in ClickHouse Cloud.

0 commit comments

Comments
 (0)