Skip to content

Commit dd61afd

Browse files
SamyOubouazizjcirinosclwyldecarvalho-docbene2k1
authored
docs(DWH): feature branch for DWH (#4766)
* docs(DWH): feature branch for DWH * docs(DWH): update * docs(srv): update * docs(dwh): update * docs(dwh): update * docs(dwh): update * docs(dwh): update * feat(dwh): add how to create * feat(dwh): update file titles * feat(dwh): add concepts MTA-5798 * feat(dwh): add connect applications MTA-5800 * feat(dwh): add connect applications MTA-5800 * feat(dwh): add connect BI tools page MTA-5801 * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * Apply suggestions from code review Co-authored-by: Jessica <[email protected]> * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): update * feat(dwh): hardcore menu update * feat(dwh): update menu * Apply suggestions from code review Co-authored-by: ldecarvalho-doc <[email protected]> Co-authored-by: Benedikt Rollik <[email protected]> * feat(dwh): update --------- Co-authored-by: Jessica <[email protected]> Co-authored-by: ldecarvalho-doc <[email protected]> Co-authored-by: Benedikt Rollik <[email protected]>
1 parent 1472438 commit dd61afd

16 files changed

+1153
-0
lines changed

‎menu/navigation.json‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2511,6 +2511,76 @@
25112511
{
25122512
"icon": "ManagedServicesCategoryIcon",
25132513
"items": [
2514+
{
2515+
"items": [
2516+
{
2517+
"label": "Overview",
2518+
"slug": "../data-warehouse"
2519+
},
2520+
{
2521+
"label": "Concepts",
2522+
"slug": "concepts"
2523+
},
2524+
{
2525+
"label": "Quickstart",
2526+
"slug": "quickstart"
2527+
},
2528+
{
2529+
"label": "FAQ",
2530+
"slug": "faq"
2531+
},
2532+
{
2533+
"items": [
2534+
{
2535+
"label": "Create a deployment",
2536+
"slug": "create-deployment"
2537+
},
2538+
{
2539+
"label": "Manage and delete a deployment",
2540+
"slug": "manage-delete-deployment"
2541+
},
2542+
{
2543+
"label": "Connect to a deployment",
2544+
"slug": "connect-applications"
2545+
},
2546+
{
2547+
"label": "Connect a deployment with BI tools",
2548+
"slug": "connect-bi-tools"
2549+
},
2550+
{
2551+
"label": "Import data into a deployment",
2552+
"slug": "import-data"
2553+
},
2554+
{
2555+
"label": "Edit the autoscaling of a deployment",
2556+
"slug": "edit-autoscaling"
2557+
},
2558+
{
2559+
"label": "Manage databases in a deployment",
2560+
"slug": "manage-databases"
2561+
},
2562+
{
2563+
"label": "Manage users in a deployment",
2564+
"slug": "manage-users"
2565+
}
2566+
],
2567+
"label": "How to",
2568+
"slug": "how-to"
2569+
},
2570+
{
2571+
"items": [
2572+
{
2573+
"label": "Example datasets",
2574+
"slug": "example-datasets"
2575+
}
2576+
],
2577+
"label": "Additional content",
2578+
"slug": "reference-content"
2579+
}
2580+
],
2581+
"label": "Data Warehouse for ClickHouse®",
2582+
"slug": "data-warehouse"
2583+
},
25142584
{
25152585
"items": [
25162586
{

‎pages/data-warehouse/concepts.mdx‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
meta:
3+
title: Data Warehouse for ClickHouse® - Concepts
4+
description: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
5+
content:
6+
h1: Data Warehouse for ClickHouse® - Concepts
7+
paragraph: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
8+
tags: data warehouse clickhouse concepts glossary terms definitions
9+
dates:
10+
published: 2025-05-07
11+
validation: 2025-06-03
12+
categories:
13+
- data-warehouse
14+
---
15+
16+
## Autoscaling
17+
18+
Autoscaling refers to the ability of a Data Warehouse for ClickHouse® deployment to automatically adjust the number of instances without manual intervention.
19+
Scaling mechanisms ensure that resources are provisioned dynamically to handle incoming requests efficiently while minimizing idle capacity and cost.
20+
21+
## Bottomless storage
22+
23+
Bottomless storage is a feature that allows ClickHouse® to separate compute and storage by offloading data to external object storage, such as Amazon S3-compatible services, while keeping frequently accessed data cached locally for fast queries. In ClickHouse®, this is implemented by transparently moving older or less-used data to remote storage, enabling virtually unlimited storage capacity without sacrificing performance for active workloads.
24+
25+
Refer to the official [ClickHouse® documentation](https://clickhouse.com/docs/guides/separation-storage-compute) for more information.
26+
27+
## ClickHouse®
28+
29+
ClickHouse® is a high-performance, column-oriented, distributed database management system designed for real-time analytics. It is optimized for handling large volumes of data with fast query performance, making it ideal for applications requiring up-to-date insights. ClickHouse® stores data in a columnar format, which reduces I/O operations and speeds up query execution. It supports distributed processing across multiple nodes, enabling horizontal scaling and fault tolerance through replication. ClickHouse® provides a powerful SQL interface and offers advanced features like real-time data ingestion, compression, and indexing, making it a robust solution for analytical workloads.
30+
31+
## ClickHouse® HTTP console
32+
33+
The ClickHouse® HTTP console is a web interface that allows you to interact easily with your Data Warehouse deployment from the Scaleway console by executing SQL queries.
34+
35+
<Message type="note">
36+
Make sure to enter valid credentials in the top-right fields before executing queries.
37+
</Message>
38+
39+
## Column-oriented storage
40+
41+
ClickHouse® stores data in a column-oriented format, which significantly optimizes read performance for analytical queries. By storing data in columns rather than rows, ClickHouse® reduces the number of I/O operations needed during query execution, as it only reads the necessary columns from disk.
42+
43+
## Compression
44+
45+
ClickHouse® uses advanced compression algorithms to reduce storage requirements and improve query performance by minimizing data transfer. Compression not only helps in saving disk space but also accelerates data retrieval and processing by reducing the amount of data that needs to be read from storage and transferred over the network.
46+
47+
## Distributed processing
48+
49+
ClickHouse® supports distributed processing across multiple nodes, allowing it to handle extremely large datasets efficiently and scale horizontally. This architecture enables ClickHouse® to distribute data and queries across a cluster, improving performance and reliability by leveraging the combined resources of all nodes.
50+
51+
## Horizontal scaling
52+
53+
Horizontal scaling refers to the process of adding more nodes to the cluster to increase its capacity and performance. This approach allows the cluster to handle larger datasets and higher query loads by distributing the data and processing tasks across additional nodes. Data Warehouse for ClickHouse® deployments [scale automatically](#autoscaling) according to the incoming workload.
54+
55+
## Indexing
56+
57+
ClickHouse® employs various indexing techniques, such as primary key and skip indexes, to speed up query execution and data retrieval. The primary key index allows for efficient point lookups and range queries, while skip indexes help in quickly skipping over large chunks of data that do not match query conditions, thus reducing the overall query time.
58+
59+
## Node
60+
61+
In the context of a distributed Data Warehouse for ClickHouse® cluster (also called "deployment"), a node refers to an individual instance that stores and processes a portion of the data. Each node participates in data distribution, query execution, and replication to ensure balanced load, fault tolerance, and high availability. Nodes communicate with each other to coordinate tasks, execute queries in parallel, and maintain synchronized data replicas. They are configured with specific settings to define their roles and manage resources, allowing the cluster to scale and perform efficiently.
62+
63+
## Replica set
64+
65+
A replica set consists of multiple nodes that store identical copies of the same data. This setup ensures fault tolerance and high availability by providing redundancy. If one node in the replica set fails, another node can take over, ensuring continuous data access and processing. ClickHouse® automatically handles data replication and failover, making it a reliable solution for mission-critical applications.
66+
67+
## SQL support
68+
69+
ClickHouse® provides a powerful SQL interface, enabling users to perform complex queries and data manipulations using familiar SQL syntax. This extensive SQL support includes a wide range of functions and features, such as subqueries, window functions, and user-defined functions, making it accessible to both analysts and developers.
70+
71+
72+
## Vertical scaling
73+
74+
Vertical scaling refers to the process of increasing the resources of individual nodes within the cluster. Vertical scaling enhances the performance and capacity of individual nodes, allowing them to handle larger datasets and more complex queries more efficiently. Vertical scaling is often used in conjunction with [horizontal scaling](#horizontal-scaling) to optimize performance and resource utilization in a Data Warehouse for ClickHouse® deployment.

‎pages/data-warehouse/faq.mdx‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
meta:
3+
title: Data Warehouse for ClickHouse® FAQ
4+
description: Discover Scaleway Data Warehouse for ClickHouse® and find answers to general questions.
5+
content:
6+
h1: Data Warehouse for ClickHouse®
7+
dates:
8+
validation: 2025-06-03
9+
category: managed-services
10+
productIcon: DataWarehouseProductIcon
11+
---
12+
13+
## What is Scaleway Data Warehouse for ClickHouse®?
14+
15+
Scaleway Data Warehouse for ClickHouse® allows you to perform queries and analytics on structured datasets, up to petabytes, without managing any infrastructure.
16+
17+
Data Warehouse for ClickHouse® supports SQL queries and Apache Spark™-compatible libraries, and can be integrated with data analysis and storage tools like Amazon S3-compatible storage, data visualization software, BI tools, or ETL solutions.
18+
19+
Scaleway handles seamless scaling, failover, settings and backups administration to let you focus on building your application.
20+
21+
Refer to the [Data Warehouse for ClickHouse® documentation](/data-warehouse/) for more information on the product and its features.

‎pages/data-warehouse/how-to/connect-applications.mdx‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
---
2+
meta:
3+
title: How to connect to your deployment
4+
description: Learn how to connect yourself or your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
5+
content:
6+
h1: How to Connect Applications to Your Deployment
7+
paragraph: Learn how to connect yourself or your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
8+
tags: connect applications deployment data warehouse clickhouse
9+
dates:
10+
validation: 2025-06-03
11+
posted: 2025-06-03
12+
categories:
13+
- data-warehouse
14+
---
15+
16+
This page explains how to connect yourself or your applications to your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com).
17+
18+
To connect your deployment with BI tools, refer to the [dedicated documentation](/data-warehouse/how-to/connect-bi-tools/).
19+
20+
<Macro id="requirements" />
21+
22+
- A Scaleway account logged into the [console](https://console.scaleway.com)
23+
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
24+
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
25+
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)
26+
27+
1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.
28+
29+
2. Click the name of the Data Warehouse deployment you want to connect to. The overview tab of the deployment displays.
30+
31+
3. Click the **Actions** button in the top-right corner of the page. A drop-down menu displays.
32+
33+
4. Select **Connect using frameworks**. The connection wizard displays.
34+
35+
<Message type="note">
36+
To connect your deployment with BI tools, refer to the [dedicated documentation](/data-warehouse/how-to/connect-bi-tools/).
37+
</Message>
38+
39+
5. Select your preferred framework:
40+
41+
**Protocols**
42+
43+
Select the appropriate protocol, then run the displayed command in a terminal. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.
44+
45+
<Tabs id="data-warehouse-connect-protocols">
46+
<TabsTab label="ClicHouse® CLI">
47+
```sh
48+
clickhouse client \
49+
--host <YOUR_DEPLOYMENT_ID>.dtwh.<REGION>.scw.cloud \
50+
--port 9440 \
51+
--secure \
52+
--user scwadmin \
53+
--password '<PASSWORD>'
54+
```
55+
</TabsTab>
56+
<TabsTab label="MySQL">
57+
```sh
58+
mysql -h <YOUR_DEPLOYMENT_ID>.dtwh.<REGION>.scw.cloud \
59+
-P 9004 \
60+
-u scwadmin \
61+
--password='<PASSWORD>' -e "SELECT 1;"
62+
```
63+
<Message type="note">
64+
MySQL connection is exposed publicly. Use ClickHouse® CLI for a secure connection.
65+
</Message>
66+
</TabsTab>
67+
<TabsTab label="HTTPS">
68+
```sh
69+
echo 'SELECT 1' | curl 'https://scwadmin:<PASSWORD>@<YOUR_DEPLOYMENT_ID>.dtwh.<REGION>.scw.cloud:443' -d @-
70+
```
71+
<Message type="note">
72+
`curl` only works with SQL queries, and does not allow direct connection to your Data Warehouse for ClickHouse® deployment.
73+
</Message>
74+
</TabsTab>
75+
</Tabs>
76+
<br/>
77+
**Languages**
78+
79+
Select the desired language, then run the code displayed to create a file that connects to your deployment, and run queries programmatically. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.
80+
81+
<Tabs id="data-warehouse-connect-languages">
82+
<TabsTab label="Python">
83+
```python
84+
pip install clickhouse-connect
85+
cat <<EOF >clickhouse.py
86+
import clickhouse_connect
87+
88+
client = clickhouse_connect.get_client(
89+
host="<YOUR_DEPLOYMENT_ID>.dtwh.<REGION>.scw.cloud",
90+
port=443,
91+
username="scwadmin",
92+
password="<PASSWORD>",
93+
)
94+
query_result = client.query("SELECT 1")
95+
print(query_result.result_set)
96+
EOF
97+
python clickhouse.py
98+
```
99+
</TabsTab>
100+
<TabsTab label="Go">
101+
```go
102+
mkdir ClickHouse-go
103+
cd ClickHouse-go
104+
go mod init ClickHouse-go
105+
cat <<EOF >main.go
106+
package main
107+
108+
import (
109+
"context"
110+
"fmt"
111+
"log"
112+
"github.com/ClickHouse/clickhouse-go/v2"
113+
)
114+
115+
func main() {
116+
conn, err := clickhouse.Open(&clickhouse.Options{
117+
Addr: []string{"f133556f-8578-486f-be7f-49f7da08b728.dtwh.fr-par.scw.cloud:9440"},
118+
Auth: clickhouse.Auth{
119+
Database: "default",
120+
Username: "scwadmin",
121+
Password: "PASSWORD",
122+
},
123+
})
124+
if err != nil {
125+
log.Fatal(err)
126+
}
127+
defer conn.Close()
128+
129+
ctx := context.Background()
130+
var result string
131+
if err := conn.QueryRow(ctx, "SELECT 1").Scan(&result); err != nil {
132+
log.Fatal(err)
133+
}
134+
fmt.Println(result)
135+
}
136+
EOF
137+
go run .
138+
```
139+
</TabsTab>
140+
<TabsTab label="Node.js">
141+
```node
142+
npm i @clickhouse/client
143+
cat <<EOF >clickhouse.js
144+
import { createClient } from '@clickhouse/client'
145+
146+
void (async () => {
147+
const client = createClient({
148+
url: 'https://f133556f-8578-486f-be7f-49f7da08b728.dtwh.fr-par.scw.cloud:443',
149+
username: 'scwadmin',
150+
password: 'PASSWORD'
151+
})
152+
const rows = await client.query({
153+
query: 'SELECT 1',
154+
format: 'JSONEachRow',
155+
})
156+
console.info(await rows.json())
157+
await client.close()
158+
})()
159+
EOF
160+
node clickhouse.js
161+
```
162+
</TabsTab>
163+
<TabsTab label="Java">
164+
```java
165+
cat <<EOF >pom.xml
166+
<?xml version="1.0" encoding="UTF-8"?>
167+
<project
168+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
169+
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
170+
<modelVersion>4.0.0</modelVersion>
171+
<groupId>clickhouse-java</groupId>
172+
<artifactId>clickhouse-java</artifactId>
173+
<version>1.0.0</version>
174+
<packaging>jar</packaging>
175+
<dependencies>
176+
<dependency>
177+
<groupId>com.clickhouse</groupId>
178+
<artifactId>clickhouse-jdbc</artifactId>
179+
<version>0.7.2</version>
180+
</dependency>
181+
</dependencies>
182+
</project>
183+
EOF
184+
mkdir -p src/main/java
185+
cat <<EOF >src/main/java/Main.java
186+
import java.sql.*;
187+
import java.lang.ClassNotFoundException;
188+
189+
public class Main {
190+
public static void main(String[] args) throws SQLException, ClassNotFoundException {
191+
Class.forName("com.clickhouse.jdbc.ClickHouseDriver");
192+
String url = "jdbc:ch://scwadmin:PASSWORD@f133556f-8578-486f-be7f-49f7da08b728.dtwh.fr-par.scw.cloud:443/default?ssl=true";
193+
try (Connection con = DriverManager.getConnection(url); Statement stmt = con.createStatement()) {
194+
ResultSet result_set = stmt.executeQuery("SELECT 1 AS one");
195+
while (result_set.next()) {
196+
System.out.println(result_set.getInt("one"));
197+
}
198+
}
199+
}
200+
}
201+
EOF
202+
mvn clean compile
203+
mvn exec:java -Dexec.mainClass=Main
204+
```
205+
</TabsTab>
206+
</Tabs>
207+
208+
You are now connected to your Data Warehouse for ClickHouse® deployment using the administrator account.

0 commit comments

Comments
 (0)