Skip to content
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8131d17
docs(DWH): feature branch for DWH
SamyOubouaziz Apr 2, 2025
83cd01f
docs(DWH): update
SamyOubouaziz Apr 10, 2025
14ce174
docs(srv): update
SamyOubouaziz May 7, 2025
9fd0a73
docs(dwh): update
SamyOubouaziz May 7, 2025
ec98dad
docs(dwh): update
SamyOubouaziz May 7, 2025
e2fe40a
docs(dwh): update
SamyOubouaziz May 7, 2025
5a4cf49
docs(dwh): update
SamyOubouaziz May 7, 2025
6615dd8
feat(dwh): add how to create
SamyOubouaziz May 13, 2025
b0f318b
feat(dwh): update file titles
SamyOubouaziz May 13, 2025
cf2cf25
feat(dwh): add concepts MTA-5798
SamyOubouaziz May 13, 2025
7fc335f
feat(dwh): add connect applications MTA-5800
SamyOubouaziz May 13, 2025
38ebab6
feat(dwh): add connect applications MTA-5800
SamyOubouaziz May 13, 2025
c912c03
feat(dwh): add connect BI tools page MTA-5801
SamyOubouaziz May 19, 2025
17cce23
feat(dwh): update
SamyOubouaziz May 19, 2025
b43ee82
feat(dwh): update
SamyOubouaziz May 21, 2025
012c8a1
feat(dwh): update
SamyOubouaziz May 22, 2025
2c7295a
feat(dwh): update
SamyOubouaziz May 22, 2025
f71b76f
feat(dwh): update
SamyOubouaziz May 23, 2025
a96e339
feat(dwh): update
SamyOubouaziz May 23, 2025
4d7db47
feat(dwh): update
SamyOubouaziz May 26, 2025
5b2a041
feat(dwh): update
SamyOubouaziz May 26, 2025
53f88c8
feat(dwh): update
SamyOubouaziz May 28, 2025
9494631
feat(dwh): update
SamyOubouaziz May 28, 2025
432d712
Apply suggestions from code review
SamyOubouaziz Jun 2, 2025
c261553
feat(dwh): update
SamyOubouaziz Jun 2, 2025
27a5b4f
feat(dwh): update
SamyOubouaziz Jun 2, 2025
43f975d
feat(dwh): update
SamyOubouaziz Jun 2, 2025
37da511
feat(dwh): update
SamyOubouaziz Jun 2, 2025
8eb83a2
feat(dwh): update
SamyOubouaziz Jun 2, 2025
8054f3d
feat(dwh): update
SamyOubouaziz Jun 2, 2025
c87698a
feat(dwh): update
SamyOubouaziz Jun 2, 2025
0abc1f4
feat(dwh): update
SamyOubouaziz Jun 3, 2025
0affa91
feat(dwh): update
SamyOubouaziz Jun 3, 2025
534154c
feat(dwh): update
SamyOubouaziz Jun 3, 2025
5c6b0ea
feat(dwh): update
SamyOubouaziz Jun 3, 2025
1255c34
feat(dwh): hardcore menu update
SamyOubouaziz Jun 3, 2025
6694627
feat(dwh): update menu
SamyOubouaziz Jun 3, 2025
7c0d9e9
Merge branch 'main' into int-feat-dwh-feature-branch
SamyOubouaziz Jun 3, 2025
9744fae
Apply suggestions from code review
SamyOubouaziz Jun 3, 2025
6b28dda
feat(dwh): update
SamyOubouaziz Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions menu/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -3013,6 +3013,60 @@
],
"label": "Distributed Data Lab",
"slug": "data-lab"
},
{
"items": [
{
"label": "Overview",
"slug": "../data-warehouse"
},
{
"label": "Concepts",
"slug": "concepts"
},
{
"label": "Quickstart",
"slug": "quickstart"
},
{
"label": "FAQ",
"slug": "faq"
},
{
"items": [
{
"label": "Create a Data Warehouse deployment",
"slug": "create-deployment"
},
{
"label": "Connect applications to a deployment",
"slug": "connect-applications"
},
{
"label": "Connect your deployment to BI tools",
"slug": "connect-bi-tools"
},
{
"label": "Manage and delete a Data Warehouse deployment",
"slug": "manage-delete-deployment"
}
],
"label": "How to",
"slug": "how-to"
},
{
"items": [
{
"label": "Example datasets",
"slug": "example-datasets"
}
],
"label": "Additional content",
"slug": "reference-content"
}
],
"label": "Data Warehouse",
"slug": "data-warehouse"
}
],
"label": "Managed Services",
Expand Down
74 changes: 74 additions & 0 deletions pages/data-warehouse/concepts.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
meta:
title: Data Warehouse for ClickHouse® - Concepts
description: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
content:
h1: Data Warehouse for ClickHouse® - Concepts
paragraph: Understand key concepts behind Scaleway's Data Warehouse for ClickHouse®.
tags: data warehouse clickhouse concepts glossary terms definitions
dates:
published: 2025-05-07
validation: 2025-05-07
categories:
- data-warehouse
---

## Autoscaling

Autoscaling refers to the ability of a Data Warehouse for ClickHouse® to automatically adjust the number of instances without manual intervention.
Scaling mechanisms ensure that resources are provisioned dynamically to handle incoming requests efficiently while minimizing idle capacity and cost.

## Bottomless storage

Bottomless storage is a feature that allows ClickHouse® to separate compute and storage by offloading data to external object storage, such as Amazon S3-compatible services, while keeping frequently accessed data cached locally for fast queries. In ClickHouse®, this is implemented by transparently moving older or less-used data to remote storage, enabling virtually unlimited storage capacity without sacrificing performance for active workloads.

Refer to the official [ClickHouse® documentation](https://clickhouse.com/docs/guides/separation-storage-compute) for more information.

## ClickHouse®

ClickHouse® is a high-performance, column-oriented, distributed database management system designed for real-time analytics. It is optimized for handling large volumes of data with fast query performance, making it ideal for applications requiring up-to-date insights. ClickHouse® stores data in a columnar format, which reduces I/O operations and speeds up query execution. It supports distributed processing across multiple nodes, enabling horizontal scaling and fault tolerance through replication. ClickHouse® provides a powerful SQL interface and offers advanced features like real-time data ingestion, compression, and indexing, making it a robust solution for analytical workloads.

## ClickHouse® HTTP console

The ClickHouse® HTTP console is a web interface that allows you to interact easily interact with your Data Warehouse deployment from the Scaleway console by executing SQL queries.

<Message type="note">
Make sure to enter valid credentials in the top-right fields before executing queries.
</Message>

## Column-oriented storage

ClickHouse® stores data in a column-oriented format, which significantly optimizes read performance for analytical queries. By storing data in columns rather than rows, ClickHouse® reduces the amount of I/O operations needed during query execution, as it only reads the necessary columns from disk.

## Compression

ClickHouse® uses advanced compression algorithms to reduce storage requirements and improve query performance by minimizing data transfer. Compression not only helps in saving disk space but also accelerates data retrieval and processing by reducing the amount of data that needs to be read from storage and transferred over the network.

## Distributed processing

ClickHouse® supports distributed processing across multiple nodes, allowing it to handle extremely large datasets efficiently and scale horizontally. This architecture enables ClickHouse® to distribute data and queries across a cluster, improving performance and reliability by leveraging the combined resources of all nodes.

## Horizontal scaling

Horizontal scaling refers to the process of adding more nodes to the cluster to increase its capacity and performance. This approach allows the cluster to handle larger datasets and higher query loads by distributing the data and processing tasks across additional nodes. Data Warehouse for ClickHouse® deployments [scale automatically](#autoscaling) according to the incoming workload.

## Indexing

ClickHouse® employs various indexing techniques, such as primary key and skip indexes, to speed up query execution and data retrieval. The primary key index allows for efficient point lookups and range queries, while skip indexes help in quickly skipping over large chunks of data that do not match query conditions, thus reducing the overall query time.

## Node

In the context of a distributed Data Warehouse for ClickHouse® cluster, a node refers to an individual instance that stores and processes a portion of the data. Each node participates in data distribution, query execution, and replication to ensure balanced load, fault tolerance, and high availability. Nodes communicate with each other to coordinate tasks, execute queries in parallel, and maintain synchronized data replicas. They are configured with specific settings to define their roles and manage resources, allowing the cluster to scale and perform efficiently.

## Replica set

A replica set consists of multiple nodes that store identical copies of the same data. This setup ensures fault tolerance and high availability by providing redundancy. If one node in the replica set fails, another node can take over, ensuring continuous data access and processing. ClickHouse® automatically handles data replication and failover, making it a reliable solution for mission-critical applications.

## SQL support

ClickHouse® provides a powerful SQL interface, enabling users to perform complex queries and data manipulations using familiar SQL syntax. This extensive SQL support includes a wide range of functions and features, such as subqueries, window functions, and user-defined functions, making it accessible to both analysts and developers.


## Vertical scaling

Vertical scaling refers to the process of increasing the resources of individual nodes within the cluster. Vertical scaling enhances the performance and capacity of individual nodes, allowing them to handle larger datasets and more complex queries more efficiently. Vertical scaling is often used in conjunction with [horizontal scaling](#horizontal-scaling) to optimize performance and resource utilization in a Data Warehouse for ClickHouse® deployment.
15 changes: 15 additions & 0 deletions pages/data-warehouse/faq.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
meta:
title: Data Warehouse for ClickHouse® FAQ
description: Discover Scaleway Data Warehouse for ClickHouse® and find answers to general questions.
content:
h1: Data Warehouse for ClickHouse®
dates:
validation: 2025-05-07
category: managed-services
productIcon: DataWarehouseProductIcon
---

## What is Scaleway Data Warehouse for ClickHouse®?

Its like a warehouse but for data, and its for ClickHouse® also.
45 changes: 45 additions & 0 deletions pages/data-warehouse/how-to/connect-applications.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
meta:
title: How to connect to your deployment
description: Learn how to connect yourself or your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
content:
h1: How to Connect Applications to Your Deployment
paragraph: Learn how to connect yourself or your applications to your Scaleway Data Warehouse for ClickHouse® deployment.
tags: connect applications deployment data warehouse clickhouse
dates:
validation: 2025-05-07
posted: 2025-05-07
categories:
- data-warehouse
---

This page explains how to connect yourself or your applications to your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com).

To connect your deployment with BI tools, refer to the [dedicated documentation](/data-warehouse/how-to/connect-bi-tools/).

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

1. Click **Data Warehouse** under **Managed Services** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the Data Warehouse deployment you want to connect to. The overview tab of the deployment displays.

3. Click the **Actions** button in the top-right corner of the page. A drop-down menu displays.

4. Select ¨**Connect using frameworks**. The connection wizard displays.

<Message type="note">
To connect your deployment with BI tools, refer to the [dedicated documentation](/data-warehouse/how-to/connect-bi-tools/).
</Message>

5. Click the **Download** link to download the required TLS certificate.

6. Select your preferred framework:
- Protocols: Select the appropriate protocol, then run the displayed command in a terminal. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.
- Languages: Select the desired language, then add the code displayed to your application code. Remember to replace the placeholders with the appropriate values, and to specify the correct path to the certificate file.

You are now connected to your Data Warehouse for ClickHouse® deployment using the administrator account.
115 changes: 115 additions & 0 deletions pages/data-warehouse/how-to/connect-bi-tools.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
meta:
title: How to Connect Your Deployment to BI Tools
description: Learn how to connect your Scaleway Data Warehouse for ClickHouse® deployment to Business Intelligence (BI) tools.
content:
h1: How to Connect Your Deployment to BI Tools
paragraph: Learn how to connect your Scaleway Data Warehouse for ClickHouse® deployment to Business Intelligence (BI) tools.
tags: connect bi-tools deployment data warehouse ClickHouse® integrate integration
dates:
validation: 2025-05-19
posted: 2025-05-19
categories:
- data-warehouse
---

This page explains how to integrate your Data Warehouse for ClickHouse® deployment with your preferred Business Intelligence (BI) tools.

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## Tableau

Tableau is a powerful data visualization and business intelligence tool that enables users to transform complex data into interactive and shareable dashboards and reports, providing actionable insights through intuitive drag-and-drop interfaces.

### Download and install the ClickHouse® connector for Tableau

5. Download and install [Tableau Desktop](https://www.tableau.com/products/desktop/download).

6. Follow the instructions for `clickhouse-tableau-connector-jdbc` to [download the compatible version](https://github.com/ClickHouse/clickhouse-java/releases/) of the ClickHouse® JDBC driver.

4. Store the JDBC driver in the `Drivers` folder. If the directory does not exist, create it:
- macOS/Linux: `~/Library/Tableau/Drivers`
- Windows: `C:\Program Files\Tableau\Drivers`

### Configure a ClickHouse® data source in Tableau

1. Start or restart Tableau.

2. From the **Connect** left-side menu, click on **More** under the **To a Server** section.

3. Select **ClickHouse® by ClickHouse®** from the connectors list.

4. Enter the following connection parameters:

6. Enter your connection details:

| Setting | Value |
| -------- |--------------------------------------------------------|
| Server | Your ClickHouse® host (with no prefixes or suffix) |
| Port | 8443 |
| Database | default |
| Username | default |
| Password | Password set at deployment creation |

<Message type="note">
Make sure to enable the SSL if you are using ClickHouse® Cloud.
</Message>

7. Click **Sign In**. A new Tableau workbook appears.

8. Select **default** from the **Schema** dropdown menu in the left side panel. A list of tables appear.

9. Your Data Warehouse for ClickHouse® is now integrated into your Tableau platform.

Refer to the official [ClickHouse®](https://clickhouse.com/docs/integrations/tableau) and [Tableau](https://exchange.tableau.com/products/1064) documentation portals for more information.

## Metabase

Metabase is an open-source business intelligence tool that allows users to create and share customizable dashboards and reports from various data sources.

Integrating ClickHouse® into Metabase enhances query performance and scalability, for faster data analysis on large datasets, and provides robust support for complex queries and real-time analytics.

### Download the ClickHouse® plugin for Metabase

1. Download the latest version of the plugin (JAR file named `clickhouse.metabase-driver.jar`) from the (official CLickHouse® GitHub repository)[https://github.com/clickhouse/metabase-clickhouse-driver/releases/latest].

2. Save `clickhouse.metabase-driver.jar` in your Metabase plugins folder.
<Message type="note">
If you do not have a If you do not have a **plugins** folder, create it in the same directory as the `metabase.jar` file. Refer to the [official Metabase documentation](https://www.metabase.com/docs/latest/developers-guide/drivers/basics) for more information on plugins.
</Message>

3. Start (or restart) Metabase to load the new plugin.

4. Access your Metabase server.
<Message type="note">
On the initial startup, if prompted to select a database, select **I'll add my data later**.
</Message>

### Connect Metabase to ClickHouse®

1. Click on the gear icon in the top-right corner and select **Admin Settings** to visit your Metabase admin page.

2. Click **Add a database**. Alternately, you can click the **Add database** button from the **Databases** tab.

3. If your driver installation worked, you will see ClickHouse® in the dropdown menu for Database type.

4. Enter a display name for your database.

5. Enter the credentials Data Warehouse for ClickHouse® deployment. Toggle on **Use a secure connection (SSL)**if your ClickHouse® server is configured to use SSL.

6. Click **Connect database** to finish.

Your Data Warehouse for ClickHouse® is now integrated into your Metabase platform.

Refer to the official [ClickHouse®](https://clickhouse.com/docs/integrations/metabase) and [Metabase](https://www.metabase.com/docs/latest/databases/connections/clickhouse) documentation portals for more information.

## PowerBI

Power BI Desktop is a robust data visualization tool that allows users to create dynamic reports and dashboards from various data sources. Before connecting your Data Warehouse for ClickHouse® deployment, Make sure you have downloaded and installed [Power BI Desktop](https://www.microsoft.com/en-us/download/details.aspx?id=58494).

Refer to the official [ClickHouse®](https://clickhouse.com/docs/integrations/powerbi) and [PowerBI](https://learn.microsoft.com/en-us/power-query/connectors/clickhouse) documentation portals for more information.
54 changes: 54 additions & 0 deletions pages/data-warehouse/how-to/create-deployment.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
meta:
title: How to create a Data Warehouse for ClickHouse® deployment
description: This page provides a guide on how to set up a Data Warehouse using ClickHouse® using the Scaleway console.
content:
h1: How to create a Data Warehouse for ClickHouse® deployment
paragraph: This page provides a guide on how to set up a Data Warehouse using ClickHouse® using the Scaleway console.
tags: clickhouse data-warehouse deployment
dates:
validation: 2025-05-13
posted: 2025-05-07
categories:
- data-warehouse
---

A Data Warehouse is a centralized repository designed to store, manage, and analyze large volumes of structured and semi-structured data. When powered by ClickHouse®, it provides performance for real-time analytics and complex queries. This guide will walk you through the steps to deploy a Data Warehouse for ClickHouse® deployment using the Scaleway console.

<Macro id="requirements" />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

## Creating a Data Warehouse Deployment

1. Click **ClickHouse®** under **Managed Services** on the side menu. The Managed Data Warehouse for ClickHouse® page displays.

3. Click **+ Create deployment**. The Data Warehouse deployment creation wizard displays.

4. Select the **region** in which to create your deployment.

5. Choose an autoscaling configuration:
- **Presets**: select one of several pre-configured deployments adapted to the most common use cases.
- **Custom**: manually enter the vCPU autoscaling minimum and maximum values, and the number of nodes for your deployment.
<Message type="note">
You can update the autoscaling configuration of your deployment at any time after its creation.
</Message>

6. Enter a name for your deployment, or keep the automatically generated one.

7. Configure the administrator credentials for your deployment by entering a password, or by generating a random one.
<Message type="note">
The username cannot be modified.
</Message>

8. Click the **+ Advanced settings** button, then configure the following elements:

- Select a ClickHouse® version.
- Customize your deployment's storage according to your needs.
- Select a network configuration.

9. Optionally, you can use the cost estimator to simulate the cost of your deployment.

10. Click **Create deployment** to confirm. A list of your Data Warehouse for ClickHouse® deployments displays, showing the newly created resources.
Loading