Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/connect-applications.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ To connect your deployment with BI tools, refer to the [dedicated documentation]

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.
Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/connect-bi-tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This page explains how to integrate your Data Warehouse for ClickHouse® deploym

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## Tableau
Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/create-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ A Data Warehouse is a centralized repository designed to store, manage, and anal

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

## Creating a Data Warehouse Deployment

Expand Down
2 changes: 1 addition & 1 deletion pages/data-warehouse/how-to/edit-autoscaling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This page explains how to edit the autoscaling configuration of your Data Wareho

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

<Message type="important">
Expand Down
2 changes: 1 addition & 1 deletion pages/data-warehouse/how-to/import-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Scaleway Data Warehouse for ClickHouse® allows you to quickly import any type o

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to import data using the ClickHouse® CLI
Expand Down
50 changes: 39 additions & 11 deletions pages/data-warehouse/how-to/manage-databases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,29 +9,57 @@ dates:
import Requirements from '@macros/iam/requirements.mdx'


This page explains how to manage databases in your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com/).

<Message type="note">
During the private beta phase, database management can only be done with the **scwadmin** & via the ClickHouse® CLI, MySQL or HTTPS protocols.
</Message>
This page explains how to manage databases in your Data Warehouse for ClickHouse® deployment.

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage databases using the Scaleway console

During the private beta phase, the management of databases in your Data Warehouse for ClickHouse® deployment is limited to viewing databases in your deployment.
The Scaleway console allows you to create and delete databases for your Data Warehouse for ClickHouse® deployment.

### How to create a database using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Databases** tab. A list of your current databases displays.

4. Click **+ Create database**. A popup displays.

5. Enter a name for your new database. It can can only contain alphanumeric characters, underscores, and dashes.

6. Click **Create database** to confirm.

Your new database appears in the list.

### How to delete a database using the Scaleway console

<Message type="note">
The `default` database cannot be deleted.
</Message>

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click <Icon name="delete" /> next to the name of the database you want to delete. A confirmation popup displays.

4. Enter `DELETE`, then click **Delete database** to confirm.

The deleted database no longer appears in the list.

## How to manage databases using frameworks

[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.
<Message type="tip">
You can also connect using the ClickHouse HTTP console from the **Overview** tab of your deployment. Make sure to enter valid credentials in the top-right fields.
</Message>
[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.

<Message type="tip">
You can also connect using the ClickHouse HTTP console from the **Overview** tab of your deployment. Make sure to enter valid credentials in the top-right fields.
</Message>

Once connected, you can run SQL queries to `CREATE/ALTER/DROP` your `DATABASE/TABLE`.

Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/manage-delete-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This page explains how to manage and delete your Data Warehouse deployment.

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage a Data Warehouse deployment
Expand Down
141 changes: 141 additions & 0 deletions pages/data-warehouse/how-to/manage-storage.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
---
title: How to manage data storage in a Data Warehouse for ClickHouse® deployment
description: This page explains how to manage data storage in a Data Warehouse for ClickHouse® deployment using storage policies
tags: data storage policy strategy options object block
dates:
validation: 2025-10-16
posted: 2025-10-16
---

import Requirements from '@macros/iam/requirements.mdx'

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization

## Overview

Scaleway Data Warehouse for ClickHouse® uses ClickHouse®'s native storage policies to decide where table data lives: on Scaleway Block storage, and/or on Object Storage (Amazon S3). Policies are applied at table-level during table creation, using the settings presented below. You can choose between three policies for your deployment:

- `tiered` (default in Scaleway Data Warehouse for ClickHouse®)
- `s3_cache` (cache mode)
- `default` (local-only mode)

The table below provides an overview of each storage policy's behavior and use case.

| Storage&nbsp;policy | Behavior | When to use |
|------|---------|------------|
| `tiered` | Writes on Block Storage first, and parts are automatically moved to Object Storage when disk is 90% full. | General purpose setting for fast local writes and transparent spillover to Object Storage as the dataset grows. |
| `s3_cache` | Data is stored on Object Storage. A local Block Storage cache layer keeps frequently-read parts to accelerate repeated reads. | Large datasets that mostly live in Object Storage, with repeated reads on smaller subsets that benefit from local caching. |
| `default` | Data is stored on Block storage only. | Small datasets where lowest latency for reads/writes on local disk is desired and capacity fits the Block Storage volume. |

## Tiered storage policy

### Applying the tiered policy

**Tiered** is the deployment's default policy. Data is stored on the `hot` volume (Block Storage) until it reaches 90% of its capacity, then data is moved to the `cold` volume. This mechanism is controlled by `move_factor:0.1`.

[Connect to you deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to apply the `tiered` storage policy.

<Tabs id="tiered-explicit-implicit">
<TabsTab label="Explicit">

Storage policy is explicitly defined as `tiered`.

```sql
CREATE TABLE tiered_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 'tiered';
```

</TabsTab>
<TabsTab label="Implicit">

Storage policy is not defined, ClickHouse® applies the `tiered` storage policy by default.

```sql
CREATE TABLE dw.tiered_table_implicit
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id;
```
</TabsTab>
</Tabs>

<Message type="note">
If you omit the `storage_policy` setting, ClickHouse® will automatically apply the `tiered` policy.
</Message>

### Moving data with the tiered policy

When using the `tiered` storage policy, you can manually move partitions between `hot` (Block Storage) and `cold` (Object Storage) volumes using the SQL queries below.

- Moving specific partitions to the `cold` volume (Object Storage):

```sql
ALTER TABLE tiered_table MOVE PARTITION '2024-08' TO VOLUME 'cold';
```

- Moving a non-partitioned table (`tuple()`) to the `cold` volume (Object Storage):

```sql
ALTER TABLE tiered_table_without_partitions MOVE PARTITION tuple() TO VOLUME 'cold';
```

- Move a partition to the `hot` volume (Block Storage):

```sql
ALTER TABLE tiered_table MOVE PARTITION '2024-08' TO VOLUME 'hot';
```
- Moving a non-partitioned table (`tuple()`) to the `hot` volume (Block Storage):

```sql
ALTER TABLE tiered_table_without_partitions MOVE PARTITION tuple() TO VOLUME 'hot';
```

## S3 cache policy

[Connect to you deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to create a new table with the `s3_cache` policy enabled.
```sql
CREATE TABLE cache_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 's3_cache';
```

## Local-only policy

[Connect to you deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to create a new table with the `default` (Block Storage only) policy enabled.

```sql
CREATE TABLE local_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 'default';
```

107 changes: 97 additions & 10 deletions pages/data-warehouse/how-to/manage-users.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,63 @@ dates:
import Requirements from '@macros/iam/requirements.mdx'


This page explains how to manage users in your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com/).

<Message type="note">
During the private beta phase, user management can only be done with the **scwadmin** user via the ClickHouse® CLI, MySQL or HTTPS protocols.
</Message>
This page explains how to manage users in your Data Warehouse for ClickHouse® deployment.

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage users using the Scaleway console

During the private beta phase, the management of users for your Data Warehouse for ClickHouse® deployment is limited to:
- Viewing users in your deployment
- Changing the password of an existing user
The Scaleway console allows you to create users for your Data Warehouse for ClickHouse® deployment, to grant them admin rights, to update their password, and to delete them.

### How to create a user using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click **+ Create user**. A popup displays.

5. Enter a username. It must be unique can contain up to 63 characters, and must start with a letter. Only alphanumeric characters, underscores, and dashes are accepted.

6. Enter a password, or generate one automatically.

7. Enable the toggle if you want to grant admin rights to the user.

8. Click **Create user** to proceed.

The newly created user appears in the list of your deployment's users.

### How to update a user's password and admin rights

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click <Icon name="more" /> next to the name of the user you want to update, then click ¨**Update user**. A popup displays.

5. Update the user's password and/or admin rights, then click **Update user** to confirm.

### How to delete a user using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click <Icon name="delete" /> next to the name of the user you want to delete. A confirmation popup displays.

5. Enter `DELETE`, then click **Delete user** to confirm.``

The user is deleted and no longer appears in the list.

## How to manage users using frameworks

Expand All @@ -37,4 +76,52 @@ During the private beta phase, the management of users for your Data Warehouse f

Once connected, you can run SQL queries using `USER`, `ROLE`, and `ROW POLICY` to create new users and grant them the desired permissions via roles.

Refer to the official ClickHouse® documentation on [users and roles](https://clickhouse.com/docs/sql-reference/statements/create/user) for more information.
Refer to the official ClickHouse® documentation on [users and roles](https://clickhouse.com/docs/sql-reference/statements/create/user) for more information.

## Manage user permissions

[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.

Once connected, you can run SQL as shown below to grant the desired permissions to users.

**Grant permission to create databases**

```sql
GRANT CREATE DATABASE ON . TO user;
```

**Grant permission to create tables in any database**

```sql
GRANT CREATE TABLE ON . TO user;
```

**Grant permission to create tables in a specific database**

```sql
GRANT CREATE TABLE ON database_name.* TO user;
```

**Grant data manipulation permissions for any table in any database**

```sql
GRANT INSERT, UPDATE, DELETE ON . TO user;
```

**Grant data manipulation permissions for any table in a specific database**

```sql
GRANT INSERT, UPDATE, DELETE ON database_name.* TO user;
```

**Grant data manipulation permissions for a specific table in a specific database**

```sql
GRANT INSERT, UPDATE, DELETE ON database_name.table_name TO user;
```

**Grant every permissions on every database**

```sql
GRANT CREATE DATABASE, CREATE TABLE, INSERT, UPDATE, DELETE ON . TO user;
```
Loading
Loading