Skip to content

Commit dd1629f

Browse files
thild42thild42
committed
Update Kafka capabilities guide
1 parent df6178c commit dd1629f

15 files changed

+886
-346
lines changed

‎pages/platform/databases/kafka_01_capabilities/guide.de-de.md

Lines changed: 60 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,10 @@ section: Kafka - Guides
66
order: 010
77
routes:
88
canonical: 'https://docs.ovh.com/gb/en/publiccloud/databases/kafka/capabilities/'
9-
updated: 2023-01-19
9+
updated: 2023-04-24
1010
---
1111

12-
**Last updated January 19th, 2023**
12+
**Last updated April 24th, 2023**
1313

1414
## Objective
1515

@@ -36,22 +36,23 @@ Kafka nodes have to be in the same region. Multi-AZ is currently not supported.
3636

3737
The Public Cloud Databases offer supports the following Kafka versions:
3838

39-
- Kafka 2.8
4039
- Kafka 3.0
40+
- Kafka 3.1
41+
- Kafka 3.2
42+
- Kafka 3.3
4143

42-
You can follow Kafka Release Cycle on their official page : <https://kafka.apache.org/downloads>
44+
Please refer to the [DBMS lifecycle policy guide](https://docs.ovh.com/ie/en/publiccloud/databases/lifecycle-policy/) for recommendations on version upgrades and end of life announcements of major versions. Additionally, you can follow Kafka Release Cycle on their official page: <https://kafka.apache.org/downloads>
4345

4446
### Kafka clients
4547

4648
You can use any of the Kafka-recommended clients to access your cluster.
4749

48-
Please note that Kafka Connect isn't available so far.
50+
Kafka Connect and Kafka Mirrormaker 2 are available at OVHcloud.
4951

5052
### Plans
5153

52-
Three plans are available:
54+
Two plans are available:
5355

54-
- *Essential*
5556
- *Business*
5657
- *Enterprise*
5758

@@ -64,10 +65,13 @@ Here is an overview of the various plans' capabilities:
6465

6566
Your choice of plan affects the number of nodes your cluster can run, the SLA, and a few other features such as read replicas or backup retention.
6667

68+
> [!primary]
69+
> Be aware that you will be able to upgrade your plan but will not be able to downgrade it afterwards.
70+
6771
#### Nodes and replicas
6872

69-
- **Business**: The cluster is delivered with 3 nodes by default.
70-
- **Enterprise**: The cluster is delivered with 6 nodes by default.
73+
- **Business**: the cluster is delivered with 3 nodes by default.
74+
- **Enterprise**: the cluster is delivered with 6 nodes by default.
7175

7276
#### License type
7377

@@ -80,24 +84,33 @@ Here are the node types you can choose from:
8084

8185
**Business plans**
8286

83-
| Name | Disk (GB) | Cores | Memory (GB) |
84-
| ------- | --------- | ----- | ----------- |
85-
| db1-7 | 960 | 2 | 7 |
86-
| db1-15 | 1920 | 4 | 15 |
87-
| db1-30 | 3840 | 8 | 30 |
88-
| db1-60 | 7680 | 16 | 60 |
87+
| Name | Storage | vCore | Memory (GB) |
88+
| ------- | ------------------------ | ----- | ----------- |
89+
| db1-4 | From 480 GB to 1.44 TB | 2 | 4 |
90+
| db1-7 | From 960 GB to 2.88 TB | 2 | 7 |
91+
| db1-15 | From 1.92 TB to 5.76 TB | 4 | 15 |
92+
| db1-30 | From 3.84 TB to 11.52 TB | 8 | 30 |
93+
| db1-60 | From 7.68 TB to 23.04 TB | 16 | 60 |
8994

9095
**Enterprise plans**
9196

92-
| Name | Disk (GB) | Cores | Memory (GB) |
93-
| ------- | --------- | ----- | ----------- |
94-
| db1-7 | 1920 | 2 | 7 |
95-
| db1-15 | 3840 | 4 | 15 |
96-
| db1-30 | 7680 | 8 | 30 |
97-
| db1-60 | 15360 | 16 | 60 |
97+
| Name | Storage | vCore | Memory (GB) |
98+
| ------- | ------------------------- | ----- | ----------- |
99+
| db1-7 | From 1.92 TB to 5.76 TB | 2 | 7 |
100+
| db1-15 | From 3.84 TB to 11.52 TB | 4 | 15 |
101+
| db1-30 | From 7.68 TB to 23.04 TB | 8 | 30 |
102+
| db1-60 | From 15.36 TB to 46.08 TB | 16 | 60 |
98103

99104
Right now, all nodes of a given cluster should be of the same type and distributed in the same region.
100105

106+
#### Flexible storage
107+
108+
You can increased the storage of your cluster up to the maximum allowed for a given reference. Please refer to the [Resize your cluster storage guide](https://docs.ovh.com/ie/en/publiccloud/databases/resize-cluster-storage/) for more information.
109+
110+
#### Node template upgrade
111+
112+
You can upgrade the node template of your cluster to scale your hardware resources up. This operation causes no interruption of service but be aware that you will not be able to downgrade the node template afterwards.
113+
101114
#### Disk type
102115

103116
The type of storage available may vary according to the region your cluster lives in: see [Availability of Public Cloud products](https://www.ovhcloud.com/de/public-cloud/regions-availability/) for more information about block storage type availability depending on region. Thus, your cluster may be backed by e.g. *High Speed* or *High Speed Gen2* block storage.
@@ -106,6 +119,18 @@ Also, the performance caracteristics of the various storage offerings may vary d
106119

107120
Public Cloud Databases will select the most efficient disk type for your cluster depending on your cluster parameters.
108121

122+
#### Effective storage
123+
124+
The disk size listed above is the total disk size of the underlying machine. However, a small part of it goes towards the OS install.
125+
126+
We try hard to avoid "disk full" situations that could be harmful to cluster health. Therefore:
127+
128+
1. When reaching a concerning level of disk usage, a warning email is sent.
129+
2. When reaching a concerning level of disk usage, the service is moved in the "DISK_FULL" state, and "read-only" mode, meaning no more writes can be done.
130+
3. You then have the ability to upgrade to a higher service plan with more storage.
131+
132+
See the [Handling «Disk Full» situations documentation](https://docs.ovh.com/ie/en/publiccloud/databases/handling-disk-full/) for more information.
133+
109134
### Features
110135
#### Network
111136
Kafka clusters are reachable through a random port, attributed during cluster creation. Once your cluster is in **RUNNING** status, the Service URI will display the port to use.
@@ -121,25 +146,36 @@ Here are some considerations to take into account when using private network:
121146
- When connecting from outside subnet, Openstack IP gateway must be enabled in the subnet use for the Database service. The customer is responsible for any other custom network setup.
122147

123148

149+
##### Authorised IPs
150+
151+
Once your service is up and running, you will be able to specify IP addresses (or CIDR blocks) to authorise incoming traffic, until then your service will be unreachable.
152+
124153
#### Maximum simultaneous connections
125154

126155
The number of simultaneous connections in Public Cloud Databases for Kafka depends on the available total memory on the node.
127156
We allow approximately 100 connections per 4 GB of RAM memory, capped to a maximum of 1000 active connections.
128157

129158
So for example on a server with 7 GB memory, you will get approximately 200 connections, and with 15 GB memory you will get 400 connections.
130159

160+
#### Schema registry
161+
162+
Schema registry is supported and can be activated on a running Kafka service by setting the `schemaRegistry` to `true` with an API call:
163+
164+
> [!api]
165+
>
166+
> @api {PUT} /cloud/project/{serviceName}/database/kafka/{clusterId}
167+
131168
#### Advanced parameters
132169

133-
We do not currently support Kafka advanced parameters.
170+
You can customize your Kafka further through the use of advanced parameters. See the [Advanced parameters references documentation](https://help.ovhcloud.com/csm/en-gb-public-cloud-databases-kafka-advanced-parameters-references?id=kb_article_view&sysparm_article=KB0054654) for more information about which ones are supported.
134171

135172
#### Backups
136173

137174
Kafka is a streaming tool. We don't backup Kafka data.
138175

139176
#### Logs and metrics
140177

141-
Logs and metrics are available via the OVHcloud Public Cloud Control Panel.
142-
As of today, you can't export logs and metrics, nor plug them into a remote tool.
178+
Logs and metrics are available through the Control Panel and the API. Additionally, cross service integration could be configured to leverage your logs and metrics in other Public Cloud Database services. You could then view your MySQL logs in Opensearch and metrics in Grafana (metrics have to be exported first in a time series compatible engine such as PostgreSQL or M3db). See the [Cross Service Integration documentation](https://help.ovhcloud.com/csm/en-gb-public-cloud-databases-kafka-advanced-parameters-references?id=kb_article_view&sysparm_article=KB0054654) for more information.
143179

144180
- **Logs retention**: 1000 lines of logs
145181
- **Metrics retention**: 1 calendar month

‎pages/platform/databases/kafka_01_capabilities/guide.en-asia.md

Lines changed: 59 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ slug: kafka/capabilities
44
excerpt: Discover the capabilities and limitations of Public Cloud Databases for Kafka
55
section: Kafka - Guides
66
order: 010
7-
updated: 2023-01-19
7+
updated: 2023-04-24
88
---
99

10-
**Last updated January 19th, 2023**
10+
**Last updated April 24th, 2023**
1111

1212
## Objective
1313

@@ -34,10 +34,12 @@ Kafka nodes have to be in the same region. Multi-AZ is currently not supported.
3434

3535
The Public Cloud Databases offer supports the following Kafka versions:
3636

37-
- Kafka 2.8
3837
- Kafka 3.0
38+
- Kafka 3.1
39+
- Kafka 3.2
40+
- Kafka 3.3
3941

40-
You can follow Kafka Release Cycle on their official page : <https://kafka.apache.org/downloads>
42+
Please refer to the [DBMS lifecycle policy guide](https://docs.ovh.com/ie/en/publiccloud/databases/lifecycle-policy/) for recommendations on version upgrades and end of life announcements of major versions. Additionally, you can follow Kafka Release Cycle on their official page: <https://kafka.apache.org/downloads>
4143

4244
### Kafka clients
4345

@@ -47,9 +49,8 @@ Please note that Kafka Connect and Kafka Mirrormaker 2 aren't available so far.
4749

4850
### Plans
4951

50-
Three plans are available:
52+
Two plans are available:
5153

52-
- *Essential*
5354
- *Business*
5455
- *Enterprise*
5556

@@ -62,10 +63,13 @@ Here is an overview of the various plans' capabilities:
6263

6364
Your choice of plan affects the number of nodes your cluster can run, the SLA, and a few other features such as read replicas or backup retention.
6465

66+
> [!primary]
67+
> Be aware that you will be able to upgrade your plan but will not be able to downgrade it afterwards.
68+
6569
#### Nodes and replicas
6670

67-
- **Business**: The cluster is delivered with 3 nodes by default.
68-
- **Enterprise**: The cluster is delivered with 6 nodes by default.
71+
- **Business**: the cluster is delivered with 3 nodes by default.
72+
- **Enterprise**: the cluster is delivered with 6 nodes by default.
6973

7074
#### License type
7175

@@ -78,24 +82,33 @@ Here are the node types you can choose from:
7882

7983
**Business plans**
8084

81-
| Name | Disk (GB) | Cores | Memory (GB) |
82-
| ------- | --------- | ----- | ----------- |
83-
| db1-7 | 960 | 2 | 7 |
84-
| db1-15 | 1920 | 4 | 15 |
85-
| db1-30 | 3840 | 8 | 30 |
86-
| db1-60 | 7680 | 16 | 60 |
85+
| Name | Storage | vCore | Memory (GB) |
86+
| ------- | ------------------------ | ----- | ----------- |
87+
| db1-4 | From 480 GB to 1.44 TB | 2 | 4 |
88+
| db1-7 | From 960 GB to 2.88 TB | 2 | 7 |
89+
| db1-15 | From 1.92 TB to 5.76 TB | 4 | 15 |
90+
| db1-30 | From 3.84 TB to 11.52 TB | 8 | 30 |
91+
| db1-60 | From 7.68 TB to 23.04 TB | 16 | 60 |
8792

8893
**Enterprise plans**
8994

90-
| Name | Disk (GB) | Cores | Memory (GB) |
91-
| ------- | --------- | ----- | ----------- |
92-
| db1-7 | 1920 | 2 | 7 |
93-
| db1-15 | 3840 | 4 | 15 |
94-
| db1-30 | 7680 | 8 | 30 |
95-
| db1-60 | 15360 | 16 | 60 |
95+
| Name | Storage | vCore | Memory (GB) |
96+
| ------- | ------------------------- | ----- | ----------- |
97+
| db1-7 | From 1.92 TB to 5.76 TB | 2 | 7 |
98+
| db1-15 | From 3.84 TB to 11.52 TB | 4 | 15 |
99+
| db1-30 | From 7.68 TB to 23.04 TB | 8 | 30 |
100+
| db1-60 | From 15.36 TB to 46.08 TB | 16 | 60 |
96101

97102
Right now, all nodes of a given cluster should be of the same type and distributed in the same region.
98103

104+
#### Flexible storage
105+
106+
You can increased the storage of your cluster up to the maximum allowed for a given reference. Please refer to the [Resize your cluster storage guide](https://docs.ovh.com/ie/en/publiccloud/databases/resize-cluster-storage/) for more information.
107+
108+
#### Node template upgrade
109+
110+
You can upgrade the node template of your cluster to scale your hardware resources up. This operation causes no interruption of service but be aware that you will not be able to downgrade the node template afterwards.
111+
99112
#### Disk type
100113

101114
The type of storage available may vary according to the region your cluster lives in: see [Availability of Public Cloud products](https://www.ovhcloud.com/asia/public-cloud/regions-availability/) for more information about block storage type availability depending on region. Thus, your cluster may be backed by e.g. *High Speed* or *High Speed Gen2* block storage.
@@ -104,6 +117,18 @@ Also, the performance caracteristics of the various storage offerings may vary d
104117

105118
Public Cloud Databases will select the most efficient disk type for your cluster depending on your cluster parameters.
106119

120+
#### Effective storage
121+
122+
The disk size listed above is the total disk size of the underlying machine. However, a small part of it goes towards the OS install.
123+
124+
We try hard to avoid "disk full" situations that could be harmful to cluster health. Therefore:
125+
126+
1. When reaching a concerning level of disk usage, a warning email is sent.
127+
2. When reaching a concerning level of disk usage, the service is moved in the "DISK_FULL" state, and "read-only" mode, meaning no more writes can be done.
128+
3. You then have the ability to upgrade to a higher service plan with more storage.
129+
130+
See the [Handling «Disk Full» situations documentation](https://docs.ovh.com/ie/en/publiccloud/databases/handling-disk-full/) for more information.
131+
107132
### Features
108133
#### Network
109134
Kafka clusters are reachable through a random port, attributed during cluster creation. Once your cluster is in **RUNNING** status, the Service URI will display the port to use.
@@ -119,25 +144,36 @@ Here are some considerations to take into account when using private network:
119144
- When connecting from outside subnet, Openstack IP gateway must be enabled in the subnet use for the Database service. The customer is responsible for any other custom network setup.
120145

121146

147+
##### Authorised IPs
148+
149+
Once your service is up and running, you will be able to specify IP addresses (or CIDR blocks) to authorise incoming traffic, until then your service will be unreachable.
150+
122151
#### Maximum simultaneous connections
123152

124153
The number of simultaneous connections in Public Cloud Databases for Kafka depends on the available total memory on the node.
125154
We allow approximately 100 connections per 4 GB of RAM memory, capped to a maximum of 1000 active connections.
126155

127156
So for example on a server with 7 GB memory, you will get approximately 200 connections, and with 15 GB memory you will get 400 connections.
128157

158+
#### Schema registry
159+
160+
Schema registry is supported and can be activated on a running Kafka service by setting the `schemaRegistry` to `true` with an API call:
161+
162+
> [!api]
163+
>
164+
> @api {PUT} /cloud/project/{serviceName}/database/kafka/{clusterId}
165+
129166
#### Advanced parameters
130167

131-
We do not currently support Kafka advanced parameters.
168+
You can customize your Kafka further through the use of advanced parameters. See the [Advanced parameters references documentation](https://help.ovhcloud.com/csm/en-gb-public-cloud-databases-kafka-advanced-parameters-references?id=kb_article_view&sysparm_article=KB0054654) for more information about which ones are supported.
132169

133170
#### Backups
134171

135172
Kafka is a streaming tool. We don't backup Kafka data.
136173

137174
#### Logs and metrics
138175

139-
Logs and metrics are available via the OVHcloud Public Cloud Control Panel.
140-
As of today, you can't export logs and metrics, nor plug them into a remote tool.
176+
Logs and metrics are available through the Control Panel and the API. Additionally, cross service integration could be configured to leverage your logs and metrics in other Public Cloud Database services. You could then view your MySQL logs in Opensearch and metrics in Grafana (metrics have to be exported first in a time series compatible engine such as PostgreSQL or M3db). See the [Cross Service Integration documentation](https://help.ovhcloud.com/csm/en-gb-public-cloud-databases-kafka-advanced-parameters-references?id=kb_article_view&sysparm_article=KB0054654) for more information.
141177

142178
- **Logs retention**: 1000 lines of logs
143179
- **Metrics retention**: 1 calendar month

0 commit comments

Comments
 (0)