Skip to content

Commit c2ce836

Browse files
thild42thild42
committed
Update Kafka Connect adv param
1 parent c9b80b5 commit c2ce836

File tree

15 files changed

+330
-2415
lines changed

15 files changed

+330
-2415
lines changed

‎pages/public_cloud/public_cloud_databases/kafkaconnect_02_advanced_parameters_references/guide.de-de.md

Lines changed: 22 additions & 161 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,17 @@
11
---
22
title: Kafka Connect - Advanced parameters references
33
excerpt: This guide lists all the supported advanced parameters that allow you to configure your Public Cloud Databases for MySQL according to your use cases
4-
updated: 2023-02-06
4+
updated: 2025-08-04
55
---
66

7-
<style>
8-
#content table,
9-
.ovh-documentation table {margin-bottom:25px;overflow:unset !important;}
10-
11-
#content tbody,
12-
.ovh-documentation tbody {display: inline-table !important;width:100% !important;}
13-
14-
#content thead,
15-
.ovh-documentation thead {display:none}
16-
17-
#content tr:nth-child(2n),
18-
.ovh-documentation tr:nth-child(2n) {
19-
background: none !important;
20-
}
21-
#content td:first-child,
22-
.ovh-documentation td:first-child {
23-
background:#efefef;
24-
font-weight:600;
25-
vertical-align:top;
26-
width:11ch;
27-
}
28-
</style>
29-
307
## Objective
318

329
This guide lists all the supported advanced parameters that allow you to configure your Public Cloud Databases for Kafka Connect according to your use cases.
3310

3411
## Requirements
3512

3613
- A [Public Cloud project](/links/public-cloud/public-cloud) in your OVHcloud account
37-
- A database running on your OVHcloud Public Cloud Databases ([this guide](/pages/public_cloud/public_cloud_databases/databases_01_order_control_panel) can help you to meet this requirement)
14+
- An analytics service running on your OVHcloud Public Cloud project ([this guide](/pages/public_cloud/data_analytics/analytics/analytics_getting_started) can help you to meet this requirement)
3815
- Access to your [OVHcloud Control Panel](/links/manager) or to the [OVHcloud API](https://api.ovh.com/console/)
3916

4017
## Instructions
@@ -46,150 +23,34 @@ This guide lists all the supported advanced parameters that allow you to configu
4623
4724
### Using the OVHcloud Control Panel
4825

49-
Please refer to this [guide](/pages/public_cloud/public_cloud_databases/databases_03_advanced_configuration#using-the-ovhcloud-control-panel) to find out how to change your advanced parameters from the OVHcloud Control Panel.
26+
Please refer to this [guide](/pages/public_cloud/data_analytics/analytics/analytics_advanced_parameters#using-the-ovhcloud-control-panel) to find out how to change your advanced parameters from the OVHcloud Control Panel.
5027

5128
### Using API
5229

53-
Please refer to this [guide](/pages/public_cloud/public_cloud_databases/databases_03_advanced_configuration#using-api) to find out how to change your advanced parameters from the OVHcloud API.
30+
Please refer to this [guide](/pages/public_cloud/data_analytics/analytics/analytics_advanced_parameters#using-api) to find out how to change your advanced parameters from the OVHcloud API.
5431

5532
### Advanced parameters for Kafka Connect
5633

5734
Below you can find a summary of every configuration option available for Kafka Connect service:
5835

59-
| | |
60-
|---|---|
61-
| Parameter | `kafka_connect.connector_client_config_override_policy` |
62-
| Value type | string |
63-
| Minimum | |
64-
| Maximum | |
65-
| Values | "All", "None" |
66-
| Description | Defines which client configurations can be overridden by the connector. Default is None |
67-
68-
| | |
69-
|---|---|
70-
| Parameter | `kafka_connect.consumer_auto_offset_reset` |
71-
| Value type | string |
72-
| Minimum | |
73-
| Maximum | |
74-
| Values | "earliest", "latest" |
75-
| Description | What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest |
76-
77-
| | |
78-
|---|---|
79-
| Parameter | `kafka_connect.consumer_fetch_max_bytes` |
80-
| Value type | long |
81-
| Minimum | 1048576 |
82-
| Maximum | 104857600 |
83-
| Values | |
84-
| Description | Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum. |
85-
86-
| | |
87-
|---|---|
88-
| Parameter | `kafka_connect.consumer_isolation_level` |
89-
| Value type | string |
90-
| Minimum | |
91-
| Maximum | |
92-
| Values | "read_committed", "read_uncommitted" |
93-
| Description | Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired. |
94-
95-
| | |
96-
|---|---|
97-
| Parameter | `kafka_connect.consumer_max_partition_fetch_bytes` |
98-
| Value type | long |
99-
| Minimum | 1048576 |
100-
| Maximum | 104857600 |
101-
| Values | |
102-
| Description | Records are fetched in batches by the consumer. If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. |
103-
104-
| | |
105-
|---|---|
106-
| Parameter | `kafka_connect.consumer_max_poll_interval_ms` |
107-
| Value type | long |
108-
| Minimum | 1 |
109-
| Maximum | 2147483647 |
110-
| Values | |
111-
| Description | The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000). |
112-
113-
| | |
114-
|---|---|
115-
| Parameter | `kafka_connect.consumer_max_poll_records` |
116-
| Value type | long |
117-
| Minimum | 1 |
118-
| Maximum | 10000 |
119-
| Values | |
120-
| Description | The maximum number of records returned in a single call to poll() (defaults to 500). |
121-
122-
| | |
123-
|---|---|
124-
| Parameter | `kafka_connect.offset_flush_interval_ms` |
125-
| Value type | long |
126-
| Minimum | 1 |
127-
| Maximum | 100000000 |
128-
| Values | |
129-
| Description | The interval at which to try committing offsets for tasks (defaults to 60000). |
130-
131-
| | |
132-
|---|---|
133-
| Parameter | `kafka_connect.offset_flush_timeout_ms` |
134-
| Value type | long |
135-
| Minimum | 1 |
136-
| Maximum | 2147483647 |
137-
| Values | |
138-
| Description | Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000). |
139-
140-
| | |
141-
|---|---|
142-
| Parameter | `kafka_connect.producer_batch_size` |
143-
| Value type | long |
144-
| Minimum | 0 |
145-
| Maximum | 5242880 |
146-
| Values | |
147-
| Description | This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will 'linger' for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384). |
148-
149-
| | |
150-
|---|---|
151-
| Parameter | `kafka_connect.producer_buffer_memory` |
152-
| Value type | long |
153-
| Minimum | 5242880 |
154-
| Maximum | 134217728 |
155-
| Values | |
156-
| Description | The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432). |
157-
158-
| | |
159-
|---|---|
160-
| Parameter | `kafka_connect.producer_compression_type` |
161-
| Value type | string |
162-
| Minimum | |
163-
| Maximum | |
164-
| Values | "gzip", "lz4", "none", "snappy", "zstd" |
165-
| Description | Specify the default compression type for producers. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'none' which is the default and equivalent to no compression. |
166-
167-
| | |
168-
|---|---|
169-
| Parameter | `kafka_connect.producer_linger_ms` |
170-
| Value type | long |
171-
| Minimum | 0 |
172-
| Maximum | 5000 |
173-
| Values | |
174-
| Description | This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will 'linger' for the specified time waiting for more records to show up. Defaults to 0. |
175-
176-
| | |
177-
|---|---|
178-
| Parameter | `kafka_connect.producer_max_request_size` |
179-
| Value type | long |
180-
| Minimum | 131072 |
181-
| Maximum | 67108864 |
182-
| Values | |
183-
| Description | This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests. |
184-
185-
| | |
186-
|---|---|
187-
| Parameter | `kafka_connect.session_timeout_ms` |
188-
| Value type | long |
189-
| Minimum | 1 |
190-
| Maximum | 2147483647 |
191-
| Values | |
192-
| Description | The timeout in milliseconds used to detect failures when using Kafka’s group management facilities (defaults to 10000). |
36+
| Parameters |
37+
|------------|
38+
|**kafka_connect.connector_client_config_override_policy** `string`<br>values: `All`, `None`<br>Defines what client configurations can be overridden by the connector. Default is None<br>|
39+
|**kafka_connect.consumer_auto_offset_reset** `string`<br>values: `earliest`, `latest`<br>What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest<br>|
40+
|**kafka_connect.consumer_fetch_max_bytes** `long`<br>min: `1048576` max: `104857600`<br>Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.<br>|
41+
|**kafka_connect.consumer_isolation_level** `string`<br>values: `read_committed`, `read_uncommitted`<br>Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.<br>|
42+
|**kafka_connect.consumer_max_partition_fetch_bytes** `long`<br>min: `1048576` max: `104857600`<br>Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. <br>|
43+
|**kafka_connect.consumer_max_poll_interval_ms** `long`<br>min: `1` max: `2147483647`<br>The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).<br>|
44+
|**kafka_connect.consumer_max_poll_records** `long`<br>min: `1` max: `10000`<br>The maximum number of records returned in a single call to poll() (defaults to 500).<br>|
45+
|**kafka_connect.offset_flush_interval_ms** `long`<br>min: `1` max: `100000000`<br>The interval at which to try committing offsets for tasks (defaults to 60000).<br>|
46+
|**kafka_connect.offset_flush_timeout_ms** `long`<br>min: `1` max: `2147483647`<br>Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).<br>|
47+
|**kafka_connect.producer_batch_size** `long`<br>min: `0` max: `5242880`<br>This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will 'linger' for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).<br>|
48+
|**kafka_connect.producer_buffer_memory** `long`<br>min: `5242880` max: `134217728`<br>The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).<br>|
49+
|**kafka_connect.producer_compression_type** `string`<br>values: `gzip`, `lz4`, `none`, `snappy`, `zstd`<br>Specify the default compression type for producers. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'none' which is the default and equivalent to no compression.<br>|
50+
|**kafka_connect.producer_linger_ms** `long`<br>min: `0` max: `5000`<br>This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will 'linger' for the specified time waiting for more records to show up. Defaults to 0.<br>|
51+
|**kafka_connect.producer_max_request_size** `long`<br>min: `131072` max: `67108864`<br>This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.<br>|
52+
|**kafka_connect.scheduled_rebalance_max_delay_ms** `long`<br>min: `0` max: `600000`<br>The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.<br>|
53+
|**kafka_connect.session_timeout_ms** `long`<br>min: `1` max: `2147483647`<br>The timeout in milliseconds used to detect failures when using Kafka’s group management facilities (defaults to 10000).<br>|
19354

19455
## Go further
19556

0 commit comments

Comments
 (0)