Skip to content

Commit c993fd6

Browse files
authored
Merge pull request #50544 from abrennan89/kafkabroker
[SRVKE-1271] Add kafka broker config map options
2 parents 4c48e89 + a0dbb84 commit c993fd6

File tree

3 files changed

+107
-0
lines changed

3 files changed

+107
-0
lines changed
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * /serverless/admin_guide/serverless-kafka-admin.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="serverless-kafka-broker-configmap_{context}"]
7+
= Configuring Kafka broker settings
8+
9+
You can configure the replication factor, bootstrap servers, and the number of topic partitions for a Kafka broker, by creating a config map and referencing this config map in the Kafka `Broker` object.
10+
11+
:FeatureName: Kafka broker
12+
include::snippets/technology-preview.adoc[]
13+
14+
.Prerequisites
15+
16+
* You have cluster or dedicated administrator permissions on {product-title}.
17+
* The {ServerlessOperatorName}, Knative Eventing, and the `KnativeKafka` custom resource (CR) are installed on your {product-title} cluster.
18+
* You have created a project or have access to a project that has the appropriate roles and permissions to create applications and other workloads in {product-title}.
19+
* You have installed the OpenShift (`oc`) CLI.
20+
21+
.Procedure
22+
23+
. Modify the `kafka-broker-config` config map, or create your own config map that contains the following configuration:
24+
+
25+
[source,yaml]
26+
----
27+
apiVersion: v1
28+
kind: ConfigMap
29+
metadata:
30+
name: <config_map_name> <1>
31+
namespace: <namespace> <2>
32+
data:
33+
default.topic.partitions: <integer> <3>
34+
default.topic.replication.factor: <integer> <4>
35+
bootstrap.servers: <list_of_servers> <5>
36+
----
37+
<1> The config map name.
38+
<2> The namespace where the config map exists.
39+
<3> The number of topic partitions for the Kafka broker. This controls how quickly events can be sent to the broker. A higher number of partitions requires greater compute resources.
40+
<4> The replication factor of topic messages. This prevents against data loss. A higher replication factor requires greater compute resources and more storage.
41+
<5> A comma separated list of bootstrap servers. This can be inside or outside of the {product-title} cluster, and is a list of Kafka clusters that the broker receives events from and sends events to.
42+
+
43+
[IMPORTANT]
44+
====
45+
The `default.topic.replication.factor` value must be less than or equal to the number of Kafka broker instances in your cluster. For example, if you only have one Kafka broker, the `default.topic.replication.factor` value should not be more than `"1"`.
46+
====
47+
+
48+
.Example Kafka broker config map
49+
[source,yaml]
50+
----
51+
apiVersion: v1
52+
kind: ConfigMap
53+
metadata:
54+
name: kafka-broker-config
55+
namespace: knative-eventing
56+
data:
57+
default.topic.partitions: "10"
58+
default.topic.replication.factor: "3"
59+
bootstrap.servers: "my-cluster-kafka-bootstrap.kafka:9092"
60+
----
61+
62+
. Apply the config map:
63+
+
64+
[source,yaml]
65+
----
66+
$ oc apply -f <config_map_filename>
67+
----
68+
69+
. Specify the config map for the Kafka `Broker` object:
70+
+
71+
.Example Broker object
72+
[source,yaml]
73+
----
74+
apiVersion: eventing.knative.dev/v1
75+
kind: Broker
76+
metadata:
77+
name: <broker_name> <1>
78+
namespace: <namespace> <2>
79+
annotations:
80+
eventing.knative.dev/broker.class: Kafka <3>
81+
spec:
82+
config:
83+
apiVersion: v1
84+
kind: ConfigMap
85+
name: <config_map_name> <4>
86+
namespace: <namespace> <5>
87+
...
88+
----
89+
<1> The broker name.
90+
<2> The namespace where the broker exists.
91+
<3> The broker class annotation. In this example, the broker is a Kafka broker that uses the class value `Kafka`.
92+
<4> The config map name.
93+
<5> The namespace where the config map exists.
94+
95+
. Apply the broker:
96+
+
97+
[source,yaml]
98+
----
99+
$ oc apply -f <broker_filename>
100+
----

serverless/admin_guide/serverless-kafka-admin.adoc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,12 @@ include::modules/serverless-kafka-broker-sasl-default-config.adoc[leveloffset=+2
4747
include::modules/serverless-kafka-tls-channels.adoc[leveloffset=+2]
4848
include::modules/serverless-kafka-sasl-channels.adoc[leveloffset=+2]
4949

50+
include::modules/serverless-kafka-broker-configmap.adoc[leveloffset=+1]
51+
52+
[role="_additional-resources"]
53+
.Additional resources
54+
* xref:../../serverless/develop/serverless-using-brokers.adoc#serverless-using-brokers[Creating brokers]
55+
5056
[id="additional-resources_serverless-kafka-admin"]
5157
[role="_additional-resources"]
5258
== Additional resources

serverless/develop/serverless-using-brokers.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,3 +51,4 @@ include::modules/serverless-describe-broker-kn.adoc[leveloffset=+2]
5151
xref:../../serverless/discover/knative-event-sources.adoc#knative-event-sources[Event sources]
5252
* xref:../../serverless/develop/serverless-event-delivery.adoc#serverless-event-delivery[Event delivery]
5353
* xref:../../serverless/develop/serverless-kafka-developer.adoc#serverless-kafka-developer-broker[Kafka broker]
54+
* xref:../../serverless/admin_guide/serverless-kafka-admin.adoc#serverless-kafka-admin[Configuring Knative Kafka]

0 commit comments

Comments
 (0)