max.request.size not working

[smoi]

we are using kafka cluster deployed on kubernetes through terraform we're trying to change the message size but the changes are not applied. It seems that there's a lack of documentation on this theme

this is our configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: kafka
namespace: "{{ $.Values.global.namespace }}"
spec:
entityOperator:
topicOperator: {}
userOperator: {}
kafkaExporter:
topicRegex: "."
groupRegex: "."
kafka:
template:
pod:
config:
default.replication.factor: "{{ .Values.kafka.defaultReplication }}"
min.insync.replicas: "{{ .Values.kafka.minInsyncReplicas }}"
offsets.topic.replication.factor: "{{ .Values.kafka.offsetsTopicReplicationFactor }}"
transaction.state.log.replication.factor: "{{ .Values.kafka.transactionStateLogReplicationFactor }}"
transaction.state.log.min.isr: "{{ .Values.kafka.transactionStateLogMinIsr }}"
log.message.format.version: "3.1"
inter.broker.protocol.version: "3.1"
producer.max.request.size: 7179317
message.max.bytes: 7179317
max.message.bytes: 7179317
socket.request.max.bytes: 7179317
request.max.bytes: 7179317
version: 3.1.0
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 2Gi
deleteClaim: false
replicas: "{{ .Values.kafka.kafkaReplicas }}"
readinessProbe:
initialDelaySeconds: 60
timeoutSeconds: 25
periodSeconds: 30
failureThreshold: 15
livenessProbe:
initialDelaySeconds: 90
timeoutSeconds: 15
periodSeconds: 30
failureThreshold: 5
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
zookeeper:
readinessProbe:
initialDelaySeconds: 60
timeoutSeconds: 25
periodSeconds: 30
failureThreshold: 15
livenessProbe:
initialDelaySeconds: 90
timeoutSeconds: 15
periodSeconds: 30
failureThreshold: 5
storage:
type: persistent-claim
size: 2Gi
deleteClaim: false
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: zookeeper-metrics-config.yml
replicas: "{{ .Values.kafka.zookeeperReplicas }}"
template:
pod:

[scholzj]

You should share an actual custom resource. Not some template you generate it in. You also have to make sure it is properly formatted so that people can read it. This way, nobody can read and understand what it says as it doesn't have any new lines and white spaces -> so it is not really a valid YAML.

[smoi]

this is part of the actual configuration of our Kafka, it should be a valid yaml I don't know why github does not format it correctly, I try to update it with the full file

[scholzj]

It's not an actual Kafka YAML => it is clearly some custom template to generate it. But that is useless for people who do not know the template and do not know how you use it. You have to share the actual Kafka resource => kubectl get kafka -o yaml.

[smoi]

that is basically the same as the one in the example of the kafka operator https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/kafka/kafka-persistent.yaml