bug: Fluentd cluster output for Kafka does not load defaultTopic from values.yaml which is mandatory parameter

[rajasree1993]

Describe the issue

When Kafka output plugin is enabled in fluentd via values.yaml, fluentd is throwing error that default topic is mandatory and while trying to set via values.yaml, it is not read in the yaml file inside templates

To Reproduce

Enable Kafka using values.yaml.

fluentd:
output:
kafka:
enable: true
brokers: "cloudflow-strimzi-kafka-bootstrap.cloudflow.svc:9092"
topicKey: app
defaultTopic: app
useEventTime: true

Check the fluentd-0 logs. Throws issue for mandatory default_topic for kafka output plugin

kubectl edit clusteroutputs.fluentd.fluent.io fluentd-output-kafka

Output:

kafka:
  brokers: cloudflow-strimzi-kafka-bootstrap.cloudflow.svc:9092
  topicKey: app
  useEventTime: true

Manually adding and restarting fluentd pod , its working

Expected behavior

templates
/fluentd-output-kafka.yaml must read defaultTopic also from values.yaml.

Your Environment

- Fluent Operator version: v1.6.1
- Container Runtime: 
NAME="Red Hat Enterprise Linux"
VERSION="8.4 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.4"
- Operating system: Kubernetes 1.27
- Kernel version: 4.18.0-305.el8.x86_64

How did you install fluent operator?

helm install fluent-operator https://github.com/fluent/fluent-operator/releases/download/v1.6.1/fluent-operator.tgz -f values_updated.yaml

Additional context

No response

[rajasree1993]

Kindly update on this issue as its seen even in latest version

[joshuabaird]

This issue is being closed due to inactivity. Please re-open if it is still relevant.