Optional automatic metadata refresh & retry on NotLeaderForPartition

[justramesh2000]

Currently, kafka-go returns a NotLeaderForPartition error if a producer writes to a partition
whose leader info is stale. In other clients (e.g., Java Kafka client), this is handled automatically:
metadata is refreshed and the write is retried transparently.

Suggestion/Proposal : potentially add an optional WriterConfig flag, e.g., AutoRefreshMetadata bool,
which automatically retries after refreshing partition metadata ?

Description : The idea is to add a config to like
writer := kafka.NewWriter(kafka.WriterConfig{
Brokers: []string{"kafka1:9092", "kafka2:9092", "kafka3:9092"},
Topic: topic,
Balancer: &kafka.LeastBytes{},
AutoRefreshMetadata: true, // new config flag, may be false by default to avoid impact performance and B/w compatibility
})

With this enabled, the writer would:

Detect a NotLeaderForPartition error on a write.

Refresh partition metadata from the cluster.

Retry the write automatically (maybe a configurable number of times ? ).

Supporting documentation

https://issues.apache.org/jira/browse/KAFKA-16395

https://aiven.io/docs/products/kafka/troubleshooting/non-leader-for-partition#:~:text=Most%20producers%20libraries,in%20fact%20harmless.

I am happy to pick this if team is okay with the proposal. Thanks !