out_calyptia: retry agent registration on flush callback

[niedbalski]

Description

if register_retry_on_flush is set (default true, my proposal)0, agent registration is retried on each flush callback.
if set to false then registration will cause to abort the plugin initialisation.

Testing

• Example configuration file for the change
• Debug log output from testing the change
  Use with this configuration

[SERVICE]
    Log_Level debug

[CUSTOM]
    name                calyptia
    api_key xxx
    log_level           debug
    calyptia_host       localhost
    calyptia_port       5000
    calyptia_tls        off
    calyptia_tls.verify off
    register_retry_on_flush false


If set to true (my proposed default)

[2024/11/27 22:09:00] [ warn] [output:calyptia:calyptia.0] agent registration failed
[2024/11/27 22:09:00] [debug] [out flush] cb_destroy coro_id=0
[2024/11/27 22:09:00] [debug] [retry] new retry created for task_id=0 attempts=1
[2024/11/27 22:09:00] [ warn] [engine] failed to flush chunk '57405-1732741739.505948000.flb', retry in 11 seconds: task_id=0, input=fluentbit_metrics.0 > output=calyptia.0 (out_id=0)

[2024/11/27 22:09:11] [ info] [output:calyptia:calyptia.0] agent_id not found and register_retry_on_flush=true, attempting registration
[2024/11/27 22:09:11] [debug] [net] socket #36 could not connect to ::1:5000
[2024/11/27 22:09:11] [debug] [net] socket #36 could not connect to 127.0.0.1:5000
[2024/11/27 22:09:11] [debug] [net] could not connect to localhost:5000
[2024/11/27 22:09:11] [debug] [upstream] connection #-1 failed to localhost:5000
[2024/11/27 22:09:11] [ warn] [output:calyptia:calyptia.0] agent registration failed
[2024/11/27 22:09:11] [debug] [out flush] cb_destroy coro_id=1
[2024/11/27 22:09:11] [debug] [task] task_id=0 reached retry-attempts limit 1/1
[2024/11/27 22:09:11] [error] [engine] chunk '57405-1732741739.505948000.flb' cannot be retried: task_id=0, input=fluentbit_metrics.0 > output=calyptia.0
[2024/11/27 22:09:11] [debug] [task] destroy task=0x60f000007c00 (task_id=0)

If set to false, registration terminates the initialisation.

[2024/11/27 22:10:12] [debug] [calyptia:calyptia.0] created event channels: read=29 write=30
[2024/11/27 22:10:12] [debug] [output:calyptia:calyptia.0] machine_id=73a2e3fa6b6844dd36fcd9970d39537980905f85972aa7f9c210080c55dcf3e2
[2024/11/27 22:10:12] [debug] [http_client] not using http_proxy for header
[2024/11/27 22:10:12] [error] [/Users/niedbalski/code/fluent/fluent-bit/src/flb_http_client.c:1376 errno=32] Broken pipe
[2024/11/27 22:10:12] [ warn] [output:calyptia:calyptia.0] http_do=-1
[2024/11/27 22:10:12] [ warn] [output:calyptia:calyptia.0] agent registration failed
[2024/11/27 22:10:12] [error] [output:calyptia:calyptia.0] agent registration failed and register_retry_on_flush=false
[2024/11/27 22:10:12] [error] [output] failed to initialize 'calyptia' plugin
[2024/11/27 22:10:12] [error] [engine] output initialization failed
[2024/11/27 22:10:12] [ info] [input] pausing fluentbit_metrics.0
[2024/11/27 22:10:12] [ info] [input] pausing storage_backlog.1

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[patrick-stephens]

macOS failures unrelated - due to cprofiles: fluent/cprofiles#7