syslog output plugin with JSON

[SarimMuqeet]

Hi,

I have recently been trying to send input memory metrics from fluent-bit to a remote syslog server.

When I set the syslog_message_key to something like Mem.total, I successfully start seeing "2006712" on my remote syslog server. However, when I try sending the entire JSON payload, my syslog server receives a message but the contents of it are empty.

Currently, I tried the Nest filter plugin to nest everything under a single Memstats key. When I do this, the standard output looks as expected:

[0] mem.local: [1694367396.133191260, {"Memstats"=>{"Mem.total"=>2006712, "Mem.used"=>1899256, "Mem.free"=>107456, "Swap.total"=>2744316, "Swap.used"=>737692, "Swap.free"=>2006624}}], so it does successfully add the Memstats key.

Essentially, I would like to send everything under Memstats. However, on my remote syslog server, I see blank logs appear. The same is noted when I run fluent bit on this remote system with syslog input (to receive from the sender). At first, I thought perhaps there may be an issue with the parsing or receiver configuration, but when testing with the logger command, I am successfully able to see the JSON appear on my syslog-ng server.

Observed Outputs:

//Using Mem.total as syslog_message_key:
<date> <time> <IP> 1 2023-09-10T19:14:01.132671Z - - - - - 2006712

//Using the Memstats with Nest Filter as syslog_message_key OR using no key (not setting anything):
<date> <time> <IP> 1 2023-09-10T19:14:33.133314Z - - - - -

//Using logger command to send a simple test nested JSON to the syslog-ng server:
<date> <time> <IP> 1 2023-09-10T14:01:33.708649-04:00 UbuntuVM vboxuser1 - - [timeQuality tzKnown="1" isSynced="0"] {"Memstats": {"test":"hi", "test2: "hello"}}

I have also tried using the record_modifier and the modify plugins but I get backend failure. I was wondering how to resolve this, and if there was a way to send a JSON message with multiple key-value pairs through the syslog output (and receive them through the input)?

Thank you!

[SarimMuqeet]

Config File:

[INPUT]
Name mem
Tag mem.local

[FILTER]
Name nest
Match *
Operation nest
Wildcard *
Nest_under Memstats

[OUTPUT]
Name syslog
Match *
Host test_ip
Port test_port
Mode udp
syslog_message_key Memstats

[OUTPUT]
Name stdout
Match *