F5 BIG-IP intergration pb

[JBG74]

I try using F5 telemetry streaming with Security onion (siem) based on ElasticSearch.
For do this i use F5 BIG-IP integration.
Unfortunately its not working.

F5 :
Version : BIG-IP 15.1.10.4 Build 0.0.5 Point Release 4
Module :
- f5-appsvcs-3.53.0-7.noarch
- f5-telemetry-1.37.0-1.noarch

Déclaration Telemetry Streaming :

Send with Postman , post to https://F5IP/mgmt/shared/telemetry/declare

{
    "class": "Telemetry",
    "controls": {
         "class":"Controls",
         "logLevel": "error",
         "debug" : true
     },
       "My_System": {
        "class": "Telemetry_System",
        "systemPoller": {
            "interval": 60
        }
    },
   "My_Listener": {
        "class": "Telemetry_Listener",
        "port": 6514,
        "trace": [
            {
                "type": "input"
            },
            {
                "type": "output"
            }
        ]
    },
    "My_Consumer": {
        "class": "Telemetry_Consumer",
        "type": "Generic_HTTP",
        "host": "SOC_IP",
        "allowSelfSignedCert": true,
        "protocol": "http",
        "trace": true,
        "path": "/",
        "method": "POST",
        "port": "9570",
        "headers": [
            {
                "name": "content-type",
                "value": "application/json"
            }
        ]
    }
}

Result 200 ok

AS3 logging sources Déclaration :
Send with Postman , post to https://F5IP>/mgmt/shared/appsvcs/declare

{
    "class": "ADC",
    "schemaVersion": "3.10.0",
    "remark": "Example depicting creation of BIG-IP module log profiles",
    "Common": {
    	"class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "telemetry_local_rule": {
                "remark": "Only required when TS is a local listener",
                "class": "iRule",
                "iRule": "when CLIENT_ACCEPTED {\n  node 127.0.0.1 6514\n}"
            },
            
            "telemetry_local": {
                "remark": "Only required when TS is a local listener",
                "class": "Service_TCP",
                "virtualAddresses": [
                    "255.255.255.254"
                ],
                "virtualPort": 6514,
                "iRules": [
                    "telemetry_local_rule"
                ]
            },
            
            "telemetry": {
                "class": "Pool",
                "members": [
                    {
                        "enable": true,
                        "serverAddresses": [
                            "255.255.255.254"
                        ],
                        "servicePort": 6514
                    }
                ],
                "monitors": [
                    {
                        "bigip": "/Common/tcp"
                    }
                ]
            },
            "telemetry_hsl": {
                "class": "Log_Destination",
                "type": "remote-high-speed-log",
                "protocol": "tcp",
                "pool": {
                    "use": "telemetry"
                }
            },
            "telemetry_formatted": {
                "class": "Log_Destination",
                "type": "splunk",
                "forwardTo": {
                    "use": "telemetry_hsl"
                }
            },
            "telemetry_publisher": {
                "class": "Log_Publisher",
                "destinations": [
                    {
                        "use": "telemetry_formatted"
                    }
                ]
            },
            "telemetry_traffic_log_profile": {
                "class": "Traffic_Log_Profile",
                "requestSettings": {
                    "requestEnabled": true,
                    "requestProtocol": "mds-tcp",
                    "requestPool": {
                        "use": "telemetry"
                    },
                    "requestTemplate": "event_source=\"request_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\""
                },
                "responseSettings": {
                    "responseEnabled": true,
                    "responseProtocol": "mds-tcp",
                    "responsePool": {
                        "use": "telemetry"
                    },
                    "responseTemplate": "event_source=\"response_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\",http_statcode=\"$HTTP_STATCODE\",http_status=\"$HTTP_STATUS\",response_ms=\"$RESPONSE_MSECS\""
                }
            }
        }
    }
}

Result 200 ok

I add the logging Source Sytem log with the GUI => System => Logs => Configuration =>Remote logging => Modify the system syslog configuration by adding a destination 127.0.0.1 remote-port 6514
I config 4 vhosts with (not in the same partitions)=> Request Logging Profile => telemetry_traffic_log_profile

I check the log in for check the result:

[root]# tail -f /var/log/restnoded/restnoded.log 

Wed, 12 Mar 2025 07:46:01 GMT - info: [telemetry.service.RESTAPIService] Request dd78e received: POST /shared/telemetry/declare
Wed, 12 Mar 2025 07:46:01 GMT - info: [telemetry] Global logLevel set to 'error'
Wed, 12 Mar 2025 07:46:01 GMT - severe: [telemetry.service.RuntimeConfigService.task] Task done!
Wed, 12 Mar 2025 07:46:06 GMT - finest: socket 2233 closed

With the trace at true for the listener and consumer i get the files below and à lot of events:

[root]# ls  /var/tmp/telemetry/Telemetry_*
/var/tmp/telemetry/Telemetry_Consumer.f5telemetry_default::My_Consumer  /var/tmp/telemetry/Telemetry_Listener.f5telemetry_default::My_Listener

If i try to check if my big-ip TS Event Listener is sending data to my consumer => Documentation https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/troubleshooting.html
Send with Postman , post to https://F5IP>/mgmt/shared/telemetry/eventListener/My_Listener

   Body:
    {
        "message": "my debugging message"
    }

Result => "code": 200,

log My_Listener :

    {
        "data": {
            "data": {
                "data": "{\"message\":\"my debugging message\"}",
                "telemetryEventCategory": "event",
                "originalRawData": "{\"message\":\"my debugging message\"}"
            },
            "type": "event",
            "sourceId": "cbf1e402-6f30-4aba-9526-3f005dba08d0",
            "destinationIds": [
                "663094f2-3480-4620-8bf4-b4f5cfca722f"
            ]
        },
        "timestamp": "2025-03-12T09:17:24.435Z"
    }

log My_Consummer :

{
    "data": {
        "allowSelfSignedCert": true,
        "body": {
            "data": "{\"message\":\"my debugging message\"}",
            "telemetryEventCategory": "event",
            "originalRawData": "{\"message\":\"my debugging message\"}"
        },
        "compressionType": "none",
        "host": "10.250.130.33",
        "fallbackHosts": [],
        "headers": {
            "content-type": "application/json"
        },
        "method": "POST",
        "port": 9570,
        "protocol": "http",
        "uri": "/"
    },
    "timestamp": "2025-03-12T09:17:24.435Z"
 }

In security onion (ELK 8.14.3) with distributed installation
the firewall is open for the source ip_F5 and the tcp port 9570
The module integration F5 Big IP (v1.17.0) is correctly ( i hope) configured :

• Collect F5 BIG-IP logs via HTTP Endpoint :
  Integration name => f5_bigip-1
  Namespace => default
  Enable Collect F5 BIG-IP logs via HTTP Endpoint
  Listen Address => 0.0.0.0
  F5 BIG-IP logs via HTTP Endpoint listen port: 9570
  url => /
  tags => f5_bigip-log , forwarded

[siem@soc ~]$ sudo iptables -nvL | grep 9570
[sudo] password for siem: 
 2842 3125K ACCEPT     tcp  --  *      *       IP_F5        0.0.0.0/0            tcp dpt:9570
    0     0 ACCEPT     tcp  --  *      *       IP_F5-HA        0.0.0.0/0            tcp dpt:9570
[siem@soc ~]$ sudo ss -tunlp | grep -E "9570"
tcp   LISTEN 0      4096               *:9570            *:*    users:(("agentbeat",pid=1903,fd=11))

in ssh on the F5 i force sending data to my nod => "originalRawData": "{\"message\":\"my debugging message\"}" and try to see if i recept him in my node

[siem@soc ~]$ sudo tcpdump -i ens192 tcp and host F5_IP and port 9570 -A | grep "debugging message"
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), snapshot length 262144 bytes
{"data":"{\"message\":\"my debugging message\"}","telemetryEventCategory":"event","originalRawData":"{\"message\":\"my debugging message\"}"}
70 packets captured
73 packets received by filter
0 packets dropped by kernel

When i try to go to Kibana - Discover - search with "F5" => Nothing
The F5 send the paquet , the node manager with elastic agent and the policy "so-grid-nodes_general" who carries the F5 integration receive the paquet .. but nothing come in Discover
i try to search the good log file F5 integration local or in the dockers , but i can't find it. It sounds like a bad configuration of the F5 module

If anyone has already encountered this problem or has an idea that would allow me to resolve the problem, I am interested.

[InfosecGoon]

What version of Security Onion are you using? There was an Elastic update in 2.4.130, I would suggest making sure you're on the latest build in case there's a bug in the integration causing issues.

[JBG74]

I did it this morning and incredible ... its works , i finally receive the logs in my discover.. thx for the good idea @InfosecGoon