devlink-exporter exports packets and bytes statistics for each packet
trap. In case per-{flow, trap} information is required, it is possible
to use ebpf_exporter with the trapagg.yaml
configuration file.
After installing ebpf_exporter, run:
$ ebpf_exporter --debug --config.file=trapagg.yaml
It will export metrics such as the below to Prometheus:
# HELP ebpf_exporter_flow_trap_packets Per-{flow, trap} statistics
# TYPE ebpf_exporter_flow_trap_packets counter
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.10",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 10
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.11",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 9
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.12",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 9
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.13",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 9
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.14",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 9
ebpf_exporter_flow_trap_packets{addr_proto="ipv4",daddrv4="198.51.100.15",daddrv6="::",dport="34512",ip_proto="17",is_encap="0",saddrv4="192.0.2.1",saddrv6="::",sport="12345",trap="unresolved_neigh"} 9
Grafana can then be used to visualize the information:
The per-{flow, trap} information is obtained by attaching an eBPF
program to a kprobe in the kernel. The program dissects incoming
packets and stores number of packets received in a LRU hash table keyed
by flow information and trap reason. The contents of the hash table are
then formatted by the ebpf_exporter and exported to Prometheus.
By default, the size of the LRU hash table is 1,024 entries. In case the table overflows, it will be flushed. It is possible to change the size of the hash table by modifying the configuration file.
With kernel 5.10 and above it is possible to use trapagg_tp.yaml
instead of trapagg.yaml. It attaches an eBPF program to the
devlink_trap_report raw tracepoint, which should provide a more stable
interface and better performance compared to the previously mentioned
kprobe.