diff --git a/README.md b/README.md index 9761a1efc..7b0a58450 100644 --- a/README.md +++ b/README.md @@ -6,117 +6,102 @@
-The ipfixprobe is a high-performance flow exporter. It creates bidirectional flows from packet input and exports them to output interface. The ipfixprobe support vide variety of flow extenstion for application layer protocol information. The flow extension can be turned on with process plugins. We support TLS, QUIC, HTTP, DNS and many more. Just check our [documentation](https://ipfixprobe.cesnet.cz). +The ipfixprobe is a high-performance, modular flow exporter that processes packets into bidirectional flows and exports them via a selected output plugin. +It supports a wide range of application-layer protocol parsers, including TLS, QUIC, HTTP, DNS, and many others. These protocol-specific extensions can be enabled via process plugins in the configuration. + +Need more details? Check out our [documentation](https://ipfixprobe.cesnet.cz) for a full list of supported protocols and usage examples. [![](https://img.shields.io/badge/license-BSD-blue.svg)](https://github.com/CESNET/ipfixprobe/blob/master/LICENSE) [![](https://img.shields.io/badge/docs-ipfixprobe-blue.svg)](https://ipfixprobe.cesnet.cz) ![Coverity Scan](https://img.shields.io/coverity/scan/22112) ![GitHub top language](https://img.shields.io/github/languages/top/CESNET/ipfixprobe) +## ✨ Key Features -## πŸ› οΈ Installation -The RPM packages for RHEL based distributions can be downloaded from our [copr repository](https://copr.fedorainfracloud.org/coprs/g/CESNET/ipfixprobe/package/ipfixprobe/). Or just simply run: +- Modular input–parser–output architecture +- High-speed processing (DPDK, multi-threaded, NUMA-aware) +- Built-in protocol parsers: TLS, QUIC, HTTP, DNS, … +- Bidirectional flow (biflow) support +- Real-time telemetry and statistics -``` -dnf install -y dnf-plugins-core &&dnf copr enable @CESNET/ipfixprobe -dnf install ipfixprobe -``` - -## πŸ”§ Parameters -### Module specific parameters -- `-i ARGS` Activate input plugin (-h input for help) -- `-s ARGS` Activate storage plugin (-h storage for help) -- `-o ARGS` Activate output plugin (-h output for help) -- `-p ARGS` Activate processing plugin (-h process for help) -- `-q SIZE` Size of queue between input and storage plugins -- `-b SIZE` Size of input queue packet block -- `-Q SIZE` Size of queue between storage and output plugins -- `-B SIZE` Size of packet buffer -- `-f NUM` Export max flows per second -- `-c SIZE` Quit after number of packets are processed on each interface -- `-P FILE` Create pid file -- `-t PATH` Mount point of AppFs telemetry directory -- `-d` Run as a standalone process -- `-h [PLUGIN]` Print help text. Supported help for input, storage, output and process plugins -- `-V` Show version and exit +## πŸ“¦ Installation -### ❓ Help -Printing general help is done using the `-h` parameter. To print help for specific plugins, `-h` with parameter is used. -This parameter accepts `input`, `storage`, `process`, `output` or name of a plugin (or path to a .so file with plugin). +If you are running a RHEL system or one of its derivatives (e.g. Oracle Linux, Rocky Linux, CentOS Stream), the easiest way to install +ipfixprobe is from our [copr repository](https://copr.fedorainfracloud.org/coprs/g/CESNET/ipfixprobe/). -## πŸ“– Example -Here are the examples of various plugins usage: ```bash -# Capture from wlp2s0 interface using raw sockets, print flows to console -./ipfixprobe -i 'raw;ifc=wlp2s0' -o 'text' - -# Capture from wlp2s0 interface and scale packet processing using 2 instances of plugins, send flow to ifpfix collector using UDP -./ipfixprobe -i 'raw;ifc=wlp2s0;f' -i 'raw;ifc=wlp2s0;f' -o 'ipfix;u;host=collector.example.com;port=4739' +$ dnf install dnf-plugins-core # Extra step necessary on some systems +$ dnf copr enable @CESNET/ipfixprobe +$ dnf install ipfixprobe +``` -# Capture from a COMBO card using ndp plugin, sends ipfix data to 127.0.0.1:4739 using TCP by default -./ipfixprobe -i 'ndp;dev=/dev/nfb0:0' -i 'ndp;dev=/dev/nfb0:1' -i 'ndp;dev=/dev/nfb0:2' +This installs the main `ipfixprobe` binary along with core functionality. -# Capture from eth0 interface using pcap plugin, split biflows into flows and prints them to console without mac addresses, telemetry data are exposed via the appFs library in /var/run/ipfixprobe directory -./ipfixprobe -i 'pcap;ifc=eth0' -s 'cache;split' -o 'text;m' -t /var/run/ipfixprobe +#### πŸ—ƒοΈ Available Packages -# Read packets from pcap file, enable 4 processing plugins, sends L7 HTTP extended biflows to unirec interface named `http` and data from 3 other plugins to the `stats` interface -./ipfixprobe -i 'pcap;file=pcaps/http.pcap' -p http -p pstats -p idpcontent -p phists -o 'unirec;i=u:http:timeout=WAIT,u:stats:timeout=WAIT;p=http,(pstats,phists,idpcontent)' +The Copr repository provides modular RPM packages, so you can install only what you need. +The following packages are available and can be installed individually as needed: -# Read packets using DPDK input interface and 1 DPDK queue, enable plugins for basic statistics, http and tls, output to IPFIX on a local machine -# DPDK EAL parameters are passed in `e, eal` parameters -# DPDK plugin configuration has to be specified in the first input interface. -# The following `dpdk` interfaces are given without parameters; their configuration is inherited from the first one. -# Example for the queue of 3 DPDK input plugins (q=3): -`./ipfixprobe -i "dpdk;p=0;q=3;e=-c 0x1 -a <[domain:]bus:devid.func>" -i dpdk -i dpdk -p http "-p" bstats -p tls -o "ipfix;h=127.0.0.1"` +| Package Name | Description | +|---------------------------------------------|-----------------------------------------------------------------------------| +| `ipfixprobe` | Core binary with common process/output plugins. | +| `ipfixprobe-msec` | Core binary with common process/output plugins. Uses millisecond timestamps (compatible with Flowmon collector) | +| `ipfixprobe-input-pcap` | Input plugin for PCAP files and live capture | +| `ipfixprobe-input-dpdk` | High-speed input plugin using DPDK | +| `ipfixprobe-input-nfb` | Input plugin for CESNET NFB/NDP cards | +| `ipfixprobe-process-experimental` | Extra (possibly unstable) process plugins | -# Same example for the multiport read from ports 0 and 1, note comma separated ports: -`./ipfixprobe -i "dpdk;p=0,1;q=3;e=-c 0x1 -a <[domain:]bus:devid.func>" -i dpdk -i dpdk -p http "-p" bstats -p tls -o "ipfix;h=127.0.0.1"` +For other systems, follow the build instructions below. +## πŸ› οΈ Build -# Read packets using DPDK input interface as secondary process with shared memory (DPDK rings) - in this case, 4 DPDK rings are used -`./ipfixprobe -i 'dpdk-ring;r=rx_ipfixprobe_0;e= --proc-type=secondary' -i 'dpdk-ring;r=rx_ipfixprobe_1' -i 'dpdk-ring;r=rx_ipfixprobe_2' -i 'dpdk-ring;r=rx_ipfixprobe_3' -o 'text'` -``` +You can build ipfixprobe from source using standard CMake. +This lets you customize the build by enabling optional plugins and features as needed. -## Build +**Note:** Some plugins may require additional dependencies beyond the basic requirements. -### πŸ“¦ Requirements +### RHEL/CentOS: -- `libatomic` -- [telemetry](https://github.com/CESNET/telemetry) (**required**) Installable from the [COPR repository](https://copr.fedorainfracloud.org/coprs/g/CESNET/NEMEA-stable/package/telemetry/) or buildable from source -- Linux kernel version **β‰₯ 3.19** -- [libpcap](http://www.tcpdump.org/) β€” required for PCAP input plugin (`-DENABLE_INPUT_PCAP`) -- `netcope-common` β€” required for NDP input plugin with [COMBO cards](https://www.liberouter.org/technologies/cards/) (`-DENABLE_INPUT_NFB`) -- `libunwind-devel` -- [NEMEA](http://github.com/CESNET/Nemea-Framework) β€” required for UniRec output plugin (`-DENABLE_NEMEA`, `-DENABLE_OUTPUT_UNIREC`) -- [DPDK](http://core.dpdk.org/download/) β€” required for DPDK input plugin (`-DENABLE_INPUT_DPDK`) +#### 🧰 Requirements -> For most conventional monitoring use-cases (not requiring high-speed packet libraries like DPDK or NDP), you can install required dependencies using the following commands: +```bash +$ dnf install epel-release git make cmake gcc-c++ rpm-build +$ dnf install libunwind-devel lz4-devel openssl-devel fuse3-devel -#### 🐧 RHEL9-based distributions +# for RHEL 8/9 +$ dnf install gcc-toolset-14-libatomic-devel -```bash -sudo yum-config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL9/codeready/builder/x86_64/ -sudo dnf copr enable @CESNET/NEMEA-stable -sudo dnf install -y git wget curl net-tools gcc gcc-c++ \ - libtool lz4-devel rpm-build fuse3-devel make cmake rpm \ - libatomic libunwind-devel openssl-devel pkgconf-pkg-config \ - telemetry gcc-toolset-14-libatomic-devel +# for RHEL 10+ +$ dnf install libatomic ``` ---- +### Debian/Ubuntu: +#### 🧰 Requirements -### βš™οΈ Project build with CMake +TODO -This project uses the standard CMake build system. Example setup: +#### 🧱 Build steps ```bash -git clone --recurse-submodules https://github.com/CESNET/ipfixprobe +git clone https://github.com/CESNET/ipfixprobe.git cd ipfixprobe mkdir build && cd build -cmake -DCMAKE_INSTALL_PREFIX=/usr .. +cmake .. +make -j$(nproc) +# make install ``` -#### πŸ”§ Notable CMake Build Options +#### βš™οΈ Optional build flags +You can enable or disable optional plugins and features via CMake flags: + +| Flag | Default | Description | +| ---------------------------------- | ------- | ---------------------------------------------------------------- | +| `-DENABLE_MILLISECONDS_TIMESTAMP=ON` | OFF | Use millisecond precision timestamps (for Flowmon compatibility) | +| `-DENABLE_INPUT_PCAP=ON` | OFF | Enable PCAP input plugin (live & file) (requires `libpcap`) | +| `-DENABLE_INPUT_DPDK=ON` | OFF | Enable high-speed DPDK input plugin (requires `dpdk-devel`) | +| `-DENABLE_INPUT_NFB=ON` | OFF | Enable input plugin for CESNET NFB/NDP cards (requires `netcope-common`) | +| `-DENABLE_PROCESS_EXPERIMENTAL=ON` | OFF | Enable experimental process plugins | +| `-DENABLE_NEMEA=ON` | OFF | Enable support for NEMEA modules (requires `nemea-framework-devel` ) | Run the command to view all available build options: @@ -124,187 +109,245 @@ Run the command to view all available build options: cmake -LAH ``` -The most notable options are: - -| Option | Description | -|--------------------------------|--------------------------------------------------------------------------| -| `ENABLE_MILISECONDS_TIMESTAMPS`| Enable millisecond timestamp precision | -| `ENABLE_NEMEA` | Enable support for NEMEA modules | -| `ENABLE_RPMBUILD` | Enable building of RPM packages (enabled by default) | -| `ENABLE_TESTS` | Enable building of unit and integration tests | -| `ENABLE_INPUT_PCAP` | Build PCAP input plugin (requires `libpcap`) | -| `ENABLE_INPUT_NFB` | Build NFB input plugin (requires `netcope-common`) | -| `ENABLE_INPUT_DPDK` | Build DPDK input plugin (requires `dpdk`) | - - ---- - -#### πŸ› οΈ Build from Source - -Once the CMake project is configured, build the project using: +#### Example +To build with DPDK and PCAP input support, and install to /usr: -```bash -make -j +```cmake +cmake .. \ + -DCMAKE_INSTALL_PREFIX=/usr \ + -DENABLE_INPUT_PCAP=ON \ + -DENABLE_INPUT_DPDK=ON ``` -The resulting binary will be located at: -```bash -ipfixprobe/build/src/core/ipfixprobe -``` +## 🧩 Available Plugins -To install the binary system-wide: +### Input Plugins +List of input plugins with estimated performance and configuration complexity. -```bash -make install -``` +| Plugin | Max Throughput | Usage Complexity | Description | +|---------------|----------------|------------------|-------------------------------------------| +| [`pcap_live`](./src/plugins/input/pcap/README.md#pcap-live-input-plugin) | ~1 Gbps | Easy | captures packets from a live network interface | +| [`pcap_file`](./src/plugins/input/pcap/README.md#pcap-file-input-plugin) | ~1 Gbps | Easy | reads packets from an offline PCAP file | +| [`raw`](./src/plugins/input/raw/README.md) | ~1 Gbps | Easy | captures packets using a raw socket | +| [`ndp`](./src/plugins/input/nfb/README.md) | 400 Gbps | Medium | uses CESNET NFB/NDP hardware for packet input | +| [`dpdk`](./src/plugins/input/dpdk/README.md#dpdk-input-plugin) | 400 Gbps | Complex | receives packets via high-performance DPDK | +| [`dpdk-ring`](./src/plugins/input/dpdk/README.md) | 400 Gbps | Complex | receives packets from a shared DPDK memory ring | --- -#### πŸ“¦ Build RPM Packages +### Process Plugins + +These plugins extract protocol-specific or behavioral information from packets and enrich flow records with metadata. + +| Plugin | Description | +|---------------|--------------------------------------------------------------| +| [`basic`](./src/plugins/process/basic/README.md) | extracts basic L3/L4 flow fields (IPs, ports, protocol) | +| [`icmp`](./src/plugins/process/icmp/README.md) | extracts ICMP type/code and related metadata | +| [`http`](./src/plugins/process/http/README.md) | extracts HTTP methods, hosts, URIs, status codes | +| [`tls`](./src/plugins/process/tls/README.md) | extracts TLS handshake info (SNI, version, JA3, etc.) | +| [`ovpn`](./src/plugins/process/ovpn/README.md) | extracts metadata from OpenVPN tunnels | +| [`wg`](./src/plugins/process/wg/README.md) | parses WireGuard handshake and endpoint metadata | +| [`quic`](./src/plugins/process/quic/README.md) | parses QUIC protocol including SNI, versions, ALPN | +| [`basicplus`](./src/plugins/process/basicplus/README.md) | adds common L3/L4 flow fields (e.g., ports, IPs, TCP flags) | +| [`bstats`](./src/plugins/process/bstats/README.md) | basic flow statistics (packet/byte counters, duration, ...) | +| [`dns`](./src/plugins/process/dns/README.md) | extracts DNS queries, responses, and domains | +| [`dnssd`](./src/plugins/process/dnssd/README.md) | parses DNS Service Discovery (mDNS) traffic | +| [`flowHash`](./src/plugins/process/flowHash/README.md) | extracts a flow hash | +| [`idpContent`](./src/plugins/process/idpContent/README.md) | parses IDP content in flows | +| [`mpls`](./src/plugins/process/mpls/README.md) | extracts MPLS labels and encapsulation metadata | +| [`mqtt`](./src/plugins/process/mqtt/README.md) | parses MQTT protocol traffic (IoT messaging) | +| [`netbios`](./src/plugins/process/netbios/README.md) | extracts NetBIOS session and name service info | +| [`nettisa`](./src/plugins/process/nettisa/README.md) | parses NETTISA related metadata (experimental) | +| [`ntp`](./src/plugins/process/ntp/README.md) | extracts NTP timestamps and server info | +| [`osquery`](./src/plugins/process/osquery/README.md) | parses osquery-generated data streams | +| [`passiveDns`](./src/plugins/process/passiveDns/README.md) | generates passive DNS entries from observed DNS traffic | +| [`phists`](./src/plugins/process/phists/README.md) | parses phishing-related signatures (heuristic) | +| [`pstats`](./src/plugins/process/pstats/README.md) | advanced packet statistics (e.g., inter-packet gaps) | +| [`rtsp`](./src/plugins/process/rtsp/README.md) | extracts RTSP stream metadata | +| [`sip`](./src/plugins/process/sip/README.md) | parses SIP call setup, headers, and codecs | +| [`smtp`](./src/plugins/process/smtp/README.md) | extracts SMTP envelope data (from, to, subject, etc.) | +| [`ssaDetector`](./src/plugins/process/ssaDetector/README.md) | performs simple anomaly detection based on traffic patterns | +| [`ssdp`](./src/plugins/process/ssdp/README.md) | parses SSDP (UPnP discovery) protocol | +| [`vlan`](./src/plugins/process/vlan/README.md) | extracts VLAN IDs and QinQ encapsulation | -RPM packages are created automatically based on the enabled CMake options. +--- +### Output Plugins -If the project is configured with `ENABLE_RPMBUILD` (enabled by default), you can build RPM packages using: +These plugins export flow records to various formats and external systems. -```bash -make -j rpm -``` +| Plugin | Description | +|---------------|-----------------------------------------------------------------------------| +| [`ipfix`](./src/plugins/output/ipfix/README.md) | exports flow records in IPFIX format to a remote collector (UDP/TCP) | +| [`text`](./src/plugins/output/text/README.md) | writes flow records in human-readable text to a file or stdout | +| [`unirec`](./src/plugins/output/unirec/README.md) | exports flow records using the UniRec format for NEMEA/TRAP ecosystem | -The resulting RPM files will be located in: +--- -``` -ipfixprobe/build/pkg/rpm/rpmbuild/ -``` +## πŸ“˜ Usage -## πŸ“ˆ Telemetry +The `ipfixprobe` processing pipeline can be launched directly or via the `ipfixprobed` wrapper β€” a lightweight runtime designed to simplify plugin configuration using YAML files. +This method is preferred for production deployments due to its clarity and flexibility. -`ipfixprobe` exports statistics and other diagnostic information through a telemetry interface based on appFs library, which leverages the fuse3 library (filesystem in userspace) to allow telemetry data to be accessed and manipulated -through standard filesystem operations. +### βœ… Recommended (YAML-based configuration) -``` -/var/run/ipfixprobe/ -β”œβ”€β”€ output -β”‚Β Β  └── ipxRing -β”‚Β Β  └── stats -└── pipeline - └── queues - β”œβ”€β”€ 0 - β”‚Β Β  β”œβ”€β”€ cache-stats - β”‚Β Β  β”œβ”€β”€ fragmentation-cache-stats - β”‚Β Β  β”œβ”€β”€ input-stats - β”‚Β Β  └── parser-stats - β”œβ”€β”€ 1 - ... +```bash +/usr/bin/ipfixprobed [LIBRARY_PATH] ``` -From telemetry stats you can read: +- `` – Name of the YAML configuration file (without the .conf extension). +The full path must be `/etc/ipfixprobe/.conf`. -**Output Directory:** +- `[LIBRARY_PATH]` – (Optional) Path to the ipfixprobe plugin library directory. +If not provided, the default `/usr/lib64/ipfixprobe` is used. -In the output directory, you can monitor the capacity and current usage of the ipxRing. This helps determine whether the output plugin can export flows quickly enough or if there are bottlenecks caused by insufficient ring capacity. - -***Example: ipxRing/stats*** -``` -count: 204 -size: 16536 -usage: 1.23 (%) +#### Example: +```bash +/usr/bin/ipfixprobed example ``` -**Pipeline Directory:** - -The pipeline directory provides statistics for all worker queues. Each queue is identified by its ID (e.g., 0, 1) and includes the following files: -- cache-stats: Provides detailed metrics about flow cache usage and exported flow statistics. +This will run `ipfixprobe` using the YAML configuration from `/etc/ipfixprobe/example.conf`. - ***Example:*** +πŸ‘‰ See the full YAML configuration reference here: https://github.com/CESNET/ipfixprobe/blob/master/init/link0.conf.example - ``` - FlowCacheUsage: 3.81 (%) - FlowEndReason:ActiveTimeout: 34666654 - FlowEndReason:Collision: 4272143 - FlowEndReason:EndOfFlow: 486129363 - FlowEndReason:Forced: 58905 - FlowEndReason:InactiveTimeout: 2169352600 - FlowRecordStats:11-20packets: 178735501 - FlowRecordStats:1packet: 1824500140 - FlowRecordStats:2-5packets: 376268956 - FlowRecordStats:21-50packets: 87971544 - FlowRecordStats:51-plusPackets: 55424342 - FlowRecordStats:6-10packets: 171579322 - FlowsInCache: 39986 - TotalExportedFlows: 2694479805 - ``` +### ⚠️ Legacy usage (CLI parameters) -- fragmentation-cache-stats: Provides metrics related to packet fragmentation. +You may also run the processing pipeline using the `ipfixprobe` binary directly, with CLI parameters. +This method is not recommended for production use, as it lacks flexibility and clarity. - ***Example:*** +```bash +/usr/bin/ipfixprobe [OPTIONS] +``` - ``` - firstFragments: 163634416 - fragmentedPackets: 395736897 - fragmentedTraffic: 0.13 (%) - notFoundFragments: 85585913 - totalPackets: 314829930486 - ``` +#### Available options: -- input-stats: Provides metrics on the data received by by the queue. +- `-i ARGS` Activate input plugin (`-h input` for help) +- `-s ARGS` Activate storage plugin (`-h storage` for help) +- `-o ARGS` Activate output plugin (`-h output` for help) +- `-p ARGS` Activate processing plugin (`-h process` for help) +- `-q SIZE` Size of queue between input and storage plugins +- `-b SIZE` Size of input queue packet block +- `-Q SIZE` Size of queue between storage and output plugins +- `-B SIZE` Size of packet buffer +- `-f NUM` Export max flows per second +- `-c SIZE` Quit after number of packets are processed on each interface +- `-P FILE` Create a PID file +- `-t PATH` Mount point of AppFs telemetry directory +- `-d` Run as a standalone process +- `-h [PLUGIN]` Print help text. Supported help for input, storage, output and process plugins +- `-V` Show version and exit - ***Example:*** - ``` - received_bytes: 388582006601530 - received_packets: 314788702409 - ``` +## πŸ“– Examples +Below are practical examples showcasing common plugin configurations using both CLI and YAML formats. +#### **πŸ”Ή Basic Interface Capture** +Capture network traffic from the `wlp2s0` interface using a raw socket. Flow records are printed in plain-text format to the console. -- parser-stats: Provides detailed information about the types of packets processed by the parser. +πŸ§ͺ Command-line usage: +```bash +/usr/bin/ipfixprobe -i 'raw;ifc=wlp2s0' -o 'text' +``` - ***Example:*** - ``` - ipv4_packets: 193213761481 - ipv6_packets: 121566104060 - mpls_packets: 0 - pppoe_packets: 0 - seen_packets: 314791928764 - tcp_packets: 301552123188 - trill_packets: 0 - udp_packets: 12783568334 - unknown_packets: 11601117 - vlan_packets: 31477986554 - ``` +πŸ“„ Equivalent YAML configuration: +```yaml +input_plugin: + raw: + interface: 'wlp2s0' +output_plugin: + text: {} +``` -## πŸ“₯ Input / Output of the flow exporter +---- -The availability of input and output types depends on the installed packages or enabled build options (see section of README Project Setup with CMake). -By default, installing the `ipfixprobe` package enables standard raw socket output. +#### **πŸ”Ή PCAP File β†’ IPFIX Collector** +Process packets from a `.pcap` file and export flows via IPFIX over UDP -To enable additional input capabilities, install the corresponding input plugin packages: +πŸ§ͺ Command-line usage: +```bash +/usr/bin/ipfixprobe -i 'pcap;file=/data/capture.pcap' -o 'ipfix;udp;host=collector.example.com;port=4739' +``` -- `ipfixprobe-input-dpdk` – enables DPDK input support -- `ipfixprobe-input-nfb` – enables NFB input support -- `ipfixprobe-input-pcap` – enables libpcap input support +πŸ“„ Equivalent YAML configuration: +```yaml +input_plugin: + pcap_file: + file: '/data/capture.pcap' +output_plugin: + ipfix: + collector: + host: collector.example.com + port: 4739 + protocol: + udp: {} +``` -For more information, visit the [input plugin documentation](https://ipfixprobe.cesnet.cz/en/plugins) or run `ipfixprobe -h input` for more information and complete list of input plugins and their parameters. +---- +#### **πŸ”ΉLive Capture with Cache Configuration and Telemetry** +Capture from `eth0` interface using libpcap, split biflows into uniflows and use active/inactive timeouts, print flows to console. +Telemetry data are exposed via the appFs library in `/var/run/ipfixprobe` directory. -### πŸ“€ Output +πŸ§ͺ Command-line usage: +```bash +/usr/bin/ipfixprobe -i 'pcap;ifc=eth0' -s 'cache;split;active=300;inactive=60' -o 'text' -t /var/run/ipfixprobe +``` -Similarly as in input, the output availability also depends on the installed packages. -By default, installed the `ipfixprobe` package enables standard `ipfix` and `text` output. +πŸ“„ Equivalent YAML configuration: +```yaml +input_plugin: + pcap_live: + interface: 'eth0' + +storage: + cache: {} + timeouts: + active: 300 + inactive: 60 + split_biflow: true + +output_plugin: + text: {} + +telemetry: + appfs: + enabled: true + mount_point: /var/run/ipfixprobe +``` -To add [NEMEA system](https://nemea.liberouter.org) output capability, you should install `ipfixprobe-nemea` instead of ipfixprobe +---- -See `ipfixprobe -h output` for more information and complete list of output plugins and their parameters. +#### πŸ”Ή High-speed DPDK Capture with HTTP, TLS, and QUIC Processing +Capture packets using DPDK from port 0 with 2 queues bound to a specific PCI device (`0000:17:00.0`), enable HTTP, TLS, and QUIC process plugins, and export flows via IPFIX to a local collector at 127.0.0.1. -#### LZ4 compression: -ipfix plugin supports LZ4 compression algorithm over tcp. See plugin's help for more information. +πŸ§ͺ Command-line usage: +```bash +/usr/bin/ipfixprobe -i "dpdk;p=0;q=2;e=-a 0000:17:00.0" -i dpdk -p http -p tls -p quic -o "ipfix;h=127.0.0.1" +``` +πŸ“„ Equivalent YAML configuration: +```yaml +input_plugin: + dpdk: + allowed_nics: "0000:17:00.0" + rx_queues: 2 + +process_plugins: + - http + - tls + - quic + +output_plugin: + ipfix: + collector: + host: '127.0.0.1' + port: 4739 + protocol: + udp: {} +``` -## ⚠️ Possible issues -### Flows are not send to output interface when reading small pcap file (NEMEA output) +## πŸ“Š Telemetry -Turn off message buffering using `buffer=off` option and set `timeout=WAIT` on output interfaces. +## πŸ§ͺ Testing & Validation -``` -./ipfixprobe -i 'pcap;file=traffic.pcap' -o 'unirec;i=u:out:timeout=WAIT:buffer=off' -``` +## 🧰 FAQ diff --git a/src/plugins/input/dpdk/README.md b/src/plugins/input/dpdk/README.md index e69de29bb..b565710fd 100644 --- a/src/plugins/input/dpdk/README.md +++ b/src/plugins/input/dpdk/README.md @@ -0,0 +1,258 @@ +# DPDK (input plugin) + +DPDK (Data Plane Development Kit) is used for high-performance packet processing. It enables +direct access to network interfaces, bypassing the kernel, and is suitable for use in environments +requiring high throughput, low latency, and high packet processing rates. + +## Example configuration + +```yaml +input_plugin: + dpdk: + allowed_nics: "0000:ca:00.0" + ### Optional parameters + burst_size: 64 + mempool_size: 8192 + rx_queues: 8 + workers_cpu_list: [] + eal_opts: null + mtu: 1518 +``` + +## Parameters + +**Mandatory parameters:** + +|Parameter | Description | +|---|---| +|__allowed_nics__|List of allowed NICs in PCI address format `0000:XX:YY.Z` separated with `,` | + +**Optional parameters:** +|Parameter | Default | Description | +|---|---|---| +|__burst_size__ | 64 | Number of packets processed in each burst cycle. Affects batch processing efficiency. | +|__mempool_size__ | 8192 | Size of the memory pool used for buffering incoming packets. Must be a power of 2.| +|__rx_queues__ | 1| Number of RX queues workers. Increasing this can help distribute load across multiple CPU cores. | +|__workers_cpu_list__| [] (autofill) | List of CPU cores assigned to RX queues (must match number of rx_queues) | +|__eal_opts__ | null | Extra options to be passed to the DPDK EAL (Environment Abstraction Layer). Can be used for fine-tuning DPDK behavior.| +|__mtu__ | 1518 | Maximum Transmission Unit size for the interface. Defines the maximum packet size that can be received.| + +## How to use + +To use the DPDK input plugin, you must ensure that your system is properly configured for DPDK operation. This includes the following steps: + +### 1. Install DPDK Tools + +To begin with, you will need to install DPDK and its associated tools. Follow the installation instructions for your operating system: + +- **On RHEL/CentOS**: +```sh +dnf install dpdk-tools +``` + +- **On Debian/Ubuntu**: +```sh +apt-get install dpdk +``` + +### 2. Identify the PCI Address of the Network Interface +DPDK operates directly with network interfaces identified by their PCI addresses, not by traditional interface names like `eth0` or `ens3`. + +A PCI address looks like this: + +``` +0000:ca:00.0 +``` + +This format includes: +- Domain: `0000` – typically 0000 on most systems +- Bus: `ca` +- Device: `00` +- Function: `0` + +Each network interface has a unique PCI address, and this is how DPDK identifies which interface to bind and use. + +πŸ” How to find the PCI address of a network interface +The recommended way to identify the PCI address is using the DPDK helper tool, dpdk-devbind.py. This tool lists all the NICs in the system with their PCI addresses and shows which drivers are currently bound to them. + +``` +dpdk-devbind.py --status +``` + +This shows the PCI addresses of all detected NICs along with the drivers they're bound to. If the NIC is not already bound to a DPDK-compatible driver (e.g., `vfio-pci`), you can bind it using this tool. + +### 3. Identify the Numa node of the Network interface + +DPDK operates efficiently on systems with multiple **NUMA (Non-Uniform Memory Access)** nodes, and it is essential to know which NUMA node a network interface belongs to, as this affects memory locality and performance. + +Each physical device (like a network interface card) is associated with a specific NUMA node, which influences the memory accesses and CPU affinity during packet processing. You can use the NUMA node information to optimize the performance of your application by binding the NIC to a NUMA node that is also closest to the processing CPU cores. + +**πŸ” How to find the NUMA node of a network interface?** + +You can identify the NUMA node to which a specific network interface is attached by directly checking the /sys filesystem. + +Using the `/sys/bus/pci/devices/{pci_address}/numa_node` path: + +The most direct method to get the NUMA node for a network interface is by querying the numa_node file in the /sys directory. + +First, you need to know the PCI address of your network interface. Once you have the PCI address (e.g., 0000:ca:00.0), you can check the NUMA node for that interface by reading the numa_node file: + +``` +cat /sys/bus/pci/devices/0000:ca:00.0/numa_node +``` + +This will output the NUMA node number where the NIC is located. For example: +``` +0 +``` + +This indicates that the NIC is attached to NUMA node 0. + + +If the output is -1, it means that the device does not have an associated NUMA node or the system does not have NUMA support. + +### 4. Allocate Hugepages + +DPDK requires hugepages for optimal performance, as they provide large, contiguous memory blocks that reduce overhead and improve data throughput. Hugepages are critical for performance in high-speed networking environments, such as those used by DPDK, where low latency and high throughput are required. + +**πŸ› οΈ Configuring Hugepages via Kernel Parameters [Recommended]** + +You can configure hugepages directly at the kernel level using the grubby tool. This approach is recommended if you want to make the hugepages configuration persistent across system reboots. + +To configure hugepages via grubby, use the following command: + +``` +grubby --update-kernel ALL --args "default_hugepagesz=1GB hugepagesz=1G hugepages=4" +``` +This command will: + +- Set the default hugepage size to 1GB (default_hugepagesz=1GB). +- Set the hugepage size to 1GB (hugepagesz=1G). +- Allocate 4 hugepages (hugepages=4). + +**⚠️ Important Note:** When using this method, it is not possible to specify a particular NUMA node. The hugepages will be distributed evenly across all available NUMA nodes on the system. This means the memory for hugepages will be shared equally among the NUMA nodes without considering any specific NUMA affinity for your application. + +After running the command, you need to reboot the system for the changes to take effect. + +--- +**πŸ“Œ Allocating Hugepages Using dpdk-hugepages.py** + +Alternatively, you can use the dpdk-hugepages.py script to allocate hugepages at runtime. This method allows you to allocate hugepages dynamically and specify NUMA nodes. + +To allocate hugepages using dpdk-hugepages.py, you can run the following command: + +``` +dpdk-hugepages.py -p 1G --setup 2G --node 0 +``` + +This command allocates 2GB of hugepages with a 1GB page size on NUMA node 0. You can adjust these values based on your system's memory requirements and the NUMA node of your NIC. + +If you require more hugepages, you can increase the amount by modifying the --setup parameter, as follows: + +``` +dpdk-hugepages.py -p 1G --setup 4G --node 0 +``` + +This will allocate 4GB of hugepages with a 1GB page size on NUMA node 0. + +**πŸ“Œ Recommended Hugepages Configuration for High-Speed Links (100G, 200G, 400G)** + +For high-speed links such as 100G, 200G, or 400G, it is crucial to allocate enough hugepages to handle the massive packet processing and memory requirements. The following values are recommended based on typical usage for each type of link: + +**100G link:** +For 100G links, you should allocate at least 4GB of hugepages with a 1GB page size. This is generally sufficient for moderate traffic and packet processing needs. + +Recommended command: +``` +dpdk-hugepages.py -p 1G --setup 4G --node 0 +``` +--- +***200G link:*** +For 200G links, you will likely need 8GB of hugepages with a 1GB page size, as the traffic and memory bandwidth requirements will be higher. + +Recommended command: +``` +dpdk-hugepages.py -p 1G --setup 8G --node 0 +``` + +--- + +***400G link:*** +For 400G links, a significant increase in hugepages is required. You should allocate 16GB of hugepages with a 1GB page size to handle the higher throughput and memory demands. + +Recommended command: +``` +dpdk-hugepages.py -p 1G --setup 16G --node 0 +``` + +**πŸ–₯️ Verify the allocated Hugepages** + +To verify that the hugepages were allocated successfully, you can use the following command: + +``` dpdk-hugepages -s ``` + +This command shows the status of the hugepages and confirms how much memory has been allocated. + +--- + +### 5. Configure the DPDK Driver + +TODO Mellanox, broadcom, intel +TODO Lukas + +### 6. Isolate CPUs (optionally) + +Isolating specific CPUs can enhance the performance of DPDK applications by dedicating certain processors to networking tasks, reducing interference from other system processes. This isolation minimizes context switching and ensures that the CPUs are dedicated to packet processing. + +**πŸ› οΈ How to Isolate CPUs?** + +To isolate CPUs on a system, you can use the tuned package and adjust kernel parameters. The steps below demonstrate how to set up CPU isolation using tuned profiles and kernel boot parameters. These steps apply to both Intel and AMD systems, with specific configuration examples for both. + +1. **Install the necessary package** + + First, install the tuned-profiles-cpu-partitioning package, which contains CPU isolation profiles: + + ```bash + dnf install tuned-profiles-cpu-partitioning + ``` + +2. **Enable IOMMU and Isolate CPUs Using GRUB** + + **For Intel Systems**: Enable Intel IOMMU for direct device access in DPDK. Update your GRUB configuration by adding the following arguments for Intel processors: + ``` + grubby --update-kernel ALL --args "iommu=pt intel_iommu=on" + ``` + + **For AMD Systems**: Enable AMD IOMMU for direct device access in DPDK. Update your GRUB configuration by adding the following arguments for AMD processors: + + ``` + grubby --update-kernel ALL --args "iommu=pt amd_iommu=on" + ``` + +3. Isolate CPUs for DPDK + + Once IOMMU is enabled, you can isolate specific CPUs for DPDK using the isolcpus parameter. This ensures that only the isolated CPUs are used for networking tasks. + + To isolate CPUs 2-19 and 22-39 on an Intel system, use the following command: + +``` +grubby --update-kernel ALL --args "isolcpus=2-19,22-39" +``` + + +### 4. Validate with dpdk-testpmd + +TODO + +## FAQ + +|Q: | How many `rx_queues` should I configure? | +|---|---| +|A: | TODO | + +|Q: | ??? | +|---|---| +|A: | TODO | + + + diff --git a/src/plugins/input/nfb/README.md b/src/plugins/input/nfb/README.md new file mode 100644 index 000000000..038713b21 --- /dev/null +++ b/src/plugins/input/nfb/README.md @@ -0,0 +1,22 @@ +# NFB (input plugin) + +Receive packets over nfb (Netcope FPGA Board). Requires a special network card with compatible +firmware for high-speed packet processing. Suitable for speeds of 100 Gbps and above. + +## Example configuration + +```yaml +input_plugin: + ndp: + device: "/dev/nfb0" + queues: "0,1,2,3-15" +``` + +## Parameters + +**Mandatory parameters:** + +|Parameter | Description | +|---|---| +|__device__| Path to the NFB device to be used. Typically /dev/nfb0 or /dev/nfb/by-serial-no/{card-serial} | +|__queues__| List of queues to be used for packet reception. The queues can be specified as a comma-separated list (e.g., 0,1,2) or a range (e.g., 3-15). This is required to determine which specific receive queues to use on the NFB device. | diff --git a/src/plugins/input/pcap/README.md b/src/plugins/input/pcap/README.md index e69de29bb..9cdb2209e 100644 --- a/src/plugins/input/pcap/README.md +++ b/src/plugins/input/pcap/README.md @@ -0,0 +1,59 @@ +# Pcap Live (input plugin) + +The Pcap Live input plugin allows for real-time packet capture on a specified network interface. + +## Example configuration + +```yaml +input_plugin: + pcap_live: + interface: "eth0" + ### Optional parameters + snap_length: 65535 + bpf_filter: null +``` + +## Parameters + +**Mandatory parameters:** + +|Parameter | Description | +|---|---| +|__interface__| The network interface to capture packets from (e.g., eth0, ens33, etc.). This is required for the plugin to know which network interface to monitor. | + +----- + +**Optional parameters:** +|Parameter | Default | Description | +|---|---|---| +|__snap_length__ | 65535 | Maximum packet capture length in bytes. It defines the size of the packet that will be captured. The default value captures full packets (up to 65535 bytes). | +|__bpf_filter__ | null | A Berkeley Packet Filter (BPF) string for packet filtering. If null, no filter is applied. If a filter is specified, only packets matching the filter criteria will be captured. | + +# Pcap File (input plugin) + +The Pcap File input plugin allows you to read and process packets from an existing .pcap file. This is useful for analyzing historical packet captures or testing with predefined data. + +## Example configuration + +```yaml +input_plugin: + pcap_file: + file: "input.pcap" + ### Optional parameters + bpf_filter: null +``` + +## Parameters + +**Mandatory parameters:** + +|Parameter | Description | +|---|---| +|__file__| Path to the pcap file that contains the packet data to be read. | + +----- + +**Optional parameters:** +|Parameter | Default | Description | +|---|---|---| +|__bpf_filter__ | null | A Berkeley Packet Filter (BPF) string for packet filtering. If null, no filter is applied. If a filter is specified, only packets matching the filter criteria will be captured. | diff --git a/src/plugins/input/raw/README.md b/src/plugins/input/raw/README.md index e69de29bb..ae2c05650 100644 --- a/src/plugins/input/raw/README.md +++ b/src/plugins/input/raw/README.md @@ -0,0 +1,29 @@ +# Raw (Input Plugin) + +The Raw input plugin allows you to capture network traffic from a specified interface using raw sockets. It uses a circular buffer to hold the captured packets and can be optimized for specific traffic processing needs. + +## Example Configuration + +```yaml +input_plugin: + raw: + interface: "eth0" + ### Optional parameters + blocks_count: 2048 + packets_in_block: 32 +``` + +## Parameters + +**Mandatory Parameters** + +|Parameter | Description | +|---|---| +|__interface__| Network interface name (e.g., eth0) from which to capture traffic | + +**Optional parameters:** + +|Parameter | Default | Description | +|---|---|---| +|__blocks_count__ | 2048 | Number of blocks in the circular buffer, must be a power of 2. | +|__packets_in_block__ | 2048 | Number of packets per block, must be a power of 2. |