Skip to content

Commit e1f8266

Browse files
author
Pavel Siska
committed
WIP
1 parent 537eb74 commit e1f8266

File tree

1 file changed

+169
-5
lines changed

1 file changed

+169
-5
lines changed

‎README.md

Lines changed: 169 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,186 @@
66
</div>
77
</br>
88

9-
The ipfixprobe is a high-performance flow exporter. It creates bidirectional flows from packet input and exports them to output interface. The ipfixprobe support vide variety of flow extenstion for application layer protocol information. The flow extension can be turned on with process plugins. We support TLS, QUIC, HTTP, DNS and many more. Just check our [documentation](https://ipfixprobe.cesnet.cz).
9+
The ipfixprobe is a high-performance, modular flow exporter that processes packets into bidirectional flows and exports them via a selected output plugin.
10+
It supports a wide range of application-layer protocol parsers, including TLS, QUIC, HTTP, DNS, and many others. These protocol-specific extensions can be enabled via process plugins in the configuration.
11+
12+
Need more details? Check out our [documentation](https://ipfixprobe.cesnet.cz) for a full list of supported protocols and usage examples.
1013

1114
[![](https://img.shields.io/badge/license-BSD-blue.svg)](https://github.com/CESNET/ipfixprobe/blob/master/LICENSE)
1215
[![](https://img.shields.io/badge/docs-ipfixprobe-blue.svg)](https://ipfixprobe.cesnet.cz)
1316
![Coverity Scan](https://img.shields.io/coverity/scan/22112)
1417
![GitHub top language](https://img.shields.io/github/languages/top/CESNET/ipfixprobe)
1518

19+
## ✨ Key Features
20+
21+
- Modular input–parser–output architecture
22+
- High-speed processing (DPDK, multi-threaded, NUMA-aware)
23+
- Built-in protocol parsers: TLS, QUIC, HTTP, DNS, …
24+
- Bidirectional flow (biflow) support
25+
- Real-time telemetry and statistics
26+
27+
## 📦 Installation
28+
29+
If you are running a RHEL system or one of its derivatives (e.g. Oracle Linux, Rocky Linux, CentOS Stream), the easiest way to install
30+
ipfixprobe is from our [copr repository](https://copr.fedorainfracloud.org/coprs/g/CESNET/ipfixprobe/).
31+
32+
```bash
33+
$ dnf install dnf-plugins-core # Extra step necessary on some systems
34+
$ dnf copr enable @CESNET/ipfixprobe
35+
$ dnf install ipfixprobe
36+
```
37+
38+
This installs the main `ipfixprobe` binary along with core functionality.
39+
40+
#### 🗃️ Available Packages
41+
42+
The Copr repository provides modular RPM packages, so you can install only what you need.
43+
The following packages are available and can be installed individually as needed:
44+
45+
| Package Name | Description |
46+
|--------------------------------------------|-----------------------------------------------------------------------------|
47+
| `ipfixprobe` | Core binary with common process/output plugins. |
48+
| `ipfixprobe-msec` | Core binary with common process/output plugins. Uses millisecond timestamps (compatible with Flowmon collector) |
49+
| `ipfixprobe-input-pcap` | Input plugin for PCAP files and live capture |
50+
| `ipfixprobe-input-dpdk` | High-speed input plugin using DPDK |
51+
| `ipfixprobe-input-nfb` | Input plugin for CESNET NFB/NDP cards |
52+
| `ipfixprobe-process-experimental` | Extra (possibly unstable) process plugins |
53+
54+
For other systems, follow the build instructions below.
55+
56+
## 🛠️ Build
57+
58+
You can build ipfixprobe from source using standard CMake.
59+
This lets you customize the build by enabling optional plugins and features as needed.
60+
61+
**Note:** Some plugins may require additional dependencies beyond the basic requirements.
62+
63+
### RHEL/CentOS:
64+
65+
#### 🧰 Requirements
66+
67+
```bash
68+
$ dnf install epel-release git make cmake gcc-c++ rpm-build
69+
$ dnf install libunwind-devel lz4-devel openssl-devel fuse3-devel
70+
71+
# for RHEL 8/9
72+
$ dnf install gcc-toolset-14-libatomic-devel
73+
74+
# for RHEL 10+
75+
$ dnf install libatomic
76+
```
77+
78+
### Debian/Ubuntu:
79+
#### 🧰 Requirements
80+
81+
TODO
82+
83+
#### 🧱 Build steps
84+
85+
```bash
86+
git clone https://github.com/CESNET/ipfixprobe.git
87+
cd ipfixprobe
88+
mkdir build && cd build
89+
cmake ..
90+
make -j$(nproc)
91+
# make install
92+
```
1693

17-
## 🛠️ Installation
18-
The RPM packages for RHEL based distributions can be downloaded from our [copr repository](https://copr.fedorainfracloud.org/coprs/g/CESNET/ipfixprobe/package/ipfixprobe/). Or just simply run:
94+
#### ⚙️ Optional build flags
95+
You can enable or disable optional plugins and features via CMake flags:
96+
97+
| Flag | Default | Description |
98+
| ---------------------------------- | ------- | ---------------------------------------------------------------- |
99+
| `-DENABLE_MILLISECONDS_TIMESTAMP=ON` | OFF | Use millisecond precision timestamps (for Flowmon compatibility) |
100+
| `-DENABLE_INPUT_PCAP=ON` | OFF | Enable PCAP input plugin (live & file) (requires `libpcap`) |
101+
| `-DENABLE_INPUT_DPDK=ON` | OFF | Enable high-speed DPDK input plugin (requires `dpdk-devel`) |
102+
| `-DENABLE_INPUT_NFB=ON` | OFF | Enable input plugin for CESNET NFB/NDP cards (requires `netcope-common`) |
103+
| `-DENABLE_PROCESS_EXPERIMENTAL=ON` | OFF | Enable experimental process plugins |
104+
| `-DENABLE_NEMEA=ON` | OFF | Enable support for NEMEA modules (requires `nemea-framework-devel` ) |
105+
106+
Run the command to view all available build options:
19107

108+
```bash
109+
cmake -LAH
20110
```
21-
dnf install -y dnf-plugins-core &&dnf copr enable @CESNET/ipfixprobe
22-
dnf install ipfixprobe
111+
112+
#### Example
113+
To build with DPDK and PCAP input support, and install to /usr:
114+
115+
```cmake
116+
cmake .. \
117+
-DCMAKE_INSTALL_PREFIX=/usr \
118+
-DENABLE_INPUT_PCAP=ON \
119+
-DENABLE_INPUT_DPDK=ON
23120
```
24121

122+
123+
## 🧩 Available Plugins
124+
125+
### Input Plugins
126+
List of input plugins with estimated performance and configuration complexity.
127+
128+
| Plugin | Max Throughput | Usage Complexity | Description |
129+
|---------------|----------------|------------------|-------------------------------------------|
130+
| [`pcap_live`](./src/plugins/input/pcap/README.md#pcap-live-input-plugin) | ~1 Gbps | Easy | captures packets from a live network interface |
131+
| [`pcap_file`](./src/plugins/input/pcap/README.md#pcap-file-input-plugin) | ~1 Gbps | Easy | reads packets from an offline PCAP file |
132+
| [`raw`](./src/plugins/input/raw/README.md) | ~1 Gbps | Easy | captures packets using a raw socket |
133+
| [`ndp`](./src/plugins/input/nfb/README.md) | 400 Gbps | Medium | uses CESNET NFB/NDP hardware for packet input |
134+
| [`dpdk`](./src/plugins/input/dpdk/README.md#dpdk-input-plugin) | 400 Gbps | Complex | receives packets via high-performance DPDK |
135+
| [`dpdk-ring`](./src/plugins/input/dpdk/README.md) | 400 Gbps | Complex | receives packets from a shared DPDK memory ring |
136+
137+
---
138+
139+
### Process Plugins
140+
141+
These plugins extract protocol-specific or behavioral information from packets and enrich flow records with metadata.
142+
143+
| Plugin | Description |
144+
|---------------|--------------------------------------------------------------|
145+
| [`basic`](./src/plugins/process/basic/README.md) | extracts basic L3/L4 flow fields (IPs, ports, protocol) |
146+
| [`icmp`](./src/plugins/process/icmp/README.md) | extracts ICMP type/code and related metadata |
147+
| [`http`](./src/plugins/process/http/README.md) | extracts HTTP methods, hosts, URIs, status codes |
148+
| [`tls`](./src/plugins/process/tls/README.md) | extracts TLS handshake info (SNI, version, JA3, etc.) |
149+
| [`ovpn`](./src/plugins/process/ovpn/README.md) | extracts metadata from OpenVPN tunnels |
150+
| [`wg`](./src/plugins/process/wg/README.md) | parses WireGuard handshake and endpoint metadata |
151+
| [`quic`](./src/plugins/process/quic/README.md) | parses QUIC protocol including SNI, versions, ALPN |
152+
| [`basicplus`](./src/plugins/process/basicplus/README.md) | adds common L3/L4 flow fields (e.g., ports, IPs, TCP flags) |
153+
| [`bstats`](./src/plugins/process/bstats/README.md) | basic flow statistics (packet/byte counters, duration, ...) |
154+
| [`dns`](./src/plugins/process/dns/README.md) | extracts DNS queries, responses, and domains |
155+
| [`dnssd`](./src/plugins/process/dnssd/README.md) | parses DNS Service Discovery (mDNS) traffic |
156+
| [`flowHash`](./src/plugins/process/flowHash/README.md) | extracts a flow hash |
157+
| [`idpContent`](./src/plugins/process/idpContent/README.md) | parses IDP content in flows |
158+
| [`mpls`](./src/plugins/process/mpls/README.md) | extracts MPLS labels and encapsulation metadata |
159+
| [`mqtt`](./src/plugins/process/mqtt/README.md) | parses MQTT protocol traffic (IoT messaging) |
160+
| [`netbios`](./src/plugins/process/netbios/README.md) | extracts NetBIOS session and name service info |
161+
| [`nettisa`](./src/plugins/process/nettisa/README.md) | parses NETTISA related metadata (experimental) |
162+
| [`ntp`](./src/plugins/process/ntp/README.md) | extracts NTP timestamps and server info |
163+
| [`osquery`](./src/plugins/process/osquery/README.md) | parses osquery-generated data streams |
164+
| [`passiveDns`](./src/plugins/process/passiveDns/README.md) | generates passive DNS entries from observed DNS traffic |
165+
| [`phists`](./src/plugins/process/phists/README.md) | parses phishing-related signatures (heuristic) |
166+
| [`pstats`](./src/plugins/process/pstats/README.md) | advanced packet statistics (e.g., inter-packet gaps) |
167+
| [`rtsp`](./src/plugins/process/rtsp/README.md) | extracts RTSP stream metadata |
168+
| [`sip`](./src/plugins/process/sip/README.md) | parses SIP call setup, headers, and codecs |
169+
| [`smtp`](./src/plugins/process/smtp/README.md) | extracts SMTP envelope data (from, to, subject, etc.) |
170+
| [`ssaDetector`](./src/plugins/process/ssaDetector/README.md) | performs simple anomaly detection based on traffic patterns |
171+
| [`ssdp`](./src/plugins/process/ssdp/README.md) | parses SSDP (UPnP discovery) protocol |
172+
| [`vlan`](./src/plugins/process/vlan/README.md) | extracts VLAN IDs and QinQ encapsulation |
173+
174+
---
175+
### Output Plugins
176+
177+
These plugins export flow records to various formats and external systems.
178+
179+
| Plugin | Description |
180+
|---------------|-----------------------------------------------------------------------------|
181+
| [`ipfix`](./src/plugins/output/ipfix/README.md) | exports flow records in IPFIX format to a remote collector (UDP/TCP) |
182+
| [`text`](./src/plugins/output/text/README.md) | writes flow records in human-readable text to a file or stdout |
183+
| [`unirec`](./src/plugins/output/unirec/README.md) | exports flow records using the UniRec format for NEMEA/TRAP ecosystem |
184+
185+
---
186+
187+
188+
25189
## 🔧 Parameters
26190
### Module specific parameters
27191
- `-i ARGS` Activate input plugin (-h input for help)

0 commit comments

Comments
 (0)