|
5 | 5 |
|
6 | 6 | # Overview |
7 | 7 |
|
8 | | -**Flow-Logs Pipeline** (a.k.a. FLP) is an **observability tool** that consumes raw **network flow-logs** in |
9 | | -their original format |
10 | | -([NetFlow v5,v9](https://en.wikipedia.org/wiki/NetFlow) or [IPFIX](https://en.wikipedia.org/wiki/IP_Flow_Information_Export)) |
11 | | -and uses a pipe-line to transform the logs into |
12 | | -time series metrics in **[prometheus](https://prometheus.io/)** format and in parallel |
13 | | -transform and persist the logs also into **[loki](https://grafana.com/oss/loki/)**. |
| 8 | +**Flow-Logs Pipeline** (a.k.a. FLP) is an **observability tool** that **[consumes](./pkg/pipeline/ingest/)** logs from various inputs, **[transform](./pkg//pipeline/transform/)** them and **[export](./pkg/pipeline/write/)** logs to **[loki](https://grafana.com/oss/loki/)** and / or time series metrics to **[prometheus](https://prometheus.io/)**. |
14 | 9 |
|
15 | 10 |  |
16 | 11 |
|
| 12 | +FLP can consume: |
| 13 | +- raw **network flow-logs** in their original format |
| 14 | +([NetFlow v5,v9](https://en.wikipedia.org/wiki/NetFlow) or [IPFIX](https://en.wikipedia.org/wiki/IP_Flow_Information_Export)) |
| 15 | +- [eBPF agent](https://github.com/netobserv/netobserv-ebpf-agent) flows in binary format (protobuf+GRPC) |
| 16 | +- Kafka entries in JSON format |
| 17 | +- A simple file |
| 18 | + |
17 | 19 | FLP decorates the metrics and the transformed logs with **context**, |
18 | 20 | allowing visualization layers and analytics frameworks to present **network insights** to SRE’s, cloud operators and network experts. |
19 | 21 |
|
20 | 22 | It also allows defining mathematical transformations to generate condense metrics that encapsulate network domain knowledge. |
21 | 23 |
|
22 | | -FLP pipe-line module is built on top of [gopipes](https://github.com/netobserv/gopipes) providing customizability and parallelism |
| 24 | +FLP pipeline module is built on top of [gopipes](https://github.com/netobserv/gopipes) providing customizability and parallelism |
23 | 25 |
|
24 | 26 | In addition, along with Prometheus and its ecosystem tools such as Thanos, Cortex etc., |
25 | 27 | FLP provides an efficient scalable multi-cloud solution for comprehensive network analytics that can rely **solely on metrics data-source**. |
|
0 commit comments