Skip to content

Commit c41d465

Browse files
lpettyjolpettyjo
authored
Merge pull request #95570 from gwynnemonahan/manual-cp-4-16-integration-branch-1-9
[enterprise-4.16] [NETOBSERV] OSDOCS-14786 Add section for IPsec
2 parents 19935e3 + b772838 commit c41d465

12 files changed

+355
-84
lines changed

‎modules/network-observability-con_filter-network-flows-at-ingestion.adoc

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
// Module included in the following assemblies:
2+
3+
// * networking/network_observability/configuring-operators.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="network-observability-filter-network-flows-at-ingestion_{context}"]
7+
= Filter network flows at ingestion
8+
9+
You can create filters to reduce the number of generated network flows. Filtering network flows can reduce the resource usage of the Network Observability components.
10+
11+
You can configure two kinds of filters:
12+
13+
* eBPF agent filters
14+
* Flowlogs-pipeline filters
15+
16+
[id="ebpf-agent-filters_{context}"]
17+
== eBPF agent filters
18+
19+
eBPF agent filters maximize performance because they take effect at the earliest stage of the network flows collection process.
20+
21+
To configure eBPF agent filters with the Network Observability Operator, see "Filtering eBPF flow data using multiple rules".
22+
23+
[id="flowlogs-pipeline-filters_{context}"]
24+
== Flowlogs-pipeline filters
25+
26+
Flowlogs-pipeline filters provide greater control over traffic selection because they take effect later in the network flows collection process. They are primarily used to improve data storage.
27+
28+
Flowlogs-pipeline filters use a simple query language to filter network flow, as shown in the following example:
29+
30+
[source,terminal]
31+
----
32+
(srcnamespace="netobserv" OR (srcnamespace="ingress" AND dstnamespace="netobserv")) AND srckind!="service"
33+
----
34+
35+
The query language uses the following syntax:
36+
37+
.Query language syntax
38+
[cols="1,3", options="header"]
39+
|===
40+
| Category
41+
| Operators
42+
43+
| Logical boolean operators (not case-sensitive)
44+
| `and`, `or`
45+
46+
| Comparison operators
47+
| `=` (equals), +
48+
`!=` (not equals), +
49+
`=~` (matches regexp), +
50+
`!~` (not matches regexp), +
51+
`<` / `\<=` (less than or equal to), +
52+
`>` / `>=` (greater than or equal to)
53+
54+
| Unary operations
55+
| `with(field)` (field is present), +
56+
`without(field)` (field is absent)
57+
58+
| Parenthesis-based priority
59+
|===
60+
61+
You can configure flowlogs-pipeline filters in the `spec.processor.filters` section of the `FlowCollector` resource. For example:
62+
63+
.Example YAML Flowlogs-pipeline filter
64+
[source,yaml]
65+
----
66+
apiVersion: flows.netobserv.io/v1beta2
67+
kind: FlowCollector
68+
metadata:
69+
name: cluster
70+
spec:
71+
namespace: netobserv
72+
agent:
73+
processor:
74+
filters:
75+
- query: |
76+
(SrcK8S_Namespace="netobserv" OR (SrcK8S_Namespace="openshift-ingress" AND DstK8S_Namespace="netobserv"))
77+
outputTarget: Loki <1>
78+
sampling: 10 <2>
79+
----
80+
<1> Sends matching flows to a specific output, such as Loki, Prometheus, or an external system. When omitted, sends to all configured outputs.
81+
<2> Optional. Applies a sampling ratio to limit the number of matching flows to be stored or exported. For example, `sampling: 10` means 1/10 of the flows are kept.

‎modules/network-observability-ebpf-rule-flow-filter.adoc

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,15 @@
55
:_mod-docs-content-type: CONCEPT
66
[id="network-observability-ebpf-flow-rule-filter_{context}"]
77
= eBPF flow rule filter
8-
You can use rule-based filtering to control the volume of packets cached in the eBPF flow table. For example, a filter can specify that only packets coming from port 100 should be recorded. Then only the packets that match the filter are cached and the rest are not cached.
8+
You can use rule-based filtering to control the volume of packets cached in the eBPF flow table. For example, a filter can specify that only packets coming from port 100 should be captured. Then only the packets that match the filter are captured and the rest are dropped.
9+
10+
You can apply multiple filter rules.
911

1012
[id="ingress-and-egress-traffic-filtering_{context}"]
1113
== Ingress and egress traffic filtering
12-
CIDR notation efficiently represents IP address ranges by combining the base IP address with a prefix length. For both ingress and egress traffic, the source IP address is first used to match filter rules configured with CIDR notation. If there is a match, then the filtering proceeds. If there is no match, then the destination IP is used to match filter rules configured with CIDR notation.
14+
Classless Inter-Domain Routing (CIDR) notation efficiently represents IP address ranges by combining the base IP address with a prefix length. For both ingress and egress traffic, the source IP address is first used to match filter rules configured with CIDR notation. If there is a match, then the filtering proceeds. If there is no match, then the destination IP is used to match filter rules configured with CIDR notation.
1315

14-
After matching either the source IP or the destination IP CIDR, you can pinpoint specific endpoints using the `peerIP` to differentiate the destination IP address of the packet. Based on the provisioned action, the flow data is either cached in the eBPF flow table or not cached.
16+
After matching either the source IP or the destination IP CIDR, you can pinpoint specific endpoints using the `peerIP` to differentiate the destination IP address of the packet. Based on the provisioned action, the flow data is either cached in the eBPF flow table or not cached.
1517

1618
[id="dashboard-and-metrics-integrations_{context}"]
1719
== Dashboard and metrics integrations

‎modules/network-observability-filtering-ebpf-rule.adoc

Lines changed: 50 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,28 @@
11
// Module included in the following assemblies:
2-
//
32
// network_observability/observing-network-traffic.adoc
43

54
:_mod-docs-content-type: PROCEDURE
65
[id="network-observability-filtering-ebpf-rule_{context}"]
7-
= Filtering eBPF flow data using a global rule
8-
You can configure the `FlowCollector` to filter eBPF flows using a global rule to control the flow of packets cached in the eBPF flow table.
6+
= Filtering eBPF flow data using multiple rules
7+
You can configure the `FlowCollector` custom resource to filter eBPF flows using multiple rules to control the flow of packets cached in the eBPF flow table.
8+
9+
[IMPORTANT]
10+
====
11+
* You cannot use duplicate CIDRs in filter rules.
12+
* When an IP address matches multiple filter rules, the rule with the most specific CIDR prefix (longest prefix) takes precedence.
13+
====
914

1015
.Procedure
1116
. In the web console, navigate to *Operators* -> *Installed Operators*.
1217
. Under the *Provided APIs* heading for *Network Observability*, select *Flow Collector*.
1318
. Select *cluster*, then select the *YAML* tab.
1419
. Configure the `FlowCollector` custom resource, similar to the following sample configurations:
15-
+
1620

17-
[%collapsible]
18-
.Filter Kubernetes service traffic to a specific Pod IP endpoint
19-
====
21+
22+
.Example YAML to sample all North-South traffic, and 1:50 East-West traffic
23+
24+
By default, all other flows are rejected.
25+
2026
[source, yaml]
2127
----
2228
apiVersion: flows.netobserv.io/v1beta2
@@ -30,22 +36,29 @@ spec:
3036
type: eBPF
3137
ebpf:
3238
flowFilter:
33-
action: Accept <1>
34-
cidr: 172.210.150.1/24 <2>
35-
protocol: SCTP
36-
direction: Ingress
37-
destPortRange: 80-100
38-
peerIP: 10.10.10.10
39-
enable: true <3>
39+
enable: true <1>
40+
rules:
41+
- action: Accept <2>
42+
cidr: 0.0.0.0/0 <3>
43+
sampling: 1 <4>
44+
- action: Accept
45+
cidr: 10.128.0.0/14
46+
peerCIDR: 10.128.0.0/14 <5>
47+
- action: Accept
48+
cidr: 172.30.0.0/16
49+
peerCIDR: 10.128.0.0/14
50+
sampling: 50
4051
----
41-
<1> The required `action` parameter describes the action that is taken for the flow filter rule. Possible values are `Accept` or `Reject`.
42-
<2> The required `cidr` parameter provides the IP address and CIDR mask for the flow filter rule and supports IPv4 and IPv6 address formats. If you want to match against any IP address, you can use `0.0.0.0/0` for IPv4 or `::/0` for IPv6.
43-
<3> You must set `spec.agent.ebpf.flowFilter.enable` to `true` to enable this feature.
44-
====
45-
+
46-
[%collapsible]
47-
.See flows to any addresses outside the cluster
48-
====
52+
<1> To enable eBPF flow filtering, set `spec.agent.ebpf.flowFilter.enable` to `true`.
53+
<2> To define the action for the flow filter rule, set the required `action` parameter. Valid values are `Accept` or `Reject`.
54+
<3> To define the IP address and CIDR mask for the flow filter rule, set the required `cidr` parameter. This parameter supports both IPv4 and IPv6 address formats. To match any IP address, use `0.0.0.0/0` for IPv4 or ``::/0` for IPv6.
55+
<4> To define the sampling rate for matched flows and override the global sampling setting `spec.agent.ebpf.sampling`, set the `sampling` parameter.
56+
<5> To filter flows by Peer IP CIDR, set the `peerCIDR` parameter.
57+
58+
.Example YAML to filter flows with packet drops
59+
60+
By default, all other flows are rejected.
61+
4962
[source, yaml]
5063
----
5164
apiVersion: flows.netobserv.io/v1beta2
@@ -57,18 +70,19 @@ spec:
5770
deploymentModel: Direct
5871
agent:
5972
type: eBPF
60-
ebpf:
73+
ebpf:
74+
privileged: true <1>
75+
features:
76+
- PacketDrop <2>
6177
flowFilter:
62-
action: Accept <1>
63-
cidr: 0.0.0.0/0 <2>
64-
protocol: TCP
65-
direction: Egress
66-
sourcePort: 100
67-
peerIP: 192.168.127.12 <3>
68-
enable: true <4>
69-
----
70-
<1> You can `Accept` flows based on the criteria in the `flowFilter` specification.
71-
<2> The `cidr` value of `0.0.0.0/0` matches against any IP address.
72-
<3> See flows after `peerIP` is configured with `192.168.127.12`.
73-
<4> You must set `spec.agent.ebpf.flowFilter.enable` to `true` to enable the feature.
74-
====
78+
enable: true <3>
79+
rules:
80+
- action: Accept <4>
81+
cidr: 172.30.0.0/16
82+
pktDrops: true <5>
83+
----
84+
<1> To enable packet drops, set `spec.agent.ebpf.privileged` to `true`.
85+
<2> To report packet drops for each network flow, add the `PacketDrop` value to the `spec.agent.ebpf.features` list.
86+
<3> To enable eBPF flow filtering, set `spec.agent.ebpf.flowFilter.enable` to `true`.
87+
<4> To define the action for the flow filter rule, set the required `action` parameter. Valid values are `Accept` or `Reject`.
88+
<5> To filter flows containing drops, set `pktDrops` to `true`.

‎modules/network-observability-flowcollector-api-specifications.adoc

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -444,7 +444,6 @@ To filter two ports, use a "port1,port2" in string format. For example, `ports:
444444
| `rules` defines a list of filtering rules on the eBPF Agents.
445445
When filtering is enabled, by default, flows that don't match any rule are rejected.
446446
To change the default, you can define a rule that accepts everything: `{ action: "Accept", cidr: "0.0.0.0/0" }`, and then refine with rejecting rules.
447-
Unsupported *.
448447
449448
| `sampling`
450449
| `integer`
@@ -470,7 +469,6 @@ Description::
470469
`rules` defines a list of filtering rules on the eBPF Agents.
471470
When filtering is enabled, by default, flows that don't match any rule are rejected.
472471
To change the default, you can define a rule that accepts everything: `{ action: "Accept", cidr: "0.0.0.0/0" }`, and then refine with rejecting rules.
473-
Unsupported *.
474472
--
475473
476474
Type::
@@ -483,7 +481,7 @@ Type::
483481
Description::
484482
+
485483
--
486-
`EBPFFlowFilterRule` defines the desired eBPF agent configuration regarding flow filtering rule.
484+
`EBPFFlowFilterRules` defines the desired eBPF agent configuration regarding flow filtering rules.
487485
--
488486
489487
Type::
@@ -1480,15 +1478,15 @@ Type::
14801478
14811479
| `input`
14821480
| `string`
1483-
|
1481+
|
14841482
14851483
| `multiplier`
14861484
| `integer`
1487-
|
1485+
|
14881486
14891487
| `output`
14901488
| `string`
1491-
|
1489+
|
14921490
14931491
|===
14941492
== .spec.exporters[].openTelemetry.logs

‎modules/network-observability-flowmetric-api-specifications.adoc

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -103,13 +103,12 @@ When set to `Egress`, it is equivalent to adding the regular expression filter o
103103

104104
| `filters`
105105
| `array`
106-
| `filters` is a list of fields and values used to restrict which flows are taken into account. Oftentimes, these filters must
107-
be used to eliminate duplicates: `Duplicate != "true"` and `FlowDirection = "0"`.
106+
| `filters` is a list of fields and values used to restrict which flows are taken into account.
108107
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.
109108

110109
| `flatten`
111110
| `array (string)`
112-
| `flatten` is a list of list-type fields that must be flattened, such as Interfaces and NetworkEvents. Flattened fields generate one metric per item in that field.
111+
| `flatten` is a list of array-type fields that must be flattened, such as Interfaces or NetworkEvents. Flattened fields generate one metric per item in that field.
113112
For instance, when flattening `Interfaces` on a bytes counter, a flow having Interfaces [br-ex, ens5] increases one counter for `br-ex` and another for `ens5`.
114113

115114
| `labels`
@@ -131,9 +130,10 @@ Refer to the documentation for the list of available fields: https://docs.opensh
131130

132131
| `type`
133132
| `string`
134-
| Metric type: "Counter" or "Histogram".
133+
| Metric type: "Counter", "Histogram" or "Gauge".
135134
Use "Counter" for any value that increases over time and on which you can compute a rate, such as Bytes or Packets.
136135
Use "Histogram" for any value that must be sampled independently, such as latencies.
136+
Use "Gauge" for other values that don't necessitate accuracy over time (gauges are sampled only every N seconds when Prometheus fetches the metric).
137137

138138
| `valueField`
139139
| `string`
@@ -261,8 +261,7 @@ To learn more about `promQL`, refer to the Prometheus documentation: https://pro
261261
Description::
262262
+
263263
--
264-
`filters` is a list of fields and values used to restrict which flows are taken into account. Oftentimes, these filters must
265-
be used to eliminate duplicates: `Duplicate != "true"` and `FlowDirection = "0"`.
264+
`filters` is a list of fields and values used to restrict which flows are taken into account.
266265
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.
267266
--
268267

‎modules/network-observability-flows-format.adoc

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -155,13 +155,6 @@ The "Cardinality" column gives information about the implied metric cardinality
155155
| no
156156
| fine
157157
| n/a
158-
| `Duplicate`
159-
| boolean
160-
| Indicates if this flow was also captured from another interface on the same host
161-
| n/a
162-
| no
163-
| fine
164-
| n/a
165158
| `Flags`
166159
| string[]
167160
| List of TCP flags comprised in the flow, according to RFC-9293, with additional custom flags to represent the following per-packet combinations: +
@@ -182,6 +175,13 @@ The "Cardinality" column gives information about the implied metric cardinality
182175
| yes
183176
| fine
184177
| host.direction
178+
| `IPSecStatus`
179+
| string
180+
| Status of the IPsec encryption (on egress, given by the kernel xfrm_output function) or decryption (on ingress, via xfrm_input)
181+
| `ipsec_status`
182+
| no
183+
| fine
184+
| n/a
185185
| `IcmpCode`
186186
| number
187187
| ICMP code
@@ -242,7 +242,7 @@ The "Cardinality" column gives information about the implied metric cardinality
242242
| `Packets`
243243
| number
244244
| Number of packets
245-
| `pkt_drop_cause`
245+
| n/a
246246
| no
247247
| avoid
248248
| packets
@@ -423,35 +423,35 @@ The "Cardinality" column gives information about the implied metric cardinality
423423
| n/a
424424
| `XlatDstAddr`
425425
| string
426-
| Packet translation destination address
426+
| packet translation destination address
427427
| `xlat_dst_address`
428428
| no
429429
| avoid
430430
| n/a
431431
| `XlatDstPort`
432432
| number
433-
| Packet translation destination port
433+
| packet translation destination port
434434
| `xlat_dst_port`
435435
| no
436436
| careful
437437
| n/a
438438
| `XlatSrcAddr`
439439
| string
440-
| Packet translation source address
440+
| packet translation source address
441441
| `xlat_src_address`
442442
| no
443443
| avoid
444444
| n/a
445445
| `XlatSrcPort`
446446
| number
447-
| Packet translation source port
447+
| packet translation source port
448448
| `xlat_src_port`
449449
| no
450450
| careful
451451
| n/a
452452
| `ZoneId`
453453
| number
454-
| Packet translation zone id
454+
| packet translation zone id
455455
| `xlat_zone_id`
456456
| no
457457
| avoid
@@ -470,4 +470,4 @@ The "Cardinality" column gives information about the implied metric cardinality
470470
| yes
471471
| fine
472472
| n/a
473-
|===
473+
|===

0 commit comments

Comments
 (0)