Skip to content

Commit b2b688c

Browse files
jpinsonneaujotak
jpinsonneau
and
jotak
authored
NETOBSERV-1911 CLI metrics (#106)
* run in background * allow custom namespace * add metrics capture & cleanup readme * update e2e * Update README.md Co-authored-by: Joel Takvorian <joel.takvorian@homeblocks.net> * Update README.md Co-authored-by: Joel Takvorian <joel.takvorian@homeblocks.net> * NETOBSERV-2030 missing toleration on daemonset for metrics * skip dependencies check on help and update doc --------- Co-authored-by: Joel Takvorian <joel.takvorian@homeblocks.net>
1 parent 42be610 commit b2b688c

File tree

12 files changed

+921
-142
lines changed

12 files changed

+921
-142
lines changed

‎README.md‎

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
# Network Observability CLI
22

3-
network-observability-cli is a lightweight Flow and Packet visualization tool.
3+
network-observability-cli is a lightweight Flow, Packet and Metrics visualization tool.
44
It deploys [NetObserv eBPF agent](https://github.com/netobserv/netobserv-ebpf-agent) on your k8s cluster to collect flows or packets from nodes network interfaces
55
and streams data to a local collector for analysis and visualization.
66
Output files are generated under `output/flow` and `output/pcap` directories per host name
77

8+
On Openshift environments, you can also capture metrics in your monitoring stack and display a fully configured dashboard.
9+
810
## Prerequisites
911

1012
To run this CLI, you will need:
@@ -44,7 +46,7 @@ USER=netobserv VERSION=dev make images
4446
Run the following command to start capturing flows, replacing `USER`, `VERSION` and `COMMAND_ARGS` accordingly:
4547

4648
```bash
47-
USER=netobserv VERSION=dev COMMAND_ARGS=br-ex make flows
49+
USER=netobserv VERSION=dev COMMAND_ARGS=--interfaces=br-ex make flows
4850
```
4951

5052
![flows](./img/flow-table.png)
@@ -107,24 +109,34 @@ or `dbeaver`:
107109
Run the following command to start capturing packets, replacing `USER`, `VERSION` and `COMMAND_ARGS` accordingly:
108110

109111
```bash
110-
USER=netobserv VERSION=dev COMMAND_ARGS=tcp,80 make packets
112+
USER=netobserv VERSION=dev COMMAND_ARGS="--protocol=TCP --port=80" make packets
111113
```
112114

113-
![packets](./img/packet-table.png)
114-
115-
It will display a table view with latest packets collected and write data under output/pcap directory.
115+
Similarly to flow capture, it will display a table view with latest flows. However, it will collect packets and write data under output/pcap directory.
116116
To stop capturing press Ctrl-C.
117117

118-
This will write pcap into a single file located in `./output/pcap/<CAPTURE_DATE_TIME>.pcap` that can be opened with Wireshark for example:
118+
This will write [pcapng](https://wiki.wireshark.org/Development/PcapNg) into a single file located in `./output/pcap/<CAPTURE_DATE_TIME>.pcapng` that can be opened with Wireshark for example:
119119

120120
![wireshark](./img/wireshark.png)
121121

122+
### Metrics dashboard (OCP only)
123+
124+
Run the following command to start capturing metrics, replacing `USER`, `VERSION` and `COMMAND_ARGS` accordingly:
125+
```bash
126+
USER=netobserv VERSION=dev COMMAND_ARGS='--enable_pktdrop="true" --enable_dns="true" --enable_rtt="true"' make metrics
127+
```
128+
129+
![metrics](./img/metrics-dashboard.png)
130+
131+
It will generate a monitoring dashboard called "NetObserv / On Demand" in your Openshift cluster.
132+
The url to access it is automatically generated from the CLI. Simply click on the link to open the page.
133+
122134
### Cleanup
123135

124-
The `cleanup` function will automatically remove the eBPF programs when the CLI exits. However you may need to run it manually if an error occurs.
136+
The `cleanup` function will automatically remove the eBPF programs when the CLI exits. However you may need to run it manually if running in background or an error occurs.
125137

126138
```bash
127-
./commands/netobserv-cleanup
139+
USER=netobserv VERSION=dev make cleanup
128140
```
129141

130142
## Extending OpenShift or Kubernetes CLI with plugins

‎commands/netobserv‎

Lines changed: 142 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -41,133 +41,179 @@ command=""
4141
logLevel="info"
4242

4343
# max time (default: 5min)
44-
maxTime="5m"
44+
maxTime="5m"
4545

4646
# max bytes (default: 50MB)
4747
maxBytes=50000000
4848

4949
function flows() {
5050
case "$2" in
51-
"help")
52-
flows_usage
53-
exit 0 ;;
54-
*)
55-
shift # remove first argument
56-
options="$*"
57-
# run flows command
58-
command="flows" ;;
51+
"help")
52+
flows_usage
53+
exit 0
54+
;;
55+
*)
56+
shift # remove first argument
57+
options="$*"
58+
# run flows command
59+
command="flows"
60+
;;
5961
esac
6062
}
6163

6264
function packets() {
6365
case "$2" in
64-
"help")
65-
packets_usage
66-
exit 0 ;;
67-
*)
68-
shift # remove first argument
69-
options="$*"
70-
# run packets command
71-
command="packets" ;;
66+
"help")
67+
packets_usage
68+
exit 0
69+
;;
70+
*)
71+
shift # remove first argument
72+
options="$*"
73+
# run packets command
74+
command="packets"
75+
;;
7276
esac
7377
}
7478

75-
required_yq_version="v0.0.0"
76-
supported_archs=""
77-
check_dependencies "$required_yq_version" "$supported_archs"
79+
function metrics() {
80+
case "$2" in
81+
"help")
82+
metrics_usage
83+
exit 0
84+
;;
85+
*)
86+
shift # remove first argument
87+
options="$*"
88+
# run metrics command
89+
command="metrics"
90+
;;
91+
esac
92+
}
93+
94+
if [[ ! "$*" =~ ^(.*)help|version(.*) ]]; then
95+
required_yq_version="v0.0.0"
96+
supported_archs=""
97+
check_dependencies "$required_yq_version" "$supported_archs"
98+
fi
7899

79100
case "$1" in
80101
"help")
81-
# display Help
82-
echo
83-
echo "Netobserv allows you to capture flow and packets from your cluster."
84-
echo "Find more information at: https://github.com/netobserv/network-observability-cli/"
85-
echo
86-
echo "Syntax: netobserv [flows|packets|cleanup] [options]"
87-
echo
88-
echo "commands:"
89-
echo " flows Capture flows information in JSON format."
90-
echo " Options:"
91-
flows_usage
92-
echo " packets Capture packets information in pcap format."
93-
echo " Options:"
94-
packets_usage
95-
echo " follow Follow collector logs when running in background."
96-
echo " stop Stop collection by removing agent daemonset."
97-
echo " copy Copy generated files locally."
98-
echo " cleanup Remove netobserv components."
99-
echo " version Print software version."
100-
echo
101-
exit 0 ;;
102+
# display Help
103+
echo
104+
echo "Netobserv allows you to capture flow, packets and metrics from your cluster."
105+
echo "Find more information at: https://github.com/netobserv/network-observability-cli/"
106+
echo
107+
echo "Syntax: netobserv [flows|packets|metrics|follow|stop|copy|cleanup|version] [options]"
108+
echo
109+
echo "commands:"
110+
echo " flows Capture flows information in JSON format using collector pod."
111+
echo " Options:"
112+
flows_usage
113+
echo " packets Capture packets information in pcap format using collector pod."
114+
echo " Options:"
115+
packets_usage
116+
echo " metrics Capture metrics information in Prometheus using a ServiceMonitor (OCP cluster only)."
117+
echo " Options:"
118+
metrics_usage
119+
echo " follow Follow collector logs when running in background."
120+
echo " stop Stop collection by removing agent daemonset."
121+
echo " copy Copy collector generated files locally."
122+
echo " cleanup Remove netobserv components and configurations."
123+
echo " version Print software version."
124+
echo
125+
exit 0
126+
;;
102127
"version")
103-
# display version
104-
echo "Netobserv CLI version $version"
105-
exit 0 ;;
128+
# display version
129+
echo "Netobserv CLI version $version"
130+
exit 0
131+
;;
106132
"flows")
107-
flows $* ;;
133+
flows $*
134+
;;
108135
"packets")
109-
packets $* ;;
136+
packets $*
137+
;;
138+
"metrics")
139+
metrics $*
140+
;;
110141
"follow")
111-
# run follow command
112-
follow
113-
exit 0 ;;
142+
# run follow command
143+
follow
144+
exit 0
145+
;;
114146
"stop")
115-
# run deleteDaemonset command
116-
deleteDaemonset
117-
exit 0 ;;
147+
# run deleteDaemonset command
148+
deleteDaemonset
149+
exit 0
150+
;;
118151
"copy")
119-
# run copy output command
120-
copyOutput
121-
exit 0 ;;
152+
# run copy output command
153+
copyOutput
154+
exit 0
155+
;;
122156
"cleanup")
123-
# run cleanup command
124-
cleanup
125-
exit 0 ;;
157+
# run cleanup command
158+
cleanup
159+
exit 0
160+
;;
126161
*)
127-
echo "Unknown command $1. Use 'netobserv help' to display options"
128-
exit 1
162+
echo "Unknown command $1. Use 'netobserv help' to display options"
163+
exit 1
164+
;;
129165
esac
130166

131167
trap cleanup EXIT
132168

133169
setup $command $options
134170

135-
# convert options to string
136-
optionStr="${options//--/}"
137-
optionStr="${optionStr// /|}"
138-
139-
# prepare commands & args
140-
runCommand="sleep infinity"
141-
execCommand="/network-observability-cli get-$command ${optionStr:+"--options" "${optionStr}"} --loglevel $logLevel --maxtime $maxTime --maxbytes $maxBytes"
142-
if [[ "$runBackground" == "true" ]]; then
143-
runCommand="$execCommand & $runCommand"
144-
execCommand=""
145-
fi
146-
147-
echo "Running network-observability-cli get-$command... "
148-
${K8S_CLI_BIN} run \
149-
-n $namespace \
150-
collector \
151-
--image=$img\
152-
--image-pull-policy='Always' \
153-
--overrides='{ "spec": { "serviceAccount": "netobserv-cli" } }' \
154-
--restart='Never' \
155-
--command -- $runCommand
156-
157-
${K8S_CLI_BIN} wait \
158-
-n $namespace \
159-
--for=condition=Ready pod/collector || exit 1
160-
161-
captureStarted=true
162-
163-
if [ -n "${execCommand}" ]; then
164-
${K8S_CLI_BIN} exec -i --tty \
171+
if [[ "$command" == "flows" || "$command" == "packets" ]]; then
172+
# convert options to string
173+
optionStr="${options//--/}"
174+
optionStr="${optionStr// /|}"
175+
176+
# prepare commands & args
177+
runCommand="sleep infinity"
178+
execCommand="/network-observability-cli get-$command ${optionStr:+"--options" "${optionStr}"} --loglevel $logLevel --maxtime $maxTime --maxbytes $maxBytes"
179+
if [[ "$runBackground" == "true" ]]; then
180+
runCommand="$execCommand & $runCommand"
181+
execCommand=""
182+
fi
183+
184+
echo "Running network-observability-cli get-$command... "
185+
${K8S_CLI_BIN} run \
165186
-n $namespace \
166187
collector \
167-
-- $execCommand
168-
else
169-
echo "Background capture started. Use:"
170-
echo " - '${K8S_CLI_BIN} netobserv follow' to see the capture progress"
171-
echo " - '${K8S_CLI_BIN} netobserv copy' to copy the generated files locally"
172-
echo " - '${K8S_CLI_BIN} netobserv cleanup' to remove the netobserv components"
188+
--image=$img --image-pull-policy='Always' \
189+
--overrides='{ "spec": { "serviceAccount": "netobserv-cli" } }' \
190+
--restart='Never' \
191+
--command -- $runCommand
192+
193+
${K8S_CLI_BIN} wait \
194+
-n $namespace \
195+
--for=condition=Ready pod/collector || exit 1
196+
197+
captureStarted=true
198+
199+
if [ -n "${execCommand}" ]; then
200+
${K8S_CLI_BIN} exec -i --tty \
201+
-n $namespace \
202+
collector \
203+
-- $execCommand
204+
else
205+
echo "Background capture started. Use:"
206+
echo " - '${K8S_CLI_BIN} netobserv follow' to see the capture progress"
207+
echo " - '${K8S_CLI_BIN} netobserv copy' to copy the generated files locally"
208+
echo " - '${K8S_CLI_BIN} netobserv cleanup' to remove the netobserv components"
209+
fi
210+
elif [ "$command" = "metrics" ]; then
211+
runBackground="true"
212+
echo "Metrics capture started."
213+
consoleURL="$(oc whoami --show-console)"
214+
echo "Open ${consoleURL}/monitoring/dashboards/netobserv-cli to see generated metrics."
215+
echo "Use 'oc netobserv stop' to stop the collection and 'oc netobserv cleanup' to remove everything."
216+
else
217+
echo "Unexpected exception occured on $command"
218+
exit 1
173219
fi

0 commit comments

Comments
 (0)