You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Like many other debugging tools, **bpfvv** may help you better understand **what** is happening with the verification of your BPF program but it is up to you to figure out **why** it is happening.
4
+
1
5
# How to use bpfvv
2
6
3
-
> [!WARNING]
4
-
> The bpfvv app is in early stages of development, and you should expect
5
-
> bugs, UI inconveniences and significant changes from week to week.
6
-
>
7
-
> If you're working with BPF and you think this tool (or a better
8
-
> version of it) would be useful, feel free to use it and don't be shy
9
-
> to report issues and request features via github. Thanks!
7
+
The tool itself is hosted here: https://libbpf.github.io/bpfvv/
10
8
11
-
Go here: https://libbpf.github.io/bpfvv/
9
+
You can load a log by pasting it into the text box or choosing a local file.
12
10
13
-
Load a log by pasting it into the text box or choosing a file.
11
+
You can also use the `url` query parameter to link to a raw log file, for example:
If bpfvv is aware of a helper signature, it knows the number and names of arguments and displays them in the format `name: reg`.
101
+
For known helpers its name is also a link to documentation for that helper.
102
+
81
103
Notice also that the lines not recognized by the parser are greyed
82
104
out. If you notice an unrecognized instruction, please submit a bug
83
105
report.
84
106
85
-
### Subprogram calls
107
+
#### Data dependencies
108
+
109
+
The app computes a use-def analysis [^2] and you can interactively view dependencies between the instructions.
110
+
111
+
The concept is simple. Every instruction may read some slots (registers, stack, memory) and write to others.
112
+
Knowing these it is possible to determine, for a given slot, where its value came from, from what slot, and at what instruction.
113
+
114
+
You can view the results of this analysis by clicking on some instruction operands (registers and stack slots).
115
+
116
+
The selected slot is identified by a box around it. This selection changes the log view, greying out "irrelevant" instructions, and leaving only data-dependent instructions in the foreground.
117
+
118
+
On the left side of the instruction stream are the lines visualizing the dependencies. The lines are interactive and can be used for navigation.
The state panel displays the current state of the program based on the loaded log, with the current state determined by the line selected in the instruction stream view.
96
133
97
-
### Step through the instruction stream
134
+
Remember that the verifier log is a trace through the program.
135
+
This means that a particular instruction may be visited more than once, and the state at the same instruction (but a different point of execution) is usually also different. And so a log line roughly represents a particular point of the program execution, as interpreted by the BPF verifier.
98
136
99
-
The most basic feature of the visualizer is "stepping" through the
100
-
log, similar to what you'd do in a debugger.
137
+
The verifier reports changes in the program state like this:
After the semicolon `;`, there are expressions showing relevant register and stack slot states. The visualizer accumulates this information from all the prior instructions, and in the state panel this accumulated state is displayed.
101
142
102
-
You can select a line by clicking on it, or by navigating with arrows
103
-
(you can also use pgup, pgdown, home and end). The selected line has
104
-
light-blue background.
143
+
The header of the state panel shows the context of the state: log line number, C line number, program counter (PC) and the stack frame index.
105
144
106
-
When a line is selected, current state of known values is displayed in
107
-
the panel on the right. By moving the selected line up/down the log,
108
-
you can see how the values change with each instruction.
145
+
The known values of the registers and stack slots are displayed in a table.
109
146
110
-
In the "state panel", the values that are written by selected
111
-
instruction are marked with light-red background and the previous
112
-
value is also often displayed, for example:
147
+
The background color of a row in the state panel indicates that the relevant value has been affected by the selected instruction.
148
+
Rows marked with red background indicate a "write" and the previous value is also often displayed, for example:
113
149
```
114
150
r6 scalar(id=1) -> 0
115
151
```
116
-
Means that current instruction changes the value of `r6` from
117
-
`scalar(id=1)` to `0`.
152
+
This means that current instruction changes the value of `r6` from `scalar(id=1)` to `0`.
118
153
119
-
The values that are read by current instruction have light-green
120
-
background.
154
+
The values that are read by the current instruction have a blue background.
121
155
122
-
Note that for "update" instructions (such as `r1 += 8`), the slot
123
-
will be marked as written.
156
+
Note that for "update" instructions (such as `r1 += 8`), the slot will be marked as written.
124
157
125
-
#### Sometimes a value of a slot has changed, but it's not highlighted as a write. Is that a bug?
158
+
This then allows you to "step through" the instruction stream and watch how the values are changing, similar to classic debugger UIs.
159
+
You can click on the lines that interest you, or use arrow keys to navigate.
126
160
127
-
Currently the visualizer only considers writes derived from the instructions
128
-
themselves. For example, `r1 = r2` is a write by definition, or a call would
But remember that we are looking at the BPF verifier log. BPF verifier
132
-
simulates execution of a program, which requires maintaining and continuously
133
-
updating a virtual state of the program. This means that whenever the verifier
134
-
gains some knowledge about a value (which is not necesarily a write instruction),
135
-
it will update it.
163
+
#### The rows in the state panel are clickable!
136
164
137
-
For example when processing conditional jumps such as `if (r2 == 0) goto pc+6`,
138
-
the verifier usually explores both branches. But in both cases it gained information
139
-
about r2: it's either 0 or not. And so while there was no explicit write into r2,
140
-
it's value is known (and has changed) after the jump instruction, when you look at
141
-
it in the verifier log.
165
+
It is sometimes useful to jump to the source of a particular slot value from the selected instruction, even if the slot is not relevant to that instruction.
142
166
143
-
Going forward the visualizer will likely treat all value updates as writes,
144
-
as it is useful to know at what point verifier inferred a particular value.
r10 (stack frame pointer) is not clickable because it's effectively a
170
-
constant [^3].
194
+
### The bottom panel
171
195
172
-
Note that the stack slots may be accessed indirectly: if say `r6 = fp-64`
173
-
and then you do `*(u32 *)(r6 -8)` it's equivalent to `*(u32 *)(r10 -72)`.
174
-
The visualizer does not show such dependencies (yet). Although state values
175
-
are tracked correctly.
196
+
The bottom panel shows original log text for the selected line and for the current hovered line.
197
+
It is sometimes useful to check the source of the information displayed by the visualizer.
176
198
177
-
#### How deep is the displayed dependency chain?
178
199
179
-
It depends, but usually not deep.
200
+
## Not frequently asked questions
180
201
181
-
The problem with showing all dependencies is that it's too much
182
-
information, which renders it useless.
202
+
### What exactly do "read" and "written" values means here?
183
203
184
-
Currently the upstream instruction is highlighted if it's an
185
-
unambiguous dependency. For example:
186
-
```
187
-
42: r1 = 13
188
-
43: r7 = 0
189
-
44: r2 = r1
190
-
```
204
+
Here is a couple of trivial examples:
205
+
*`r1 = 0` this is a write to `r1`
206
+
*`r2 = r3` this is a read of `r3` and write to `r2`
207
+
*`r2 += 1` this is a read of `r2` and write to `r2`, aka an update
191
208
192
-
Instruction 42 is an unambiguous dependency of instruction 44, because
193
-
r1 is the only read slot, and there were no modifications to it along
194
-
the way.
209
+
Here is a couple of more complicated examples:
210
+
*`*(u64 *)(r10 -32) = r1` this is a read of `r1` and a write to `fp-32`
211
+
*`r10` is effectively constant[^3], as it is always a pointer to the top of a BPF stack frame, so stores to `r10-offset` are writes to the stack slots, identified by `fp-off` or `fp[frame]-off` in the visualizer
212
+
*`r1 = *(u64 *)(r2 -8)` this is a write to `r1` and a read of `r2`, however it may also be a read of the stack, if `r2` happens to contain a pointer to the stack slot
195
213
196
-
All such direct dependencies up the chain are shown.
214
+
Most instructions have intrinsic "read" and "write" sets, defined by its semantics. However context also matters, as you can see from the last example.
197
215
198
-
However, when more than one value is read in the upstream instruction,
199
-
the UI will stop highlighting at that instruction.
216
+
The visualizer takes into account a few important things, when determining data dependencies:
217
+
* it is aware of scratch and callee-saved register semantics of subprogram/helper calls
218
+
* it is aware of the stack frames: we enter new stack memory in a subprogram, and pop back on exit
219
+
* it is aware of indirect stack slot access and basic pointer arithmetic
200
220
201
-
Consider an example:
202
-
```
203
-
42: r1 = r2
204
-
43: r3 = *(u32 *)(r10 -16)
205
-
44: r1 += r3
206
-
45: *(u32 *)(r10 -64) = r1
207
-
```
221
+
### Side effects?
208
222
209
-
If you select `r1` at instruction 45, only instruction 44 will be
210
-
highlighted, even though 42 and 43 are its transitive dependencies
211
-
(`r1 += r3` reads both `r1` and `r3`).
223
+
One counterintuitive thing about data dependencies in the context of BPF verification is that the instructions which don't do any arithmetic or memory stores can still change the progam state.
212
224
213
-
The reason for this UI behavior is that showing all dependencies (both
214
-
r1 and r3 and in turn all their dependencies) may very quickly cover
215
-
most of the instructions. This is especially true for call
216
-
instructions, which read up to 5 registers.
225
+
Remember, we are looking at the BPF verifier log.
226
+
The BPF verifier simulates the execution of a program, which requires maintaining a virtual state of the program.
227
+
This means that whenever the verifier gains some knowledge about a value (which is not necesarily an intrinsic write instruction), it will update the program state.
217
228
218
-
On the other hand the app can't know what the user is looking for, and
219
-
there is no point in guessing. So, for an instruction like `r1 += r3`,
220
-
the user must choose specific operand (r1 or r3 in this case) to
221
-
expand the dependency chain further.
229
+
For example, when processing conditional jumps such as `if (r2 == 0) goto pc+6`,
230
+
the verifier usually explores both branches. But in both cases it gained information
231
+
about `r2`: it's either 0 or not. And so while there was no explicit write into r2,
232
+
it's value is known (and has changed) after the jump instruction, when you look at
Currently non-stack memory access is a "black hole" from the point of
226
240
view of use-def analysis in this app. The reason is that it's
@@ -235,6 +249,28 @@ dependencies. If you see `*(u32 *)(r8 +0)` down the instruction
235
249
stream, even if value of r8 hasn't changed, the analysis does not
236
250
recognize these slots as "the same".
237
251
252
+
**Unless**`r8` contains a pointer to a stack slot.
253
+
In that case you can click both on the register to see where its value came from, and on the dereference expression to see where the stack slot value came from.
This project is an experiment about visualizing Linux Kernel BPF verifier log to help BPF programmers with debugging verification failures.
7
+
BPF Verifier Visualizer is a tool to analyze Linux Kernel BPF verifier logs.
8
+
9
+
The goal of bpfvv is to help BPF programmers debug verification failures.
16
10
17
11
The user can load a text file, and the app will attempt to parse it as a verifier log. Successfully parsed lines produce a state which is then visualized in the UI. You can think of this as a primitive debugger UI, except it interprets a log and not a runtime state of a program.
18
12
13
+
For more information on how to use **bpfvv** see the [HOWTO.md](https://github.com/libbpf/bpfvv/blob/master/HOWTO.md)
14
+
19
15
## Development
20
16
21
17
- Fork the website repo: https://github.com/libbpf/bpfvv.git
0 commit comments