x86/fgraph,bpf: Fix ORC stack unwind from kprobe_multi #10846
+190
−16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Author
|
Upstream branch: 78980b4 |
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
f416fe6 to
0cc2682
Compare
Author
|
Upstream branch: 95ffdbd |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
182c73a to
f25c57c
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
0cc2682 to
6797dbd
Compare
Author
|
Upstream branch: 95ffdbd |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
f25c57c to
af893bd
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
6797dbd to
91d46f6
Compare
Author
|
Upstream branch: 78980b4 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
af893bd to
11cf628
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
91d46f6 to
9624cf2
Compare
Author
|
Upstream branch: 8016abd |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
11cf628 to
7956bb7
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
9624cf2 to
3a73c9c
Compare
Author
|
Upstream branch: 1456ebb |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
7956bb7 to
92228fe
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
3a73c9c to
aa9aae9
Compare
Author
|
Upstream branch: 35538db |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
92228fe to
99e56f8
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
aa9aae9 to
f94167c
Compare
Author
|
Upstream branch: 35538db |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
99e56f8 to
c6f0506
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
f94167c to
cd8cbf1
Compare
Author
|
Upstream branch: 35538db |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
c6f0506 to
325b029
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
cd8cbf1 to
358bea9
Compare
Author
|
Upstream branch: ae23bc8 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
325b029 to
0f979b5
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
358bea9 to
857ca44
Compare
Author
|
Upstream branch: ba22540 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
0f979b5 to
6809a4a
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
857ca44 to
32dcdce
Compare
Author
|
Upstream branch: 08a7491 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
6809a4a to
6f50b82
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
32dcdce to
ab9b393
Compare
The previous change (Fixes commit) messed up the rsp register value,
which is wrong because it's already adjusted with FRAME_SIZE, we need
the original rsp value.
This change does not affect fprobe current kernel unwind, the !perf_hw_regs
path perf_callchain_kernel:
if (perf_hw_regs(regs)) {
if (perf_callchain_store(entry, regs->ip))
return;
unwind_start(&state, current, regs, NULL);
} else {
unwind_start(&state, current, NULL, (void *)regs->sp);
}
which uses pt_regs.sp as first_frame boundary (FRAME_SIZE shift makes
no difference, unwind stil stops at the right frame).
This change fixes the other path when we want to unwind directly from
pt_regs sp/fp/ip state, which is coming in following change.
Fixes: 20a0bc1 ("x86/fgraph,bpf: Fix stack ORC unwind from kprobe_multi return probe")
Signed-off-by: Jiri Olsa <[email protected]>
Mahe reported missing function from stack trace on top of kprobe
multi program. The missing function is the very first one in the
stacktrace, the one that the bpf program is attached to.
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
('*' is used for kprobe_multi attachment)
The reason is that the previous change (the Fixes commit) fixed
stack unwind for tracepoint, but removed attached function address
from the stack trace on top of kprobe multi programs, which I also
overlooked in the related test (check following patch).
The tracepoint and kprobe_multi have different stack setup, but use
same unwind path. I think it's better to keep the previous change,
which fixed tracepoint unwind and instead change the kprobe multi
unwind as explained below.
The bpf program stack unwind calls perf_callchain_kernel for kernel
portion and it follows two unwind paths based on X86_EFLAGS_FIXED
bit in pt_regs.flags.
When the bit set we unwind from stack represented by pt_regs argument,
otherwise we unwind currently executed stack up to 'first_frame'
boundary.
The 'first_frame' value is taken from regs.rsp value, but ftrace_caller
and ftrace_regs_caller (ftrace trampoline) functions set the regs.rsp
to the previous stack frame, so we skip the attached function entry.
If we switch kprobe_multi unwind to use the X86_EFLAGS_FIXED bit,
we set the start of the unwind to the attached function address.
As another benefit we also cut extra unwind cycles needed to reach
the 'first_frame' boundary.
The speedup can be measured with trigger bench for kprobe_multi
program and stacktrace support.
- trigger bench with stacktrace on current code:
kprobe-multi : 0.810 ± 0.001M/s
kretprobe-multi: 0.808 ± 0.001M/s
- and with the fix:
kprobe-multi : 1.264 ± 0.001M/s
kretprobe-multi: 1.401 ± 0.002M/s
With the fix, the entry probe stacktrace:
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
__x64_sys_newuname+9
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
The return probe skips the attached function, because it's no longer
on the stack at the point of the unwind and this way is the same how
standard kretprobe works.
# bpftrace -e 'kretprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
Fixes: 6d08340 ("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"")
Reported-by: Mahe Tardy <[email protected]>
Signed-off-by: Jiri Olsa <[email protected]>
We now include the attached function in the stack trace, fixing the test accordingly. Fixes: c9e208f ("selftests/bpf: Add stacktrace ips test for kprobe_multi/kretprobe_multi") Signed-off-by: Jiri Olsa <[email protected]>
Adding test that attaches kprobe/kretprobe and verifies the ORC stacktrace matches expected functions. The test is only for ORC unwinder to keep it simple. Signed-off-by: Jiri Olsa <[email protected]>
Adding test that attaches fentry/fexitand verifies the ORC stacktrace matches expected functions. The test is only for ORC unwinder to keep it simple. Signed-off-by: Jiri Olsa <[email protected]>
Adding support to call bpf_get_stackid helper from trigger programs, so far added for kprobe multi. Adding the --stacktrace/-g option to enable it. Signed-off-by: Jiri Olsa <[email protected]>
Author
|
Upstream branch: 08a7491 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/1047287=>bpf-next
branch
from
6f50b82 to
fd30651
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: x86/fgraph,bpf: Fix ORC stack unwind from kprobe_multi
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1047287