-
Notifications
You must be signed in to change notification settings - Fork 5
powerpc64/bpf: Add support for bpf arena and arena atomics #5889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
powerpc64/bpf: Add support for bpf arena and arena atomics #5889
Conversation
|
Upstream branch: c9110e6 |
0e26bd3 to
4e70ac0
Compare
|
Upstream branch: abc8a95 |
6ec3579 to
3ad6950
Compare
4e70ac0 to
c04dd5a
Compare
|
Upstream branch: 929adf8 |
3ad6950 to
3ce1754
Compare
c04dd5a to
4754b79
Compare
|
Upstream branch: 2d92ef7 |
3ce1754 to
26232c7
Compare
4754b79 to
af15c58
Compare
|
Upstream branch: b338cf8 |
26232c7 to
daa99f6
Compare
af15c58 to
c979e9f
Compare
|
Upstream branch: 9621eb6 |
daa99f6 to
5e5adb0
Compare
c979e9f to
2d215b0
Compare
|
Upstream branch: e12873e |
5e5adb0 to
d69657f
Compare
2d215b0 to
05282f4
Compare
|
Upstream branch: 93a83d0 |
d69657f to
d65cdaa
Compare
05282f4 to
e2bfc81
Compare
|
Upstream branch: 60ef541 |
d65cdaa to
c5c04f5
Compare
e2bfc81 to
ba991b7
Compare
|
Upstream branch: f859813 |
c5c04f5 to
d420fe3
Compare
ba991b7 to
cf2fd3b
Compare
|
Upstream branch: 5d87e96 |
d420fe3 to
dada97c
Compare
cf2fd3b to
cd9a59e
Compare
|
Upstream branch: a578b54 |
Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] instructions. They are similar to PROBE_MEM instructions with the following differences: - PROBE_MEM32 supports store. - PROBE_MEM32 relies on the verifier to clear upper 32-bit of the src/dst register - PROBE_MEM32 adds 64-bit kern_vm_start address (which is stored in _R26 in the prologue). Due to bpf_arena constructions such _R26 + reg + off16 access is guaranteed to be within arena virtual range, so no address check at run-time. - PROBE_MEM32 allows STX and ST. If they fault the store is a nop. When LDX faults the destination register is zeroed. To support these on powerpc, we do tmp1 = _R26 + src/dst reg and then use tmp1 as the new src/dst register. This allows us to reuse most of the code for normal [LDX | STX | ST]. Additionally, bpf_jit_emit_probe_mem_store() is introduced to emit instructions for storing memory values depending on the size (byte, halfword, word, doubleword). Stack layout is adjusted to introduce a new NVR (_R26) and to make BPF_PPC_STACKFRAME quadword aligned (local_tmp_var is increased by 8 bytes). Reviewed-by: Hari Bathini <[email protected]> Tested-by: Venkat Rao Bagalkote <[email protected]> Signed-off-by: Saket Kumar Bhaskar <[email protected]>
LLVM generates bpf_addr_space_cast instruction while translating pointers between native (zero) address space and __attribute__((address_space(N))). The addr_space=0 is reserved as bpf_arena address space. rY = addr_space_cast(rX, 0, 1) is processed by the verifier and converted to normal 32-bit move: wX = wY. rY = addr_space_cast(rX, 1, 0) : used to convert a bpf arena pointer to a pointer in the userspace vma. This has to be converted by the JIT. PPC_RAW_RLDICL_DOT, a variant of PPC_RAW_RLDICL is introduced to set condition register as well. Reviewed-by: Hari Bathini <[email protected]> Tested-by: Venkat Rao Bagalkote <[email protected]> Signed-off-by: Saket Kumar Bhaskar <[email protected]>
…tructions The existing code for emitting bpf atomic instruction sequences for atomic operations such as XCHG, CMPXCHG, ADD, AND, OR, and XOR has been refactored into a reusable function, bpf_jit_emit_ppc_atomic_op(). It also computes the jump offset and tracks the instruction index for jited LDARX/LWARX to be used in case it causes a fault. Reviewed-by: Hari Bathini <[email protected]> Tested-by: Venkat Rao Bagalkote <[email protected]> Signed-off-by: Saket Kumar Bhaskar <[email protected]>
powerpc supports BPF atomic operations using a loop around Load-And-Reserve(LDARX/LWARX) and Store-Conditional(STDCX/STWCX) instructions gated by sync instructions to enforce full ordering. To implement arena_atomics, arena vm start address is added to the dst_reg to be used for both the LDARX/LWARX and STDCX/STWCX instructions. Further, an exception table entry is added for LDARX/LWARX instruction to land after the loop on fault. At the end of sequence, dst_reg is restored by subtracting arena vm start address. bpf_jit_supports_insn() is introduced to selectively enable instruction support as in other architectures like x86 and arm64. Reviewed-by: Hari Bathini <[email protected]> Tested-by: Venkat Rao Bagalkote <[email protected]> Signed-off-by: Saket Kumar Bhaskar <[email protected]>
dada97c to
c8a9217
Compare
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=998797 irrelevant now. Closing PR. |
Pull request for series with
subject: powerpc64/bpf: Add support for bpf arena and arena atomics
version: 3
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=998797