Add kfunc bpf_xdp_pull_data #5908
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Author
|
Upstream branch: 9621eb6 |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
bpf-next_base
branch
from
c979e9f to
2d215b0
Compare
Author
|
Upstream branch: e12873e |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
series/999464=>bpf-next
branch
from
1395568 to
0ccd1a9
Compare
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
bpf-next_base
branch
from
2d215b0 to
05282f4
Compare
Author
|
Upstream branch: 93a83d0 |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
series/999464=>bpf-next
branch
from
0ccd1a9 to
7590c0c
Compare
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
bpf-next_base
branch
from
05282f4 to
e2bfc81
Compare
Author
|
Upstream branch: 60ef541 |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
series/999464=>bpf-next
branch
from
7590c0c to
cff24a2
Compare
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
bpf-next_base
branch
from
e2bfc81 to
ba991b7
Compare
xdp programs can change the layout of an xdp_buff through bpf_xdp_adjust_tail() and bpf_xdp_adjust_head(). Therefore, the driver cannot assume the size of the linear data area nor fragments. Fix the bug in mlx5 by generating skb according to xdp_buff after xdp programs run. Currently, when handling multi-buf xdp, the mlx5 driver assumes the layout of an xdp_buff to be unchanged. That is, the linear data area continues to be empty and fragments remains the same. This may cause the driver to generate erroneous skb or triggering a kernel warning. When an xdp program added linear data through bpf_xdp_adjust_head(), the linear data will be ignored as mlx5e_build_linear_skb() builds an skb without linear data and then pull data from fragments to fill the linear data area. When an xdp program has shrunk the non-linear data through bpf_xdp_adjust_tail(), the delta passed to __pskb_pull_tail() may exceed the actual nonlinear data size and trigger the BUG_ON in it. To fix the issue, first record the original number of fragments. If the number of fragments changes after the xdp program runs, rewind the end fragment pointer by the difference and recalculate the truesize. Then, build the skb with linear data area matching the xdp_buff. Finally, only pull data in if there is non-linear data and fill the linear part up to 256 bytes. Fixes: f52ac70 ("net/mlx5e: RX, Add XDP multi-buffer support in Striding RQ") Signed-off-by: Amery Hung <[email protected]>
Move skb_frag_t adjustment into bpf_xdp_shrink_data() and extend its functionality to be able to shrink an xdp fragment from both head and tail. In a later patch, bpf_xdp_pull_data() will reuse it to shrink an xdp fragment from head. Additionally, in bpf_xdp_frags_shrink_tail(), breaking the loop when bpf_xdp_shrink_data() returns false (i.e., not releasing the current fragment) is not necessary as the loop condition, offset > 0, has the same effect. Remove the else branch to simplify the code. Signed-off-by: Amery Hung <[email protected]>
Add kfunc, bpf_xdp_pull_data(), to support pulling data from xdp fragments. Similar to bpf_skb_pull_data(), bpf_xdp_pull_data() makes the first len bytes of data directly readable and writable in bpf programs. If the "len" argument is larger than the linear data size, data in fragments will be copied to the linear region when there is enough room between xdp->data_end and xdp_data_hard_end(xdp), which is subject to driver implementation. A use case of the kfunc is to decapsulate headers residing in xdp fragments. It is possible for a NIC driver to place headers in xdp fragments. To keep using direct packet access for parsing and decapsulating headers, users can pull headers into the linear data area by calling bpf_xdp_pull_data() and then pop the header with bpf_xdp_adjust_head(). An unused argument, flags is reserved for future extension (e.g., tossing the data instead of copying it to the linear data area). Signed-off-by: Amery Hung <[email protected]>
bpf_xdp_pull_data() may change packet data and therefore packet pointers need to be invalidated. Add bpf_xdp_pull_data() to the special kfunc list instead of introducing a new KF_ flag until there are more kfuncs changing packet data. Signed-off-by: Amery Hung <[email protected]>
To test bpf_xdp_pull_data(), an xdp packet containing fragments as well as free linear data area after xdp->data_end needs to be created. However, bpf_prog_test_run_xdp() always fills the linear area with data_in before creating fragments, leaving no space to pull data. This patch will allow users to specify the linear data size through ctx->data_end. Currently, ctx_in->data_end must match data_size_in and will not be the final ctx->data_end seen by xdp programs. This is because ctx->data_end is populated according to the xdp_buff passed to test_run. The linear data area available in an xdp_buff, max_data_sz, is alawys filled up before copying data_in into fragments. This patch will allow users to specify the size of data that goes into the linear area. When ctx_in->data_end is different from data_size_in, only ctx_in->data_end bytes of data will be put into the linear area when creating the xdp_buff. While ctx_in->data_end will be allowed to be different from data_size_in, it cannot be larger than the data_size_in as there will be no data to copy from user space. If it is larger than the maximum linear data area size, the layout suggested by the user will not be honored. Data beyond max_data_sz bytes will still be copied into fragments. Signed-off-by: Amery Hung <[email protected]>
Test bpf_xdp_pull_data() with xdp packets with different layouts. The xdp bpf program first checks if the layout is as expected. Then, it calls bpf_xdp_pull_data(). Finally, it checks the 0xbb marker at offset 1024 using directly packet access. Signed-off-by: Amery Hung <[email protected]>
It is possible for drivers to generate xdp packets with data residing entirely in fragments. To keep parsing headers using direcy packet access, call bpf_xdp_pull_data() to pull headers into the linear data area. Signed-off-by: Amery Hung <[email protected]>
Author
|
Upstream branch: f859813 |
kernel-patches-daemon-bpf-rc
bot
force-pushed
the
series/999464=>bpf-next
branch
from
cff24a2 to
17c1083
Compare
Author
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=999464 expired. Closing PR. |
kernel-patches-daemon-bpf-rc
bot
deleted the
series/999464=>bpf-next
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: Add kfunc bpf_xdp_pull_data
version: 2
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=999464