Decouple skb metadata tracking from MAC header offset #6412
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust the driver to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data points right past the metadata area. Unlike other drivers, veth calls skb_metadata_set() after eth_type_trans(), which pulls the Ethernet header and moves skb->data. This violates the future calling convention. Adjust the driver to pull the MAC header after calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data already points past the metadata area. Adjust AF_XDP to pull from skb->data before calling skb_metadata_set(). Signed-off-by: Jakub Sitnicki <[email protected]>
Prepare to track skb metadata location independently of MAC header offset. Following changes will make skb_metadata_set() record where metadata ends relative to skb->head. Hence the helper must be called when skb->data points just past the metadata area. Tweak XDP generic mode accordingly. Signed-off-by: Jakub Sitnicki <[email protected]>
Currently skb metadata location is derived from the MAC header offset. This breaks when L2 tunnel/tagging devices (VLAN, GRE, etc.) reset the MAC offset after pulling the encapsulation header, making the metadata inaccessible. A naive fix would be to move metadata on every skb_pull() path. However, we can avoid a memmove on L2 decapsulation if we can locate metadata independently of the MAC offset. Introduce a meta_end field in skb_shared_info to track where metadata ends, decoupling it from mac_header. The new field takes 2 bytes out of the existing 4 byte hole, with structure size unchanged if we reorder the gso_type field. Update skb_metadata_set() to record meta_end at the time of the call, and adjust skb_data_move() and pskb_expand_head() to keep meta_end in sync with head buffer layout. Remove the now-unneeded metadata adjustment in skb_reorder_vlan_header(). Note that this breaks BPF skb metadata access through skb->data_meta when there is a gap between meta_end and skb->data. Following BPF verifier changes address this. Also, we still need to relocate the metadata on encapsulation on forward path. VLAN and QinQ have already been patched when fixing TC BPF helpers [1], but other tagging/tunnel code still requires similar changes. This will be done as a follow up. Signed-off-by: Jakub Sitnicki <[email protected]>
The may_access_direct_pkt_data() helper sets env->seen_direct_write as a side effect, which creates awkward calling patterns: - check_special_kfunc() has a comment warning readers about the side effect - specialize_kfunc() must save and restore the flag around the call Make the helper a pure function by moving the seen_direct_write flag setting to call sites that need it. Signed-off-by: Jakub Sitnicki <[email protected]>
Convert seen_direct_write from a boolean to a bitmap (seen_packet_access) in preparation for tracking additional packet access patterns. No functional change. Signed-off-by: Jakub Sitnicki <[email protected]>
Change gen_prologue() to accept the packet access flags bitmap. This allows gen_prologue() to inspect multiple access patterns when needed. No functional change. Signed-off-by: Jakub Sitnicki <[email protected]>
Introduce PA_F_DATA_META_LOAD flag to track when a BPF program loads the skb->data_meta pointer. This information will be used by gen_prologue() to handle cases where there is a gap between metadata end and skb->data, requiring metadata to be realigned. Signed-off-by: Jakub Sitnicki <[email protected]>
After decoupling metadata location from MAC header offset, a gap can appear between metadata and skb->data on L2 decapsulation (e.g., VLAN, GRE). This breaks the BPF data_meta pointer which assumes metadata is directly before skb->data. Introduce bpf_skb_meta_realign() kfunc to close the gap by moving metadata to immediately precede the MAC header. Inject a call to it in tc_cls_act_prologue() when the verifier detects data_meta access (PA_F_DATA_META_LOAD flag). Update skb_data_move() to handle the gap case: on skb_push(), move metadata to the top of the head buffer; on skb_pull() where metadata is already detached, leave it in place. This restores data_meta functionality for TC programs while keeping the performance benefit of avoiding memmove on L2 decapsulation for programs that don't use data_meta. Signed-off-by: Jakub Sitnicki <[email protected]>
Author
|
Upstream branch: 21f43f4 |
Author
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=1027022 expired. Closing PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: Decouple skb metadata tracking from MAC header offset
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1027022