Skip to content

Commit 5b2b7de

Browse files
edumazetkuba-moo
authored andcommitted
net: add add indirect call wrapper in skb_release_head_state()
While stress testing UDP senders on a host with expensive indirect calls, I found cpus processing TX completions where showing a very high cost (20%) in sock_wfree() due to CONFIG_MITIGATION_RETPOLINE=y. Take care of TCP and UDP TX destructors and use INDIRECT_CALL_3() macro. Signed-off-by: Eric Dumazet <[email protected]> Reviewed-by: Kuniyuki Iwashima <[email protected]> Reviewed-by: Toke Høiland-Jørgensen <[email protected]> Tested-by: Jamal Hadi Salim <[email protected]> Acked-by: Jamal Hadi Salim <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
1 parent 56cef47 commit 5b2b7de

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

net/core/skbuff.c

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1136,7 +1136,16 @@ void skb_release_head_state(struct sk_buff *skb)
11361136
skb_dst_drop(skb);
11371137
if (skb->destructor) {
11381138
DEBUG_NET_WARN_ON_ONCE(in_hardirq());
1139-
skb->destructor(skb);
1139+
#ifdef CONFIG_INET
1140+
INDIRECT_CALL_3(skb->destructor,
1141+
tcp_wfree, __sock_wfree, sock_wfree,
1142+
skb);
1143+
#else
1144+
INDIRECT_CALL_1(skb->destructor,
1145+
sock_wfree,
1146+
skb);
1147+
1148+
#endif
11401149
}
11411150
#if IS_ENABLED(CONFIG_NF_CONNTRACK)
11421151
nf_conntrack_put(skb_nfct(skb));

0 commit comments

Comments
 (0)