Integrate NVidia patches to improve IPSec performance with hw offload #171
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for contributing to our kernel. I appreciate following everything in the wiki for contribution guidelines. I have a couple asks though: Do you have testing that you can publicly share that validates that thes changes do what is expected? |
commit-author Jianbo Liu <[email protected]> commit 4861333 Add NETIF_F_GSO_ESP bit to bond's gso_partial_features if all slaves support it, such that ESP segmentation is handled by hardware if possible. Signed-off-by: Jianbo Liu <[email protected]> Reviewed-by: Boris Pismenny <[email protected]> Signed-off-by: Tariq Toukan <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]> (cherry picked from commit 4861333) Signed-off-by: Srinivas Neginhal <[email protected]>
commit-author Cosmin Ratiu <[email protected]> commit 9e6c4e6 The referenced fix is incomplete. It correctly computes bond_dev->gso_partial_features across slaves, but unfortunately netdev_fix_features discards gso_partial_features from the feature set if NETIF_F_GSO_PARTIAL isn't set in bond_dev->features. This is visible with ethtool -k bond0 | grep esp: tx-esp-segmentation: off [requested on] esp-hw-offload: on esp-tx-csum-hw-offload: on This patch reworks the bonding GSO offload support by: - making aggregating gso_partial_features across slaves similar to the other feature sets (this part is a no-op). - advertising the default partial gso features on empty bond devs, same as with other feature sets (also a no-op). - adding NETIF_F_GSO_PARTIAL to hw_enc_features filtered across slaves. - adding NETIF_F_GSO_PARTIAL to features in bond_setup() With all of these, 'ethtool -k bond0 | grep esp' now reports: tx-esp-segmentation: on esp-hw-offload: on esp-tx-csum-hw-offload: on Fixes: 4861333 ("bonding: add ESP offload features when slaves support") Signed-off-by: Hangbin Liu <[email protected]> Signed-off-by: Cosmin Ratiu <[email protected]> Acked-by: Jay Vosburgh <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Paolo Abeni <[email protected]> (cherry picked from commit 9e6c4e6) Signed-off-by: Srinivas Neginhal <[email protected]>
commit-author Cosmin Ratiu <[email protected]> commit 0aae286 The cited commit fixed a software GSO bug with VXLAN + IPSec in tunnel mode. Unfortunately, it is slightly broader than necessary, as it also severely affects performance for Geneve + IPSec transport mode over a device capable of both HW GSO and IPSec crypto offload. In this case, xfrm_output unnecessarily triggers software GSO instead of letting the HW do it. In simple iperf3 tests over Geneve + IPSec transport mode over a back-2-back pair of NICs with MTU 1500, the performance was observed to be up to 6x worse when doing software GSO compared to leaving it to the hardware. This commit makes xfrm_output only trigger software GSO in crypto offload cases for already encapsulated packets in tunnel mode, as not doing so would then cause the inner tunnel skb->inner_networking_header to be overwritten and break software GSO for that packet later if the device turns out to not be capable of HW GSO. Taking a closer look at the conditions for the original bug, to better understand the reasons for this change: - vxlan_build_skb -> iptunnel_handle_offloads sets inner_protocol and inner network header. - then, udp_tunnel_xmit_skb -> ip_tunnel_xmit adds outer transport and network headers. - later in the xmit path, xfrm_output -> xfrm_outer_mode_output -> xfrm4_prepare_output -> xfrm4_tunnel_encap_add overwrites the inner network header with the one set in ip_tunnel_xmit before adding the second outer header. - __dev_queue_xmit -> validate_xmit_skb checks whether GSO segmentation needs to happen based on dev features. In the original bug, the hw couldn't segment the packets, so skb_gso_segment was invoked. - deep in the .gso_segment callback machinery, __skb_udp_tunnel_segment tries to use the wrong inner network header, expecting the one set in iptunnel_handle_offloads but getting the one set by xfrm instead. - a bit later, ipv6_gso_segment accesses the wrong memory based on that wrong inner network header. With the new change, the original bug (or similar ones) cannot happen again, as xfrm will now trigger software GSO before applying a tunnel. This concern doesn't exist in packet offload mode, when the HW adds encapsulation headers. For the non-offloaded packets (crypto in SW), software GSO is still done unconditionally in the else branch. Reviewed-by: Dragos Tatulea <[email protected]> Reviewed-by: Yael Chemla <[email protected]> Reviewed-by: Leon Romanovsky <[email protected]> Fixes: a204aef ("xfrm: call xfrm_output_gso when inner_protocol is set in xfrm_output") Signed-off-by: Cosmin Ratiu <[email protected]> Signed-off-by: Steffen Klassert <[email protected]> (cherry picked from commit 0aae286) Signed-off-by: Srinivas Neginhal <[email protected]>
commit-author Jianbo Liu <[email protected]> commit 373b79a Previously xfrm_dev_state_advance_esn() was added for RX only. But it's possible that ESN context also need to be synced to hardware for TX, so call it for outbound in this patch. Signed-off-by: Jianbo Liu <[email protected]> Signed-off-by: Leon Romanovsky <[email protected]> Signed-off-by: Steffen Klassert <[email protected]> (cherry picked from commit 373b79a) Signed-off-by: Srinivas Neginhal <[email protected]>
sneginha
force-pushed
the
sneginhal-fips-9-compliant/5.14.0-284.30.1
branch
from
03b404b to
24b7dbb
Compare
Done.
Testing done:
|
PlaidCat
approved these changes
Mar 20, 2025
There was a problem hiding this comment.
Thank You for the update to the commits headers.
Do you have testing that you can publicly share that validates that thes changes do what is expected?
Testing done:
1. Run iperf TCP performance tests
2. Review packet captures on the tx side
3. Ensure that you see large MSS sized packets going to the NIC.
4. Also, note the increase in network performance.
And these do not impact when offloading is off, I'm assuming everything is fine so I'm going to send my approval.
PlaidCat
merged commit 636827e
into
ctrliq:fips-9-compliant/5.14.0-284.30.1
2 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Commits
Cherry-picking 4 patches from upstream:
bonding: add ESP offload features when slaves support
This patch fixes a perf issue in the bonding driver where ESP segmentation was not being offloaded to the slave devices.
bonding: Correctly support GSO ESP offload
The fix in patch 1 is incomplete and this change fixes the ESP segmentation issue.
xfrm_output: Force software GSO only in tunnel mode
Fix in xfrm_output to only trigger software GSO in crypto offload cases for already encapsulated packets in tunnel mode.
xfrm: Support ESN context update to hardware for TX
Previously xfrm_dev_state_advance_esn() was added for RX only in commit 50bd870, but is needed for TX as well.
Build
kABI Check
Reboot and Verify
Kernel Self Tests
kselftest_output.txt