Skip to content

Conversation

Manciukic
Copy link
Contributor

Changes

We noticed that, compared to main, the vsock device is up to 30% slower
when PCI is disabled. This is due to the refactor in 087e185 ("fix(vsock):
pass correct index when triggering interrupts"), as we're now sending
interrupts as soon as we detect the event, rather than deferring them
and sending only one interrupt at the end.
While with MSI we need to send multiple interrupts, so there is little
difference (still, tests show up to 5% improvement with this change),
with legacy IRQ there's only one interrupt line so we end up sending
multiple back to back interrupts rather than a single one.
This patch reverts to the previous behaviour and uses the newly
introduced trigger_queues method to deduplicate interrupts in the case
of IrqTrigger.

Reason

Improve performance of vsock device with PCI disabled by 30% and enabled by 5%.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

@Manciukic Manciukic force-pushed the pcie/vsock-perf-fix branch 3 times, most recently from d3e31fa to 409a567 Compare August 4, 2025 13:36
Copy link

codecov bot commented Aug 4, 2025

Codecov Report

❌ Patch coverage is 73.33333% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.64%. Comparing base (97eaa9e) to head (db31697).
⚠️ Report is 2 commits behind head on feature/pcie.

Files with missing lines Patch % Lines
src/vmm/src/devices/virtio/vsock/event_handler.rs 74.07% 7 Missing ⚠️
src/vmm/src/devices/virtio/transport/mod.rs 0.00% 5 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##           feature/pcie    #5343      +/-   ##
================================================
- Coverage         80.68%   80.64%   -0.05%     
================================================
  Files               265      265              
  Lines             30650    30675      +25     
================================================
+ Hits              24731    24738       +7     
- Misses             5919     5937      +18     
Flag Coverage Δ
5.10-c5n.metal 80.43% <73.33%> (-0.05%) ⬇️
5.10-m5n.metal 80.42% <73.33%> (-0.06%) ⬇️
5.10-m6a.metal 79.63% <73.33%> (-0.06%) ⬇️
5.10-m6g.metal 76.94% <73.33%> (-0.05%) ⬇️
5.10-m6i.metal 80.42% <73.33%> (-0.06%) ⬇️
5.10-m7a.metal-48xl 79.63% <73.33%> (-0.05%) ⬇️
5.10-m7g.metal 76.94% <73.33%> (-0.05%) ⬇️
5.10-m7i.metal-24xl 80.39% <73.33%> (-0.05%) ⬇️
5.10-m7i.metal-48xl 80.39% <73.33%> (-0.05%) ⬇️
5.10-m8g.metal-24xl 76.94% <73.33%> (-0.05%) ⬇️
5.10-m8g.metal-48xl 76.94% <73.33%> (-0.05%) ⬇️
6.1-c5n.metal 80.47% <73.33%> (-0.05%) ⬇️
6.1-m5n.metal 80.47% <73.33%> (-0.05%) ⬇️
6.1-m6a.metal 79.68% <73.33%> (-0.05%) ⬇️
6.1-m6g.metal 76.94% <73.33%> (-0.05%) ⬇️
6.1-m6i.metal 80.46% <73.33%> (-0.05%) ⬇️
6.1-m7a.metal-48xl 79.66% <73.33%> (-0.06%) ⬇️
6.1-m7g.metal 76.94% <73.33%> (-0.05%) ⬇️
6.1-m7i.metal-24xl 80.47% <73.33%> (-0.06%) ⬇️
6.1-m7i.metal-48xl 80.48% <73.33%> (-0.05%) ⬇️
6.1-m8g.metal-24xl 76.94% <73.33%> (?)
6.1-m8g.metal-48xl 76.94% <73.33%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Manciukic Manciukic changed the title Pcie/vsock perf fix [PCIe] Performance fix for vsock device with pci disabled Aug 4, 2025
@Manciukic Manciukic force-pushed the pcie/vsock-perf-fix branch from 409a567 to fc24d8d Compare August 4, 2025 15:55
@Manciukic Manciukic marked this pull request as ready for review August 4, 2025 16:54
@Manciukic Manciukic added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Aug 4, 2025
This method is used to notify the guest about queue events in a way
that's most performant with the underlying interrupt implementation.
As in IrqTrigger there is no distinction between different queues, it's
best to send just one interrupt notifiying that "some queues" have a
pending event.
Conversely, in VirtioInterruptMsix, we need to trigger a MSI for each
distinct queue.

Signed-off-by: Riccardo Mancini <[email protected]>
We noticed that, compared to main, the vsock device is up to 30% slower
when PCI is disabled. This is due to the refactor in 087e185
("fix(vsock): pass correct index when triggering interrupts"), as we're
now sending interrupts as soon as we detect the event, rather than
deferring them and sending only one interrupt at the end.
While with MSI we need to send multiple interrupts, so there is little
difference (still, tests show up to 5% improvement with this change),
with legacy IRQ there's only one interrupt line so we end up sending
multiple back to back interrupts rather than a single one.
This patch reverts to the previous behaviour and uses the newly
introduced `trigger_queues` method to deduplicate interrupts in the case
of IrqTrigger.

Signed-off-by: Riccardo Mancini <[email protected]>
@Manciukic Manciukic force-pushed the pcie/vsock-perf-fix branch from fc24d8d to db31697 Compare August 5, 2025 08:24
@ShadowCurse ShadowCurse merged commit c9b0e64 into firecracker-microvm:feature/pcie Aug 5, 2025
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants