Skip to content

Conversation

@roypat
Copy link
Contributor

@roypat roypat commented Jun 17, 2025

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

@codecov
Copy link

codecov bot commented Jun 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.90%. Comparing base (5c520bc) to head (92e8ec7).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5266      +/-   ##
==========================================
+ Coverage   82.84%   82.90%   +0.05%     
==========================================
  Files         250      250              
  Lines       26967    26967              
==========================================
+ Hits        22342    22356      +14     
+ Misses       4625     4611      -14     
Flag Coverage Δ
5.10-c5n.metal 83.33% <ø> (ø)
5.10-m5n.metal 83.33% <ø> (+<0.01%) ⬆️
5.10-m6a.metal 82.54% <ø> (ø)
5.10-m6g.metal 79.16% <ø> (ø)
5.10-m6i.metal 83.32% <ø> (+<0.01%) ⬆️
5.10-m7a.metal-48xl 82.53% <ø> (?)
5.10-m7g.metal 79.16% <ø> (ø)
5.10-m7i.metal-24xl 83.29% <ø> (?)
5.10-m7i.metal-48xl 83.30% <ø> (?)
5.10-m8g.metal-24xl 79.16% <ø> (?)
5.10-m8g.metal-48xl 79.16% <ø> (?)
6.1-c5n.metal 83.38% <ø> (ø)
6.1-m5n.metal 83.37% <ø> (ø)
6.1-m6a.metal 82.59% <ø> (ø)
6.1-m6g.metal 79.16% <ø> (+<0.01%) ⬆️
6.1-m6i.metal 83.36% <ø> (-0.01%) ⬇️
6.1-m7a.metal-48xl 82.58% <ø> (?)
6.1-m7g.metal 79.16% <ø> (ø)
6.1-m7i.metal-24xl 83.39% <ø> (?)
6.1-m7i.metal-48xl 83.39% <ø> (?)
6.1-m8g.metal-24xl 79.16% <ø> (?)
6.1-m8g.metal-48xl 79.16% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@roypat roypat added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Jun 17, 2025
kalyazin
kalyazin previously approved these changes Jun 17, 2025
@Manciukic
Copy link
Contributor

The optional tests are failing with some weird errors:

import file mismatch:
imported module 'test_vsock' has this __file__ attribute:
  /firecracker/tests/integration_tests/functional/test_vsock.py
which is not the same as the test file we want to collect:
  /firecracker/tests/integration_tests/performance/test_vsock.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

roypat added 4 commits June 18, 2025 07:46
So we track the metric better

Signed-off-by: Patrick Roy <[email protected]>
There is nothing that makes these tests specifically "ab" - any test can
be run as an A/B-test as long as it produces more than one data point
for a metric. So drop this suffix. It was only added originally when we
transitioned to A/B-testing and the A/B-capable versions of the tests
had to co-exist with the old baseline version of the tests in separate
files.

Signed-off-by: Patrick Roy <[email protected]>
We have a few tests that are parametrized to run with both full and diff
snapshots enabled, so let's move this into a fixture.

Signed-off-by: Patrick Roy <[email protected]>
Diff snapshots create sparse files, so they should be a lot faster than
full snapshots.

Signed-off-by: Patrick Roy <[email protected]>
@roypat roypat force-pushed the snapshot-creation-perf-tests-improvements branch from 7c4e913 to 65577a4 Compare June 18, 2025 06:46
We were seeing "import mismatch error" due to files in functional and
performance having the same name. According to [1], a fix is to just
have __init__.py files.

[1]: https://stackoverflow.com/questions/53918088/import-file-mismatch-in-pytest

Signed-off-by: Patrick Roy <[email protected]>
@roypat roypat force-pushed the snapshot-creation-perf-tests-improvements branch from 65577a4 to cf7dfbe Compare June 18, 2025 07:02
@roypat
Copy link
Contributor Author

roypat commented Jun 18, 2025

The optional tests are failing with some weird errors:

import file mismatch:
imported module 'test_vsock' has this __file__ attribute:
  /firecracker/tests/integration_tests/functional/test_vsock.py
which is not the same as the test file we want to collect:
  /firecracker/tests/integration_tests/performance/test_vsock.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

Fixed!

@roypat roypat requested a review from kalyazin June 18, 2025 08:36
@roypat roypat merged commit 797faa8 into firecracker-microvm:main Jun 18, 2025
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants