Skip to content

Conversation

bchalios
Copy link
Contributor

@bchalios bchalios commented Sep 2, 2024

Changes

Increase the timeout for kani test from 30 to 40 minutes.

Reason

Some changes that we did in the VirtIO queue in this PR: #4726 pushed kani proofs close to the time limit we set in the CI.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • If a specific issue led to this PR, this PR closes the issue.
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this
    PR.
  • API changes follow the Runbook for Firecracker API changes.
  • User-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.
  • New TODOs link to an issue.
  • Commits meet
    contribution quality standards.

  • This functionality cannot be added in rust-vmm.

@bchalios bchalios added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Sep 2, 2024
Copy link
Contributor

@roypat roypat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also pass a timeout to utils.check_output (say of 20 seconds less than the timeout applied by pytest), so that if we do get a timeout again, it'll actually print the kani stdout which will tell us what specific harness timed out

Copy link

codecov bot commented Sep 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.34%. Comparing base (9e627a7) to head (c633fd0).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4765   +/-   ##
=======================================
  Coverage   84.34%   84.34%           
=======================================
  Files         249      249           
  Lines       27460    27460           
=======================================
  Hits        23160    23160           
  Misses       4300     4300           
Flag Coverage Δ
5.10-c5n.metal 84.56% <ø> (ø)
5.10-m5n.metal 84.54% <ø> (-0.01%) ⬇️
5.10-m6a.metal 83.84% <ø> (+<0.01%) ⬆️
5.10-m6g.metal 80.91% <ø> (ø)
5.10-m6i.metal 84.54% <ø> (+<0.01%) ⬆️
5.10-m7g.metal 80.91% <ø> (ø)
6.1-c5n.metal 84.56% <ø> (+<0.01%) ⬆️
6.1-m5n.metal 84.54% <ø> (ø)
6.1-m6a.metal 83.83% <ø> (-0.01%) ⬇️
6.1-m6g.metal 80.90% <ø> (-0.01%) ⬇️
6.1-m6i.metal 84.54% <ø> (ø)
6.1-m7g.metal 80.91% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bchalios
Copy link
Contributor Author

bchalios commented Sep 2, 2024

Let's also pass a timeout to utils.check_output (say of 20 seconds less than the timeout applied by pytest), so that if we do get a timeout again, it'll actually print the kani stdout which will tell us what specific harness timed out

Does it make sense to only apply the timeout in check_output then, @roypat?

@bchalios bchalios force-pushed the increase_kani_time_limit branch from df61bd1 to 1b64879 Compare September 2, 2024 10:18
Some changes that we did in the VirtIO queue in this PR:
firecracker-microvm#4726 pushed kani
proofs close to the time limit we set in the CI. This commit just
increases the timeout from 30 to 40 minutes.

Signed-off-by: Babis Chalios <[email protected]>
@bchalios bchalios force-pushed the increase_kani_time_limit branch from 1b64879 to c633fd0 Compare September 2, 2024 10:23
@roypat
Copy link
Contributor

roypat commented Sep 2, 2024

Let's also pass a timeout to utils.check_output (say of 20 seconds less than the timeout applied by pytest), so that if we do get a timeout again, it'll actually print the kani stdout which will tell us what specific harness timed out

Does it make sense to only apply the timeout in check_output then, @roypat?

If we don't set a pytest timeout, the default of 180s will apply, so we need to overwrite it explicitly :(

@bchalios bchalios merged commit b8ca08e into firecracker-microvm:main Sep 2, 2024
5 checks passed
@bchalios bchalios deleted the increase_kani_time_limit branch September 2, 2024 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants