-
Notifications
You must be signed in to change notification settings - Fork 2k
Add option to redirect guest serial console output to file #5350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5350 +/- ##
==========================================
+ Coverage 82.27% 82.29% +0.01%
==========================================
Files 265 266 +1
Lines 30585 30619 +34
==========================================
+ Hits 25165 25199 +34
Misses 5420 5420
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
15dd953
to
78977d3
Compare
0eedf63
to
c4665a6
Compare
aa06cf0
to
c91280a
Compare
1382aca
to
1255df3
Compare
7f7e4df
to
5dd94ee
Compare
e46633c
to
049d1b3
Compare
2b363cb
to
926e8d3
Compare
This type is only useful if we are dynamically dispatching io via the Read/Write traits, but we are not doing that (we converted everything to static dispatch a while back, for better or for worse, and to avoid conflicts with the PCI branch, I'm not reverting that here). Signed-off-by: Patrick Roy <[email protected]>
The `event_observer` field in the Vmm struct is of type Option<Stdin>. There are two problems 1. With the way the code is written, it will never be `None` 2. `Stdin` is a singleton, there is no need to store it _anywhere_. With that in mind, we can just remove this field, and update its two uses to just directly operate on std::io::stdin(). Since it never `None', we can also remove the logic that matches and handles the `None` case. Furthermore, the `Drop` impl used to print the same error message twice in case resetting stdin to canonical mode failed, so fix that to only print it once. Signed-off-by: Patrick Roy <[email protected]>
We always called it right before setup_serial_device(), so just move the call inside. This will also be helpful once setup_serial_device() can setup the serial device to print to a file, where we dont need to mess with stdout in the first place. Signed-off-by: Patrick Roy <[email protected]>
Have teh aarch64 serial device restoration path use the common serial device initialization code in setup_serial_device(). The only functional change here is that now we do set_stdou_nonblocking() on the restore path, although I would wager that not doing so in the past was actually an oversight, because we do it for all booted VMs, and for restored VMs on x86. Signed-off-by: Patrick Roy <[email protected]>
we only write to it, so no need to pass .read(true). Signed-off-by: Patrick Roy <[email protected]>
Add a new field, `serial_out_path`, to the logger configuration (available both via API and CLI) which can be set to the path of a file into which the output of the guest's serial console should be dumped. Have the file behave identically to our log file (e.g. it must already exist, Firecracker does not create it). If we redirect serial output to a log file, disable serial input via stdin. Signed-off-by: Patrick Roy <[email protected]>
Copy the screen log file (which contains firecracker's stdout/guest dmesg if non-daemonized / no log file specified) to the test_results directory, to be included in artifacts, instead of explicitly reading one file and subsequently writing another file in Python code. Signed-off-by: Patrick Roy <[email protected]>
If SSH to guest is failing, we don't really get much useful in terms of logs in our ci artifacts. Enable serial console so that we get guest dmesg always, at least if we have the API server around (which is the case for the vast majority of tests). Since serial console is enabled by default now, this means we can stop explicitly passing modified bootargs in a bunch of tests now. Disable it for the tests that interact with the serial console via stdin/stdout (here we are not daemonizing, so we are capturing serial output anyway). Disable pylint's "too-many-statements" lint, because it started firing in microvm.spawn() Signed-off-by: Patrick Roy <[email protected]>
Add a test that verifies having serial output go to a file works. Signed-off-by: Patrick Roy <[email protected]>
In test_serial_console_login, we were implementing something that takes three lines (send and receive data via serial) via an overcomplicated state machine system. Eliminate the 100 lines of state machine code and instead just... do the 3 lines of serial.tx/rx. Signed-off-by: Patrick Roy <[email protected]>
If a test fails, take a snapshot fo the microvms involved, and upload it as a CI artifact, for post-mortem analysis. Signed-off-by: Patrick Roy <[email protected]>
These have only caused confusion, as they're output often makes no sense, and we generally just end up ignoring them (but have discussions about it every single time). Let's remove them. Signed-off-by: Patrick Roy <[email protected]>
When A/B-testing, the "A" revision firecracker binary does not know about the new --serial-out-path CLI argument. Thus, disable the serial console for all A/B-tests. It gets quite ugly to do for the vulnerability tests, because the .spawn() call is burried deep in the fixtures. This commit should be reverted after merging the PR Signed-off-by: Patrick Roy <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great to me. Just a comment regarding the state machine dropping
Add a new logger sub-configuration to specify a file into which guest serial console output should be written (instead of stdout).
Reason
Getting serial output from our integration tests would make debugging intermittent failures where SSH didn't work / the guest became unresponsive a lot easier
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md
.PR Checklist
tools/devtool checkbuild --all
to verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyle
to verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md
.Runbook for Firecracker API changes.
integration tests.
TODO
.rust-vmm
.