New virtual hub for more effective debugging #406
Conversation
dlech
left a comment
dlech
left a comment
There was a problem hiding this comment.
Since we dropped support for ev3dev, there isn't much use in having any unix port-based builds anymore. So I would be OK dropping the existing virtualhub and replacing it with this one.
Sounds good. It did get a bit unwieldy with the various virtual variants we have now. I'll wait with merging this until that is done.
Passing the script to run currently works with an environment variable instead of an argument so we didn't have to modify the name |
laurensvalk
force-pushed
the
simulation
branch
3 times, most recently
from
62baff8 to
6aaaeba
Compare
|
This is done now, but I've had to make a few other fixes along the way which should probably be split off to a separate PR since they do affect the real hubs. I'll try and do that tomorrow. It looks like the coverage percentage is technically going down because of this, as it just wasn't measuring most of the |
The existing VirtualHub works well enough for MicroPython coverage testing, but it is too far removed from the other Pybricks ports to be used for everyday debugging. This introduces a new port based on the minimal port for running on Unix systems. Currently supports the REPL and works well with the motor simulation and animation. It uses a simulated Bluetooth driver to provide stdio, closely mimicing the behavior or of the real hubs. Can also be used to debug importing, running mpy files, and so on.
This is not just for arm_none_eabi anymore.
For CI tests, we don't want to wait on the wall clock.
Use CI for fast tests. Locally, viewing the animation only makes sense at normal speed.
Enables shutdown from a user program, including REPL.
Drop-in for HMI module that simulates sending a multi-mpy file to the hub and then starts it. Used in the simulated hub to test different programs. The script to start is set with an environment variable and compiled with Pybricksdev.
Then we don't end up with unexpected build files in the MicroPython submodule when debugging.
This interrupts syscalls like reading data from a pipe, especially when debugging. We can instead just request polling when MicroPython drives the poll hook and the wall clock time has changed. For the CI clock variant, this was not implented at all yet. While we are at it, ensure it increments the clock every N instructions, so that even tight loops without any waits will advance. For example, this would be needed to run asynchronous scripts that don't _WFI anywhere.
Also rename desktop target to native for clarity.
This allows the virtual hub to have a proper main(int argc, char **argv), so we can parse the input argument (a user script).
There are still some processes that need it, so we have to include this for now. This can be removed once all processes have been migrated to pbio/os.
Instead of sending a stream of angle strings, we can just send packets separately. Then we can just use the Pybricks protocol, and drive the animation using data that will eventually become Port View for all hubs.
We can do this since we renamed main in sys/main recently. This is more practical for running tests.
This is needed to run the MicroPython test suite.
Needed for several tests, and this is the closest match to most of our embedded targets.
This has been added to Pybricksdev. We want to avoid using a local script since it makes relative paths complicated when running this from the MicroPython test scripts. Also explicitly pass the last argument, in order to ignore the test suite passing flags such as -X.
This is closer to the real deal.
These used to be artificially shortened on the virtual hub, so never ran in full. Split it in two parts to get under the test timeout instead.
It did not advance, and even if it did it would use its own counter in different places.
This makes more sense as the entry and exit point.
This caused a lot of headaches when the MicroPython test suite runs many instances at the same time.
Instead of using the newly introduced simhub alongside the virtualhub, we'll just replace the old one entirely now that it has reached feature parity. We can keep calling it the virtualhub.
laurensvalk
force-pushed
the
simulation
branch
from
6aaaeba to
3f9b0a3
Compare
| * @param [in] main The main program. | ||
| */ | ||
| int main(int argc, char **argv) { | ||
| void _main(void) { |
There was a problem hiding this comment.
| void _main(void) { | |
| void pbio_main(void) { |
| ifneq ($(PB_MCU_FAMILY),native) | ||
| # Main firmware build targets | ||
| TARGETS := $(BUILD)/firmware.zip | ||
| else | ||
| TARGETS := $(BUILD)/firmware.elf | ||
| endif |
There was a problem hiding this comment.
| ifneq ($(PB_MCU_FAMILY),native) | |
| # Main firmware build targets | |
| TARGETS := $(BUILD)/firmware.zip | |
| else | |
| TARGETS := $(BUILD)/firmware.elf | |
| endif | |
| ifeq ($(PB_MCU_FAMILY),native) | |
| TARGETS := $(BUILD)/firmware.elf | |
| else | |
| TARGETS := $(BUILD)/firmware.zip | |
| endif |
avoiding negation is easier on the brain
|
|
||
| # Run emulation build on a POSIX system using normal stdio | ||
| run: $(BUILD)/firmware.elf | ||
| @$(BUILD)/firmware.elf |
There was a problem hiding this comment.
Would also be nice to have one more recipe to give this a proper executable name, like pybricks-micropython-virtual-hub.
This adds a new build aimed at making the virtualhub more useful for debugging. This variant works much more like the rest of our hubs. For example, it can run multi-mpy files compiled by Pybricksdev. It uses all of
pbio/sys, including things such aspbsys/host. It also supports the REPL. Since the namevirtualhubwas already taken, this simulated build is thesimhub.It can run locally with a wall clock to mimic real life timing, or use an incrementing clock where the embedded builds would
_wfi. This means tests can run instantly on CI instead of waiting for wall time to pass. There isn't a test workflow yet, but it could work just like the one forvirtualhub. We should get more coverage this way, too.We could keep the
virtualhubaround in its current form, primarily if we want to keep running the entire MicroPython unix coverage test suite. Or we could just remove it and call this new variant the virtualhub. Simulation also has a few other interesting use cases, which are planned for a separate update.