Guidance on modifying Micropython for upcoming Pimoroni Yukon board

[ZodiusInfuser]

Hi! I am a dev at Pimoroni, working on an upcoming RP2040-powered robot controller called Yukon (https://www.tomshardware.com/news/pimoroni-stealthily-announce-rp2040-yukon-modular-robotics-platform), and am looking for some guidance in modifying Micropython for it.

Yukon is designed for high power (well higher than we have done before), supporting 30A peak inputs at 17, and features interchangeable modules to suit people's different project needs. As such it not only uses all of the RP2040's GPIO, it also have two I2C IO expanders, and ADC muxes onboard to give the number of IO necessary. One of its key features is various voltage, current, and temperature sensors onboard letting users monitor their system and turn off the power mosfet in the event of a fault.

Initial firmware development for the product took place with CircuitPython, as that was the one we had least control over, so likely the hardest to get the required functionality working. In fact it turned out to be easier, and came with some surprises that made for a nice user experience. Namely the board initialisation/resetting letting the IO be automatically configured, and the board definition system, letting the various IO be grouped into their respective slots. It also appears that the stop command from Thonny causes the board to soft reset, causing all the IO to be put back into a safe state.

I am now at the point of wanting to replicate the firmware in Micropython. As you may be aware, Pimoroni create their own MPy builds for products, so I have lot of flexibility in what I can change, though some of my planned changes could be worth upstreaming to benefit everybody. Unfortunately I am having difficulty identifying what areas of code I need to modify (and how) in order to get the behaviour I need.

Here is a list of what I am currently aware of:

• Initialisation of the I2C bus at power up for communicating with the IO expanders
• Board initialisation and resetting to put all RP2040 GPIO and IO Expander GPIO into a safe state
• Micropython access to the created I2C bus (if creating the object in MPy later causes issues)
• Have IO Expander GPIO's controllable in MPy via Pin objects (I've already made a start on this via the machine_pin_ext functions)
• Use of IO expander pins alongside CYW43 pins (for a future Pico W based Yukon being considered)
• The board returning to a safe state (aka soft reset) when stop is pressed in Thonny. I have checked the two projects and cannot figure out what command CPy is detecting that MPy is not.
• Possibly having soft resets occur on exceptions. I gave this a try in what I thought was the right area, but that had weird knock-on effects.

I briefly talked with Matt Trentini on twitter about the initialisation and resetting points, and there were certainly some valid questions raised. For instance, such resetting should be unnecessary if users followed best practices with handling all code paths and exceptions, but given one tester killed their Yukon board recently because they forgot to perform a specific function call in their loop that was in ALL the provided examples (!), that just cannot be guaranteed. I would therefore much rather encourage best practices, but have low-level systems in place as a fallback. We are also modifying the board with more hardware level protections based on that specific event.

As for the other points regarding the IO expanders, there could well be a module the user imports that lets them control those IO pins, which would be fine. After seeing what was done for the Raspberry Pi with the Pico W's WiFi module to abstract the user LED, I think this is a nice approach to apply to Yukon too. And anyway, the IO expanders are an implementation detail that are only there because the RP2040 doesn't have enough pins 😆

TL:DR, Working on a new RP2040 board, and would like guidance on modifying Micropython to support board initialisation/resetting, IO expander based Pin objects, and soft resetting under more conditions.

Thanks for your time!

P.S if this is the wrong sub-section, please let me know or move it to the correct place

[bidrohini]

You can post this to the pimoroni forum too: https://forums.pimoroni.com/

[ZodiusInfuser]

Thanks, but I am specifically after guidance from the maintainers of Micropython, since the changes I want to make relate directly to this codebase.

[ZodiusInfuser]

Hi. Just wondering if any of the devs here have had a think over the questions I posted?

[andrewleech]

While there are C hooks a board profile can use to run C functions at startup( eg. MICROPY_BOARD_EARLY_INIT ), there's currently no automatic python hooks for startup functionality.

So yeah in theory you could write some reset / init functions in C and use this or similar C hooks... but I hate writing drivers twice (python and C) so wouldn't reccomend this for myself :-)

I do think there's a real place for a <board_profile>/board.py file that's automatically imported (if it exists) probably just after boot.py? This doesn't exist but would be quite useful.

That being said... I've had discussions about trying to find a way for board profiles to include drivers for on-board devices (regardless of whether they're auto-imported or not), there's a question about where these drivers would/should live. We don't want individual copies of drivers living in the board profile folders, eg drivers for generic things like i2c expanders should not be duplicated into every board folder that uses them. So these drivers probably just need to be submitted / merged into micropython-lib and manifest.py frozen into the board build, then said hypothetical board.py could import them as needed.

This could make testing releases for boards more complicated, ie if said shared driver is upgraded do all boards that use it still work? I guess it also means getting new boards merged will be blocked by getting all their drivers merged first too... not insurmountable issues though certainly.

But on some boards there could be custom output hardware that does not really use a driver you'd consider generic, so maybe that doesn't belong in micropython-lib... does it do into the board folder? What if the manf makes 5 different boards that all use this same custom output, does that code then need replicating... maybe? Or it needs to manifest.py reference directly to their own github repo... but do we want to allow boards to automatically pull in third-party code?

Couple of questions more than answers, but I personally think an auto-import board.py would be a good starting point micropython MR :-D

[ZodiusInfuser]

Thanks for the reply.

While there are C hooks a board profile can use to run C functions at startup( eg. MICROPY_BOARD_EARLY_INIT ), there's currently no automatic python hooks for startup functionality.

I will have a look for example uses of those. It does seem though that the RP2 port lacks such hooks, though @jimmo's reply suggests that's not an intentional omission, so I will see what would fit my needs.

So yeah in theory you could write some reset / init functions in C and use this or similar C hooks... but I hate writing drivers twice (python and C) so wouldn't reccomend this for myself :-)

Funny thing is, I only have a C driver so far (even for the CircuitPython firmware), so writing a Python driver would actually be more work for me. As for the rest of the product's code, I have already resigned myself to writing separate versions for C++, Micropython and Circuitpython 😭 .

I do think there's a real place for a <board_profile>/board.py file that's automatically imported (if it exists) probably just after boot.py? This doesn't exist but would be quite useful.

...

Couple of questions more than answers, but I personally think an auto-import board.py would be a good starting point micropython MR :-D

Certainly stuff for me to think about. I hadn't necessarily thought about a board.py file, and am not sure if that would be entirely suitable for my case, but worth some exploration at least.

[jimmo]

The board returning to a safe state (aka soft reset) when stop is pressed in Thonny. I have checked the two projects and cannot figure out what command CPy is detecting that MPy is not.

@ZodiusInfuser This is because CircuitPython resets the runtime+VM before dropping to the REPL, whereas MicroPython keeps the existing program state (triggered by the Ctrl-C that Thonny sends).

Personally I really like the MicroPython behavior, it's very useful for debugging.

With regard to your other questions, the answer kind of falls into two parts:

1. The other ports (especially STM32) have more "hook points" for C code to intercept various things (such as soft reset). Anything you see in the STM32 port we should consider adding to the RP2 port.
2. The extended pins feature (on RP2 only) is definitely something we'd like to expand on further, and i2c port expander are an obvious next step. It would be really cool to be able to define these in Python, but I think for now we're likely going to need such drivers to be implemented in C. Combining multiple types of extended pins (i.e. Pico W's cyw43 pins with an i2c port expander) will need some though. Can you open a feature request issue to discuss this?

Possibly having soft resets occur on exceptions. I gave this a try in what I thought was the right area, but that had weird knock-on effects.

The current behavior is that it drops to the REPL (after displaying the exception). I feel like if your program wants to soft reset automatically on uncaught exception then it should add a global exception handler to explicitly call machine.soft_reset().

[ZodiusInfuser]

Thank you for the reply!

@ZodiusInfuser This is because CircuitPython resets the runtime+VM before dropping to the REPL, whereas MicroPython keeps the existing program state (triggered by the Ctrl-C that Thonny sends).

Thanks for clarifying. That explains the differences I see when trying to compare the codebases of the two projects.

Personally I really like the MicroPython behavior, it's very useful for debugging.

I have certainly found that valuable for some projects, and I am not necessarily wanting to change that. It does seem however that this also retains IO state, which could be actively dangerous for users of our product.

For example, we're rating this board for 30A peak output. If the user program just ends or crashes in some way without disabling the output MOSFET, then that output would still be live without any way of stopping it outside of a power cycle. If that happened whilst it was driving some motors, then that motor could be in a stalled state and burn out without the user realising it is still being driven (because they're not running any code), or a robot could carry on driving away at high speed, requiring literal running to catch up with it as it treks off down a carpark.

Possibly having soft resets occur on exceptions. I gave this a try in what I thought was the right area, but that had weird knock-on effects.

The current behaviour is that it drops to the REPL (after displaying the exception). I feel like if your program wants to soft reset automatically on uncaught exception then it should add a global exception handler to explicitly call machine.soft_reset().

This is indeed something I have been writing into all my examples for the product, and would highly encourage all users of the product to follow. There is just no way to guarantee that users actually do this. It may even be unintentional on the user's part, seeing an example but not understanding why they have to do this when they don't with a normal Pico, for instance. It happens.

Perhaps this is just my paranoia, but I would hate to receive customer complaints about dead boards, or dead motors etc that were attached to the board, due to conditions that could have been handled on the system side.

With regard to your other questions, the answer kind of falls into two parts:

1. The other ports (especially STM32) have more "hook points" for C code to intercept various things (such as soft reset). Anything you see in the STM32 port we should consider adding to the RP2 port.

Thanks! I had noticed some of those "hook points" after @andrewleech's reply (like MICROPY_BOARD_EARLY_INIT) and wondered if there was a reason for their absence. Since you're suggesting there is not, I will look and see which would be suitable for my case.

2. The extended pins feature (on RP2 only) is definitely something we'd like to expand on further, and i2c port expander are an obvious next step. It would be really cool to be able to define these in Python, but I think for now we're likely going to need such drivers to be implemented in C. Combining multiple types of extended pins (i.e. Pico W's cyw43 pins with an i2c port expander) will need some though. Can you open a feature request issue to discuss this?

Sure, I will raise a feature request to discuss. I'll try and get the i2c port expanders working with RP2 without cyw43 pins for now, since I now have a C driver written for it.