MicroPython
I2CTarget, improved floats and native emitter, STM32N6 & ESP32C2 support #17873
Replies: 7 comments 11 replies
-
|
Great release! Thank you to all the contributors! :) I look forward to playing with the I2C. Microcontrollers are so cheap, that using multiple is often a practical solution, at least in prototype or low-scale phase. Especially when one can find off-the-shelf modules that have the desired hardware on-board. But then they need to talk together to form a coherent system. I2C is great for those cases where one top micro delegates to others for more specialized tasks. |
Beta Was this translation helpful? Give feedback.
-
|
This is great! |
Beta Was this translation helpful? Give feedback.
-
|
Eagerly awaiting for the v1.26.0 firmware to appear on the downloads pages for all to enjoy. ---
config:
xyChart:
showDataLabel: true
themeVariables:
xyChart:
titleColor: "#ff0000"
---
xychart-beta
title "MicroPython Board (bars), and Variant (line) counts"
x-axis [v1.18, v1.19, v1.19.1, v1.20.0, v1.21.0, v1.22.0, v1.22.1, v1.22.2, v1.23.0, v1.24.0, v1.24.1, v1.25.0, v1.26.0]
y-axis "Count" 0 --> 200
bar[124, 141, 141, 160, 158, 163, 163, 163, 163, 172, 172, 189, 195]
line [24, 24, 24, 24, 32, 32, 32, 32, 35, 35, 35, 44, 44]
|
Beta Was this translation helpful? Give feedback.
-
|
Nice graph! What's the line represent, number of variants? Maybe you could add a key. |
Beta Was this translation helpful? Give feedback.
-
|
This is a graph of the counts I get when collecting the board names and descriptions across versions for Mpflash to use (before |
Beta Was this translation helpful? Give feedback.
-
|
It could probably count boards without a variant as having a single variant, then the line would be the total number of boards+variants. |
Beta Was this translation helpful? Give feedback.
-
|
Do the downloadable v1.26.0 images for ESP32 (generic) and ESP32-C3 (generic) support ROMFS? Sorry to ask such a basic question, but you don't explicitly state whether or not they do. ROMFS is only mentioned once in the release notes - in the last bullet point of the RP2 port section. |
Beta Was this translation helpful? Give feedback.
-
|
It seems to be enabled for just a few boards, like a buidl variant of ESP8266, a STM32H747 board, an ALIF board, QEMU and the unix port. There is a partition file for ESP32 which defined a ROMFS partition, but it seems to be unused. That all looks still like WIP. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Robert. Thanks for clarifying this. I appreciate your effort. It is not the answer I was hoping for, but at least I now know what the situation is. It would be nice if Damien or Jimmo could give an indication of when ROMFS will be available on these ports. If it not going to be soon, then I am going to have to go down the frozen bytecode path. This would mean setting up a MicroPython build system for ESP32 on my M1 iMac. TBH I had been hoping to avoid doing this. |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/mattytrentini/mpbuild makes setting up the toolchain much less painful. If one can tolerate docker, that is :) |
Beta Was this translation helpful? Give feedback.
-
|
If you want to build a firmware with ROMFS support, you can do so by:
Then build the firmware. That will give you a 128k ROMFS partition. Note that the previous file system will get lost. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks guys - a progress report. Any suggestions guys? Robert, did you actually test your instructions? Are some more changes needed, or is this feature not quite "ready to go" for esp32? In the release note for v1.25 ,Damien referred to commit 50a7362. This suggests the changes needed are more extensive than suggested, but as it is for the rp2 port, I am not sure how relevant it is to esp32. |
Beta Was this translation helpful? Give feedback.
-
|
These errors are common if rebuilding after changes. Try |
Beta Was this translation helpful? Give feedback.
-
|
Yay! That seems to have worked. As I am using mpbuild, I ran mpbuild clean ESP32_GENERIC. It is now running on my M5Stack module. Next step is to have a play with mpremote and then set up and populate the ROMFS. It's looking good! |
Beta Was this translation helpful? Give feedback.
-
|
I created and populated the ROMFS partition without any issues. I placed the font files and most of the gui files in there. About 117KB in total. I kept the ROMFS directory/file hierarchy the same as in /lib, so I didn't need to change my application at all. It just ran as normal, perhaps loaded a little faster. The free memory at startup increased by about 30KB. TBH I was expecting more, but I have already split my application into a permanent core and several "load on demand" modules. So, not all the ROMFS resources will be in use at any given time. I wonder if it would be better to load everything (or at least all the fonts) at startup? I must say that ROMFS is a lovely feature. It is so easy to use and I like that I can update the contents without affecting MicroPython itself or the Flash filesystem. I am also very impressed by mpbuild. Encouraged by this success I decided to build MicroPython for my ESP32C3 board. The build process (v1.27.0-preview) went flawlessly and I flashed the binary onto the board using Thonny. It reset continually, giving the following error: I wondered if the problem was my build, so I downloaded the corresponding Preview Build from the MicroPython website and flashed it. It worked perfectly. After some musing, I wondered if the cause was that the MicroPython team are using IDF v5.4.2 whereas mpbuild is using v5.4.1? A quick Google revealed: So, this appears to be a limitation of mpbuild. It can use a known faulty tool chain and there is no easy way to change it. I looked into the source code. The problem is at line 26 of build.py. Fixing it is a 10 second edit. Unfortunately, I don't know how to clone/fork the project and rebuild it. I will raise an Issue and see what happens. |
Beta Was this translation helpful? Give feedback.
-
|
This is what I did. Hopefully, it can provide some help. esp32/boards: esp32: Erase the flash and install the firmware. Erasing is important since we have different partition tables. We can now check romfs using mpremote (I rename it as mpr) NB! -and- If you are using a Python 3 version lower than 11, you may need to make a change. Okay, now that we have the ROMFS defined, see if that works. $ mpr u0
Connected to MicroPython at /dev/ttyUSB0
Use Ctrl-] or Ctrl-x to exit this shell
>>> import hello_world
Hello World! |
Beta Was this translation helpful? Give feedback.
-
|
Hi Sharil, thanks for this. I am using mpbuild and modifying an existing board setup, so I am not getting involved in the build details like this. However, it is interesting that the source code changes you suggest are essentially the same as those suggested by Robert. Thanks again for responding. |
Beta Was this translation helpful? Give feedback.
-
|
I have some question about ROMFS,
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x6000,
phy_init, data, phy, 0xf000, 0x1000,
factory, app, factory, 0x10000, 0x1D0000,
romfs0, data, 0x8f, 0x1E0000, 0x40000,
romfs1, data, 0x8f, 0x220000, 0x40000,
romfs2, data, 0x8f, 0x260000, 0x40000,By the way, how calculate Size # Name, Type, SubType, Offset, Size, Flags
romfs, data, 0x8f, 0x1E0000, 0x20000,
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
This release of MicroPython sees the introduction of
machine.I2CTarget, which allows Python code to implement an I2C target device. It's available on the alif, esp32, mimxrt, rp2, samd, stm32 and zephyr ports. In the simplest case it can create an I2C register/memory device that connects to a bytearray (or similar buffer) and allows an I2C controller to read from and write into that bytearray. For more complex scenarios theI2CTargetclass exposes a set of interrupts which can be acted upon by Python code, allowing an arbitrary I2C device to be created. See the documentation for more details and examples.Floating point support has been improved in a few ways in this release. First, the formatting (printing) of floats has been rewritten to significantly improve the
reprreversibility of floating-point numbers. That is, formatting and then re-parsing a float should return the original value. In MicroPython the percentage of floats that correctly repr and parse back was around 28% (single precision) and 38% (double precision), but has now been improved (in the standard build configuration) to 98.5% and 99.8% (single and double respectively). In addition to printing out more accurate floats, that helps a lot when saving floats to .mpy files, because it means they can be loaded back to the same value.Second, there is now support in the compiler for handling floats as constants. Floats can now be folded (along with integers) in arithmetic operations, and be part of
constassignments. This helps to optimize bytecode because constant float expressions can now be evaluated in the compiler instead of at runtime.The third float improvement involves object representation C, where floats are stored within the immediate object value rather than on the heap. In this object module, the float can only take up 30 bits, so the last two bits of the single precision number are lost. Previously they were just truncated, but now a heuristic is applied when recovering the float value from the object such that the last two bits are copied from the previous two. This alleviates a bias towards zero and in general improves floating point calculations when using this representation.
The native and viper emitter backends have been improved to emit more optimal machine code, for example to use more compact instructions in loads and stores. This is for all supported architectures: ARM, Thumb, Xtensa, RISC-V 32, x86 and x64. Thumb v1 (for example RP2040) now supports long jumps greater than 12 bits, allowing larger Python functions to be compiled to this native architecture. Also, the inline Xtensa assembler now implements most of the LX3 opcodes, with additions including addx2, subx2, ssl and ssr, among others.
Two new MCUs are supported by this release: STM32N6xx and ESP32-C2 (aka ESP8684). The STM32N6xx is a new STMicroelectronics MCU running at 800MHz with a large amount of RAM and machine-learning accelerators. MicroPython now supports this MCU with USB, XSPI memory-mapped external flash, a filesystem, basic peripherals and deepsleep. The ESP32-C2 is by Espressif and is a small and low cost RISC-V MCU with WiFi and BLE. MicroPython running on this MCU supports a REPL, filesystem, GPIO, I2C, ADC, PWM, timers, WiFi and BLE.
The date range supported by the
timemodule has now been standardized to work the same across all platforms. The functionstime(),localtime()andmktime()now work properly on a reasonable range of dates, from at least 1970 up to 2099, regardless of the Epoch used by the platform.The MicroPython virtual machine is now able to avoid heap-allocating slices when subscripting bytearray and memoryview objects, for example expressions like
bytearray_obj[a:b] = c. The slice object is now allocated on the C stack, helping to reduce memory churn and allowing such expressions to work when the heap is locked, for example during a hard interrupt handler. This improvement goes in the general direction of MicroPython using the heap as little as possible, making execution more deterministic.Other improvements to the core runtime include support for using
__all__in star imports, support for PEP487's__set_name__special method, support for the _b/o/x specifier instr.format, and arrays can now be extended from any iterable, not just from an object with the buffer protocol. A new MicroPython-specificsys.implementation._threadattribute has been added, which exists when threading is enabled and tells which threading model is used (GIL or unsafe/no-GIL).The
framebufmodule now supports blit'ing read-only data to aFrameBuffer, which helps when implementing custom fonts that can now be stored in ROM. The DTLS implementation in thetlsmodule now enables DTLS HelloVerify and Anti Replay protection (for the mbedTLS backend), allowing proper DTLS servers to be implemented. The lwIP socket layer now has a queue of incoming UDP and raw packets (previously it only had room for one outstanding packet), allowing for more robust and efficient UDP protocols.Mpremote has had a few improvements. There is a new
fs treecommand which mimics the Unixtreecommand, and includes the-sand-hoptions to show the file sizes in the output. Thedfcommand has been enhanced to use the new no-argumentvfs.mount()query, and shows a much better summary of the mounted filesystems. Themip installcommand now uses hashes to skip files that exist. The location of the userconfig.pyfile is now more portable across many different OSes and configuration styles, thanks to the use of theplatformdirshelper package. There is also support for targets without theerrnomodule, improved disconnect handling, and better ESP-device detection for USB-CDC ports.The
mpy_ld.pylinker can now resolve fixed-address symbols if requested, for example on ESP8266 it can link to ROM-provided functions. There is also now support for ABS32 text relocations on ARMv6M (RP2040). These improvements extend the set of C extensions that can be compiled for such targets.Libraries that have been updated in this release include: lwIP updated to STABLE-2_2_1_RELEASE, LittleFS updated to v2.11, libhydrogen updated to the latest release, and stm32lib updated to include support for STM32N6.
There have been many improvements to the test suite, to both the tests and the test runners. This is in part to aid the new Octoprobe test framework, which provides hardware-in-the-loop testing. The continuous integration testing now also includes an undefined behaviour sanitizer (UBSan) build, an address sanitizer (ASan) build, a long-long unix variant, and also testing of object representation C. MicroPython has a lot of tests!
The alif port now supports pin interrupts, has improved SPI transfers, and I2C configuration now allows changing the SCL and SDA pins.
The esp32 port is updated to use ESP-IDF v5.4.2, and now supports ESP32-C2. Most ESP32-based boards now auto-detect the size of their flash on boot and automatically create an appropriate vfs partition based on the size of the flash. This allows the same firmware image to work on boards with various flash sizes. This port has also added a new
esp32.PCNTclass along withmachine.Counterandmachine.Encoder, which can count input edges, including motor rotation. The PWM class has been improved, and its API now matches other ports. There is support for the LAN8670 PHY, and a newesp32.idf_task_info()function (useful with the newutoppackage in micropython-lib). TheUART.sendbreak()method has been rewritten so that it doesn't reconfigure the UART during its execution.The nrf port has had fixes and improvements to its UART REPL to make it more robust, and work with mpremote. It also now uses the correct iRAM address for native code execution on nRF52 MCUs. The nrf implementation of
machine.enable_irq()andmachine.disable_irq()has been reworked so it uses the code common with all other ports, and this is a breaking change (on nrf boards only) due to the signature change ofenable_irq(). Previously the signature wasenable_irq(), and now the signature matches other ports, and the docs, and isenable_irq(state), wherestateis the return value fromdisable_irq().The rp2 port now enables compressed error messages by default, which reduces firmware size by about 3000 bytes. The pico-sdk alarm pool is now used (again, instead of custom soft-time code) for power-saving delays, and lightsleep has been improved. Open drain mode has been fixed on RP2350 with more than 32 GPIOs, and support for hard IRQ timer callbacks has been added.
The webassembly port has seen improvements to its FFI interface with JavaScript: it improves "has" and "get" proxying, fixes binding of self to JavaScript methods, implements equality for
JsProxyobjects, and reusesJsProxyreferences when possible to improve equality relationships of JavaScript objects on the Python side.The zephyr port has been updated to use Zephyr v4.0.0, and many improvements have been made: PWM support has been added, UARTs are now interrupt driven with ring-buffers and can set the baudrate and other parameters, GPIO supports open-drain mode, SoftI2C and SoftSPI have been enabled, and the
zephyr.FlashAreaclass now contains constants which enumerate the available partitions. REPL reliability has also been improved and it now supports ctrl-C in the default configuration. BLE can now create services at runtime using the standardBLE.register_services()method. Other small things have been added to make the zephyr port match the behaviour of other bare-metal ports, including:boot.pyandmain.pyare now executed at start-up,/libis added tosys.pathas appropriate, stdin/out/err have been enabled in thesysmodule, along with the ability to import .mpy files, and await/async keywords are enabled. This port now uses the standardized ROM configuration levels, with the default being the"basic" level.
New boards added in this release are: GARATRONIC_PYBSTICK26_ESP32C3 and SPARKFUN_IOT_REDBOARD_ESP32 (esp32 port), SPARKFUN_REDBOARD_TURBO and SPARKFUN_SAMD21_DEV_BREAKOUT (samd port), NUCLEO_N657X0 and OPENMV_N6 (stm32 port), beagleplay_cc1352p7, nrf5340dk, nrf9151dk and rpi_pico (zephyr port).
The change in code size since the previous release for select builds of various ports is (absolute and percentage change in the text section):
The leading causes of these changes in code size are:
machine.I2CTarget, improve accuracy of float formatting and range of time functions, improve native emitter and inline assembler, update LittleFS to v2.11.io.IOBaseat core feature level, avoid heap-allocating slices when subscripting built-ins.machine.I2CTarget.machine.I2CTarget, improve accuracy of float formatting, improve native emitter, add float constant folding, update LittleFS to v2.11.io.IOBase, standardize the range of time functions.machine.I2CTarget, improve accuracy of float formatting.machine.I2CTarget, implement DTLS HelloVerify and Anti Replay.machine.I2CTarget, improve accuracy of float formatting, improve native emitter, float constant folding.Thanks to everyone who contributed to this release: Alessandro Gatti, Andrea Giammarchi, Andrew Leech, Angus Gratton, Anson Mansfield, Anton Blanchard, Ayush Singh, Chris Webb, Christian Lang, Damien George, Daniel Campora, Daniël van de Giessen, David Schneider, David Yang, Detlev Zundel, Dryw Wade, dubiousjim, Elvis Pfutzenreuter, ennyKey, Garatronic, Herwin Grobben, iabdalkader, IhorNehrutsa, Jeff Epler, Jim Mussared, Jonathan Hogg, Jos Verlinde, Koudai Aono, Malcolm McKellips, Matt Trentini, Maureen Helm, Meir Armon, Patrick Joy, Peter Harper, Phil Howard, purewack, Rick Sorensen, robert-hh, root, SiZiOUS, stijn, TianShuang Ke, Vdragon, Yanfeng Liu, Yoctopuce dev, Yuuki NAGAO.
MicroPython is a global Open Source project, and contributions were made from the following timezones: -0600, -0500, -0400, -0300, +0000, +0100, +0200, +0300, +0530, +0800, +0900, +1000, +1100.
The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Espressif, Arduino, OpenMV, Yoctopuce, Microbric and Planet Innovation.
This discussion was created from the release I2CTarget, improved floats and native emitter, STM32N6 & ESP32C2 support.
Beta Was this translation helpful? Give feedback.
All reactions