docs: Create a summary page, comparing the various ports of micropython

[massimosala]

Hi

I find it difficult to choose the right micro, there are many features enabled or disabled in specific micropython ports and the current docs are sometimes vague or inconsistent with the actual implementation.

My proposal is to have a table listing the characteristics for each micro.
The first columns...

• total flash size
• space occupied by the current firmware
• available RAM for programs
• what are the enabled/disabled firmware options ? like function attributes

The same table can grow with other columns, like:

• does the file system require flushing after write ( os.sync() ) ?
• does network.country() work?
• etc...

[rkompass]

That actually would be a not so small project. MPs Github has a Wiki which might be the right place to collect data.
Also inconsistencies in the function of peripherals are worth to be documented.
You have to consider though, that which functionality is enabled is not so much a trait of the port but rather which functionality may be switched on to be compiled in (and work).

[rkompass]

Many of the infos may be collected programmatically.
Like here:

import os
import sys
import machine
from ubinascii import hexlify
import gc

gc.collect()

def show_hw_info():
    print("ID ............",hexlify(machine.unique_id()).decode())
    print("Platform ......",sys.platform)
    print("Version .......",sys.version)
    print("Memory")
    print("   total ......",(gc.mem_alloc()+gc.mem_free())//1024, "KB")
    print("   used .......",gc.mem_alloc()//1024, "KB")
    print("   free .......",gc.mem_free()//1024, "KB")
    stat = os.statvfs('/flash')
    block_size = stat[0]
    total_blocks = stat[2]
    free_blocks  = stat[3]
    print("Flash-FS")
    print("   total ......", (total_blocks * block_size)//1024, "KB" )
    print("   used .......", (total_blocks-free_blocks)*block_size//1024, "KB")
    print("   free .......", (free_blocks * block_size)//1024, "KB")
    uname = os.uname()
    print("system name ...", uname.sysname)
    print("node name .....", uname.nodename)
    print("release .......", uname.release)
    print("version .......", uname.version)
    print("machine .......", uname.machine)
    print("frequency .....", machine.freq()//1_000_000, "MHz")

I should give credit for this to ?? (forgot it...).

[davefes]

Very nice. Would it be a good idea if useful utilities like this were in a place so that they are easily found?

[robert-hh]

If I count right, there are about 170 supported boards at the moment in the ports directory (I have a few duplicates and unmerged boards in my source tree). Making the table programmatically requires to run that code on all boards, and it will not run on all boards. And even if that work was done, the table will have to be constantly updated and edited to keep if up-to-date.
So if it is done, it's better to make a table with device families. But I see that as waste of time and effort.

[peterhinch]

This would be very useful, but I see a few difficulties. One is to distinguish between hosts and boards. For example a host might be capable of supporting SPIRAM but boards might have varying amounts including zero. My solution would be to only list hosts, providing references to boards so potential users can do their own research. Boards crop up more frequently than hosts.

Another problem is that different features are important to different people and different projects. A few examples of things that have mattered to me:

• Hard ISR support.
• Hardware FP (32 or 64 bit).
• I/O pin mapping (ability to select I/O pins for interfaces).
• Hardware features such as timers, WDT's, The ESP32 RMT, the RP2 PIO...

I think each user might come up with a different list.

Another problem is the differing feature sets of the various MP ports discussed in #5564.

[massimosala]

I read all your comments, thanks for your valuable feedback.

Also #5564, thank you Peter for pointing this out to me.

I would like to create this summary, otherwise it is impossible to choose the right board, the information is scattered in many documents and sometimes it is not updated.
Reading the comments here and on #5564 It seems it is valuable also for other users.

I think a possible solution can be to build two tables:
Table 1)
I make a program, initially running on my pc, to collect all the information from the C headers file of every port (mpconfig.h ? and other specific files), where there are the compilation options and other bits of information.

Table 2)
I make a program, like suggested by @rkompass, available to the users.
Who wants to contribute, run this program on its board to gather at runtime the details not available from 1).
This program will send the info to my server, to create this second table.

I think is a good solution to keep two separate tables:

• The first one is official; it is possibile to build it anytime (after a new firmware release, or every night in a cron job).

• The second table is not official, but IMHO it is better than nothig. It will be specified: empirical data

The two tables can be published in one or two web pages.
For the initial testing, these pages will reside on my server, available to everyone.

---

This is a draft, I appreciate further feedback.

I would like to start proving the feasibility of the first step.
Please, give me some hints about the source code of the firmwares:

• can I fetch the files thru HTTP, without credentials ?
• where can I find the firmware headers for the esp8266 ?
• where can I find the firmware headers for another micro ?

So I can see if the location of the files is consistent across the various firmwares.

(p.s. sorry for my English, I am in a hurry, this time I don't check my writing with google translator).

[rkompass]

I think we may discuss whether/which features may be collected programmatically.

• Hardware ISR: may be tested with setting up an interrupt wrapped in try/except as you did in some of your examples and then its function could be tested by triggering the interrupt by writing to the pins.
• Hardware FP (32 or 64 bit): We could inspect the cpu type (how?), and also wrap a multiplication in @native and inspect its code whether special FP commands are contained in it. Alternatively perhaps check division times.
• I/O pin mapping: Could be tested after we have simple test routines to test the function of the different interfaces.
• timers: may be tested programmatically by having a loop and checking whether different cycle numbers yield plausible timer delay measurements. And the other way round: count and then a timer-triggered ISR checks whether the count numbers are proportional/plausible and perhaps compatible with cpu frequency.

[massimosala]

I found where there are the headers.
For example:
https://github.com/micropython/micropython/blob/master/ports/esp8266/mpconfigport.h

It states:
// overriding defaults in py/mpconfig.h.

Pls, where can I find this file?

[massimosala]

Hi Robert

Yes, I see MICROPY_CONFIG_ROM_LEVEL and it seems it is working: the final tables listings the firmware symbols reflect its setting.

This is my processing:

• I read the port Makefile and extract fixed (MICROPY_... = ...) and optional definitions (MICROPY_... ?= ...)
• I read the boards mpconfigboard.mk, mpconfigboard.cmake: their defines override (if fixed) or add to (if optional) the defines of the port.
• I give these defines and the headers files as inputs to the parser.

If my reasoning is correct, I expect the Makefile to have no optional definitions: optional to what other definitions? It would imply that there are other makefiles to read before the port's Makefile.

As for the headers files I'm parsing, do you think there are other headers that must be read in order for the parser to have full knowledge and ability to evaluate the various #ifdefs #ifdefined ... ?

py/misc.h
py/mpconfig.h
PORT/mpconfigboard.h
PORT/mpconfigboard_common.h
PORT/mpconfigport.h
BOARD/mpconfigboard.h

I always refer to the basic features of the firmware, for now I leave out modules and drivers.
I need to test if it works so far, before adding more processing.

Do you know of any "difficult" port/board combo, where the board definitions override the port ones?
So I can test them and report back.

Thanks for the attention.
M

[robert-hh]

Board definitions always override the port ones. Even if compilations starts in the port directory, the order of effective processing is always:
. Board

• optional MCU
• Port
• MicroPython

If you make your parse recursive and follow the include directives, you can simply start with PORT/Makefile and PORT/mpconfigport.h, like the compile process does. At least for the non-cmake ports. For the latter you might have to call cmake first to create a Makefile.

[massimosala]

Hi

Something doesn't add up.
This in an extract from the Makefile of the mimxrt port:

# Set flash driver name, base address and internal flash flag, based on the flash type.
ifeq ($(MICROPY_HW_FLASH_TYPE),$(filter $(MICROPY_HW_FLASH_TYPE),qspi_nor_flash))
    MICROPY_HW_FLASH_BASE = 0x60000000
    FLEXSPI_FLASH_TYPE = $(MICROPY_HW_FLASH_TYPE)
else ifeq ($(MICROPY_HW_FLASH_TYPE),$(filter $(MICROPY_HW_FLASH_TYPE),qspi_hyper_flash))
    MICROPY_HW_FLASH_BASE = 0x60000000
    FLEXSPI_FLASH_TYPE = $(MICROPY_HW_FLASH_TYPE)
else ifeq ($(MICROPY_HW_FLASH_TYPE),$(filter $(MICROPY_HW_FLASH_TYPE),internal))
    # The internal flash is an SPI NOR Flash.
    MICROPY_HW_FLASH_BASE = 0x70000000
    FLEXSPI_FLASH_TYPE = qspi_nor_flash
    CFLAGS += -DMICROPY_HW_FLASH_INTERNAL
else
    $(error Error: Unknown board flash type $(MICROPY_HW_FLASH_TYPE))
endif

To work, it needs to know MICROPY_HW_FLASH_TYPE
Indeeed into the various sub-directories of the boards, the mpconfigboard.mk defines the symbol.
For example:
ports\mimxrt\boards\MIMXRT1010_EVK\mpconfigboard.mk

MICROPY_HW_FLASH_TYPE = qspi_nor_flash

You wrote:

you can simply start with PORT/Makefile and PORT/mpconfigport.h, like the compile process does

In this case, it doesn't work.
I have to evaluate the BOARD config before to evaluate the PORT Makefile.

1. Is/n't this order of processing the "makefiles" correct for all ports?

2. For the mimxrt port, there are 7 boards.
   6 of them define an absolute MICROPY_HW_FLASH_TYPE

Instead the
ports\mimxrt\boards\MIMXRT1170_EVK\mpconfigboard.mk
defines it as optional:

MICROPY_HW_FLASH_TYPE ?= qspi_nor_flash

Is this a mistype?
I know, it can work, but it's hard trying to figure out these steps without consistency in the makefiles...

[robert-hh]

As I said above: If you reach an include statement in Makefile, you have to process that file immediately. That ensures the proper order of processing. The same for #include statements in C header files.

[robert-hh]

Is this a mistype?

No. You can safely assume that what you see in the files is working. Some boards can be configured for different flash types. And assignments with ?= can as well be overruled at the make command line.

[robert-hh]

About your initial post:

• total flash size: Not in the .h files. In some ports it's in the .mk files
• space occupied by the current firmware. Not in any config file. The only thing you can get is the space reserved for the firmware. This information is usually in the .ld files, but different for all ports.
• available RAM for programs. Not in any config file. You may find information about the RAM size and the stack size, if that is static. But how much space is used for other code data is only reported by the loader when linking of the code finishes. Or by gc.mem_free().

[massimosala]

Robert, thanks for the additional info.
Yes, some data has to be collected at runtime, running a program on the specific target.
See also the comment by @rkompass, proposing some detection routines.

[rkompass]

Detection routine for interrupts:

Note: Inconsistencies pop up while testing such routines.
Inconsistency one: On the pyboard machine.freq() returns a 4-tuple
Inconsistency two: On the esp32 pin.irq(trigger=Pin.IRQ_RISING, handler=pin_isr, hard=False) results in a type error as the hard=False is not accepted.

Should I raise issues for these? I definitely agree with Peter that all those should be get rid of in the long run.

# Measure hard and soft GPIO interrupt response times in micropython.

from machine import Pin
from time import ticks_us, ticks_diff, sleep_ms, sleep_us
from array import array
from random import getrandbits
from sys import platform

if platform == 'esp32':
    pin = Pin(5, Pin.OUT, value=0)
if platform == 'mimxrt':
    pin = Pin('A1', Pin.OUT, value=0)
if platform == 'pyboard':
    pin = Pin('X1', Pin.OUT, value=0)
if platform == 'rp2':
    pin = Pin(1, Pin.OUT, value=0)
elif platform == 'w600':
    pin = Pin(Pin.PA_01, Pin.OUT, value=0)

t_stop = 0
def pin_isr(p):
    global t_stop
    t_stop = ticks_us()

def run_ints(pin, pin_act = 1, n_runs = 20, n_ints = 500):
    global t_start, t_stop
    
    print(f'Raising {n_runs*n_ints} Interrupts..')
    tdif_min = array('I',(0 for _ in range(n_runs)))
    tdif_max = array('I',(0 for _ in range(n_runs)))
    tdif_mean = array('f',(0 for _ in range(n_runs)))
    t_diffs = array('I',(0 for _ in range(n_ints)))

    for r in range(n_runs):
        print('.', end='')
        for i in range(n_ints):
            t_stop = 0
            t_start = ticks_us()
            pin(pin_act)
            while t_stop == 0:
                pass
            t_diffs[i] = ticks_diff(t_stop, t_start)
            pin(1-pin_act)
            sleep_us(getrandbits(10))
        tdif_min[r] = min(t_diffs)
        tdif_max[r] = max(t_diffs)
        tdif_mean[r] = sum(t_diffs)/n_ints

    print(f'\nInterrupt latency:')
    print(f'   mean: {sum(tdif_mean)/n_runs}µS  min: {min(tdif_min)}µS  max: {max(tdif_max)}µS')
    print(f'   lowest: {tuple(sorted(tdif_min)[0:min(20, n_runs)])} µS')
    print(f'   highest: {tuple(sorted(tdif_max)[-min(20, n_runs):n_runs])} µS')# print(f'Latency: {ticks_diff(t_stop, t_start)}µS')

sleep_ms(500) # --------------------------------------------
try:
    pin.irq(trigger=Pin.IRQ_RISING, handler=pin_isr, hard=True)
    print(f'Pin {pin}: Set up hard interrupt.')
    run_ints(pin, pin_act = 1, n_runs = 40, n_ints = 500)
except (TypeError, MemoryError):
    print(f'Pin {pin}: Could not set up hard interrupt.')

pin.irq(None) # --------------------------------------------
sleep_ms(500)
    
try:
    if platform == 'esp32':
        pin.irq(trigger=Pin.IRQ_RISING, handler=pin_isr)
    else:
        pin.irq(trigger=Pin.IRQ_RISING, handler=pin_isr, hard=False)
    print(f'Pin {pin}: Set up soft interrupt.')
    run_ints(pin, pin_act = 1, n_runs = 40, n_ints = 500)
except (TypeError, MemoryError):
    print(f'Pin {pin}: Could not set up soft interrupt.')

Results:

 -----   Results on Pyboard 1.1,  MP 1.19.1,  168 MHz -----

Pin Pin(Pin.cpu.A0, mode=Pin.OUT): Set up hard interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 10.97575001716614µS  min: 10µS  max: 13µS
   lowest: (10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10) µS
   highest: (12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13) µS
Pin Pin(Pin.cpu.A0, mode=Pin.OUT): Set up soft interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 11.99904999732971µS  min: 11µS  max: 13µS
   lowest: (11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11) µS
   highest: (13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13) µS

 -----   Results on RP2 pico,  MP 1.20,  125 MHz -----

Pin Pin(GPIO1, mode=OUT): Set up hard interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 18.1775µS  min: 13µS  max: 69µS
   lowest: (13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14) µS
   highest: (23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 69) µS
Pin Pin(GPIO1, mode=OUT): Set up soft interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 36.58075µS  min: 25µS  max: 74µS
   lowest: (25, 25, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28) µS
   highest: (47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 49, 49, 49, 50, 74) µS

 -----   Results on Teensy 4.1,  MP 1.19.1,  600 MHz -----

Pin Pin(GPIO_AD_B1_03): Set up hard interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 2.114699995517731µS  min: 2µS  max: 21µS
   lowest: (2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2) µS
   highest: (5, 6, 6, 7, 8, 9, 9, 9, 9, 10, 11, 11, 11, 11, 12, 12, 13, 15, 19, 21) µS
Pin Pin(GPIO_AD_B1_03): Set up soft interrupt.
Raising 20000 Interrupts..
Interrupt latency:
   mean: 2.846399992704391µS  min: 2µS  max: 24µS
   lowest: (2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2) µS
   highest: (10, 10, 10, 10, 11, 11, 12, 12, 13, 13, 13, 13, 13, 14, 14, 14, 15, 15, 18, 24) µS

 -----   Results on Winner w600 pico, MP 1.20, 80 MHz  -----

Pin Pin(1): Set up hard interrupt.
Raising 10000 Interrupts..
Interrupt latency:
   mean: 21.8567µS  min: 21µS  max: 42µS
   lowest: (21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21) µS
   highest: (30, 31, 31, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, 36, 42) µS
Pin Pin(1): Set up soft interrupt.
Raising 10000 Interrupts..
Interrupt latency:
   mean: 26.7499µS  min: 26µS  max: 542µS
   lowest: (26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26) µS
   highest: (40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 137, 147, 149, 542) µS

 -----   Results on ESP32 WROOM, MP 1.19, 160 MHz  -----

Pin Pin(5): Could not set up hard interrupt.
Pin Pin(5): Set up soft interrupt.
Raising 20000 Interrupts..
........................................
Interrupt latency:
   mean: 38.5056µS  min: 38µS  max: 169µS
   lowest: (38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38) µS
   highest: (58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 63, 169) µS

 -----   Results on ESP8266 Huzzah, MP 1.20, 80 MHz  -----

Pin Pin(5): Could not set up hard interrupt.
Pin Pin(5): Set up soft interrupt.
Raising 20000 Interrupts..
........................................
Interrupt latency:
   mean: 37.6529µS  min: 32µS  max: 225µS
   lowest: (32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32) µS
   highest: (88, 88, 88, 98, 100, 101, 106, 108, 111, 115, 117, 121, 140, 143, 144, 147, 160, 162, 164, 225) µS

[massimosala]

Hi @rkompass

Very interesting.

To have better measurements, what about using viper for the ISR ?

[rkompass]

That won't change much, I suppose. But you my try it. Note that every function call takes a few microseconds (less on Teensy with it's 600 Mhz). And Viper just doing the assignment when there are two nested function (the ISR and ticks_us()) should be no big difference.

[robert-hh]

Viper code is a little bit faster. I made a comparison once using a MIMXRT1176 board at 960Mhz. It made small hard irq function which toggled a pin and took the time between the trigger and the start of the response pin. For Python code, it was 1.1µs, for viper code 0.74µs. But the difference may be caused by the Pin I/O, which was the function call for Python, and direct register access for Viper.
For a teensy, the irq response time was 1.5µs for hard irq, and ~30µs for soft irq. With a MIMXRT1020 board, the IRQ response time was ~3 µs.
ESP32 response times are slow. One should assume an average value of 100µs. It can be as long a ~400µs. Attached is an old screenshot made with my oscilloscope. The yellow trace is the response pulse, the green is the trigger. The display is set to permanent, showing the full range of times for the trigger pulse. So you see the range of ~10µs to ~300µs with a few being at 380µs with the majority between 50µs and 200µs.

[rkompass]

Yeah. My thoughts regarding this were:

• To automatically test it's better to use mechanisms that are on the board.
• I'm not sure if ticks_us is as precise as it seems. Can we test that reliably?
• Viper is not available in all builds.
• The low response times are very consistent, and they are not the problem in a real problem. It's obvious that you may improve them by viper, assembler and/or register access.
• The high response times are much more relevant. They are so high, that any effect of viper or direct register access are negligible.
  We have the situation here, that other events like timer tick interrupts and/or serving a network device prevent a direct execution of the ISR.
• Such effects like networking activity should be taken into consideration while measuring the response times. How can I assure permanent network activity while measuring, any ideas?
• If there were a method (available in Micropython) to assign certain interrupts a higher priority than others it would be an important feature of the port. For example for quadrature encoding it could make it possible to follow higher rotation frequencies. This is what hard interrupts are about.
• The really high response times are very rare and they differ in different runs (the are difficult to find due to their statistics, so to say).
  So it's not easy to give an estimate how big the response time may at most. I think it may be better to try to determine confidence intervals on the response times. Say "95 % of all interrupt response times are below 80 uS" or so. But for certain time-critical application you need an absolute value, of course, which is not easy to find.

[robert-hh]

Should I raise issues for these?

No. The difference are known and sometimes documented,

[rkompass]

@robert-hh The w600 is fighting very well isn't it? Do you know what led to the abolishment of hard interrupts on the 8266? Do we have a similar situation on the w600?

[robert-hh]

Damien explained years ago, but I forgot it. It's related to the xtensa architecture. The w600 is ARM, and has hard interrupts, as you tested. But the IRQ response that I had tested was much worse. But that may be related to the different test method.

[rkompass]

The results depend on other factors. I also had 138 uS for hard interrupt already. Outliers: Every run different.

[peterhinch]

Taking STM as an example, the IRQ latency values for hard interrupts looks correct. But the worst-case soft IRQ latency is much worse than 13μs: it occurs when the IRQ happens to coincide with the start of a GC cycle. Latency depends on how much work the garbage collector has to do, but I would expect a time in the range 1-10ms. Your test routine should do a lot of allocation of temporary objects to try to provoke this.

As a general point, on platforms with SPIRAM, GC times can be 100-200ms with consequences for soft IRQ latency.

[rkompass]

Thanks, I forgot that in the above. In the program one might do a gc.collect() while being in the delay loop waiting for the interrupt:

            while t_stop == 0:
                pass

having produced and deleted a few variables beforehand. I may try this out.

[rkompass]

Result is: With hard interrupt no change. With soft interrupt the response time is > 2 ms. Probably more, if there is more to collect.

[robert-hh]

try:

from machine import idle
while t_stop == 0:
    idle()

Short loops with pass only are notorious for slowing down the code.

[massimosala]

Hi

About the interrupt timing tests, spelunking thru the firmware headers files I see at least these ports using FreeRTOS: esp32, cc3200.

Having an RTOS ins't beneficial to latency and bit-banging stuff.
I suggest prior to the tests:

• gc. collect() plus gc.enable(False)
• check if you can stop FreeRTOS (or other RT kernels), for the short time running the tests.

---

I appreciate your discussion on low level hardware, but IMHO it is off-topic here.
Can you please open a new thread?
Thanks, Masssimo

[robert-hh]

Forget the CC3200. It is old, not on sale anymore and support will be dropped soon. Same for the Teensy port, which is for Teensy 3.2. The Teensy 4.x are covered by the MIMXRT port.

[massimosala]

Forget the CC3200. It is old, not on sale anymore and support will be dropped soon. Same for the Teensy port, which is for Teensy 3.2. The Teensy 4.x are covered by the MIMXRT port.

About dropping it: I suggest to freeze the last source tree and binary firmwares, keeping them in a unsupported branch of the repository.
At least the poor guys who have CC3200 boards lying around can try Micropython.

For example last month I was trying some stuff on stm32, an evaluation board purchased 10 years ago.
I am happy platform.io still supports it.
The ridiculous thing: it was hard to find an old version of the STM tool for uploading binary files!

[rkompass]

Viper in the ISR did not change anything. Disabling the gc did not change the hard interrupt response times.
As we are in here for a characterization of the boards: I would summarize:
The availability of hard interrupts is relevant and the typical and maximal response times for hard interrupts. Soft interrupts are much slower and in case of a gc.collect well above miliseconds.

[bixb922]

pystones as a relative speed measure of CPUs might be an interesting addition. See also https://micropython.org/resources/code-dashboard/

[rkompass]

I was thinking of using benchmarks to determine whether we have floating point hardware or emulation:
My attempt is here:

from struct import pack
from time import ticks_us, ticks_diff
from math import sqrt
from machine import freq
from sys import platform

def binary_d(num):
    return ''.join('{:0>8b}'.format(c) for c in pack('!d', num))

def has_64bit_float():
    return 1.000001 * 1.000001 > 1.000002

def has_64bit_float_2():
    return pack('!d', 0.32)[7] == 123

def floatbench_mul(n):
    x = 1.0001
    t_start = ticks_us()
    for i in range(n):
        x *= 1.0001
    t_stop = ticks_us()
    return ticks_diff(t_stop, t_start) /n

def floatbench_msqrt(n):
    x = 3.9
    t_start = ticks_us()
    for i in range(n):
        x *= x
        x = sqrt(x)
    t_stop = ticks_us()
    return ticks_diff(t_stop, t_start) / n

def intbench_muld(n):
    x = 10001
    t_start = ticks_us()
    for i in range(n):
        x *= 10001
        x //= 10000
    t_stop = ticks_us()
    return ticks_diff(t_stop, t_start) / n

print('Float precision:', '64 bit' if has_64bit_float() else '32 bit')
if platform == 'pyboard':
    cyc_us = 1_000_000/(freq()[0])
else:
    cyc_us = 1_000_000/freq()
print('rel. Float speed: mul:', floatbench_mul(1000)/cyc_us, 'msqrt:', floatbench_msqrt(1000)/cyc_us,
      'int_muld:', intbench_muld(1000)/cyc_us, 'cpu cycles')

The results are:

----   Results: Ordinary MP and @native modes ---------------------

Pyboard 1.1:
Float precision: 64 bit
rel. Float speed: mul: 1424.64 msqrt: 3589.152 int_muld: 811.104 cpu cycles
@native:
rel. Float speed: mul: 1003.46 msqrt: 3021.984 int_muld: 385.896 cpu cycles

Teensy 4.1 (mimxrt port):
Float precision: 64 bit
rel. Float speed: mul: 838.2 msqrt: 2200.8 int_muld: 580.2 cpu cycles
@native:
rel. Float speed: mul: 565.2 msqrt: 1850.4 int_muld: 280.2 cpu cycles

RP2 Pico:
Float precision: 32 bit
rel. Float speed: mul: 1314.375 msqrt: 2363.875 int_muld: 1066.375 cpu cycles
@native:
rel. Float speed: mul: 908.9999 msqrt: 1787.25 int_muld: 490.25 cpu cycles

Winner W600 Pico:
Float precision: 32 bit
rel. Float speed: mul: 1044.88 msqrt: 2144.64 int_muld: 890.16 cpu cycles
@native:
rel. Float speed: mul: 709.6 msqrt: 1669.6 int_muld: 413.04 cpu cycles

ESP8266 (Feather Huzzah):  Xtensa architecture
Float precision: 32 bit
rel. Float speed: mul: 1542.08 msqrt: 4050.96 int_muld: 2009.12 cpu cycles
@native: Memory error

ESP32 (ESP32 Wroom):   Xtensa architecture
Float precision: 32 bit
rel. Float speed: mul: 1195.36 msqrt: 2159.36 int_muld: 1069.76 cpu cycles
@native:
rel. Float speed: mul: 719.36 msqrt: 1599.84 int_muld: 416.32 cpu cycles

I was thinking that the RP2 Pico (ARM M0+) had FP emulation, but this apparently is not obvious from the above benchmark.
I conclude: These are essentially interpreter times. The FP operations are not so relevant.
We could test integer operations in @viper mode. For the FP operations we should do assembler?
Does that mean we don't have to care whether FP is emulated or done in hardware?

[rkompass]

Perhaps a more differentiated picture appears when benchmarking compiled code. Ulab comes to mind here.
Is there a version of ulab that may be just imported/put on the flash (the new MP feature)?

[robert-hh]

Is there a version of ulab that may be just imported/put on the flash (the new MP feature)?

No, as far as I know. And the new MP feature that you mention does not yet exist.

[peterhinch]

Pyboard 1.1:
Float precision: 64 bit

This is incorrect. It has 32 bit hardware FP.

[rkompass]

You are right, as far as I recall, I knew the same, so I was wondering. Nevertheless on MicroPython v1.19.1 on 2022-06-18; PYBv1.1 with STM32F405RG
I get:

>>> 1.000001 * 1.000001
1.000002000001

and with

from struct import pack
def binary_d(num):
    return ''.join('{:0>8b}'.format(c) for c in pack('!d', num))

>>> binary_d(1.000002000001)
'0011111111110000000000000000001000011000110111110000010110101110'

which clearly shows that the 64 bits are there.
I assume from number and date I had this binary installed.

[rkompass]

With the daily MP build MicroPython v1.20.0-162-gd080d427e on 2023-06-06; PYBv1.1 with STM32F405RG
my pyboard again has 32bits :-)

[peterhinch]

I think there is a build option for 64 bit soft FP. Note that the Pyboard D SF6W has 64 bit hardware FP.

[Josverl]

I think I have some - not all - of that information already gathered for a lot of ports / boards and versions as part of the micropython-stubs project.
I have automated the generation of most of the information - getting all the boards setup and reliably flashed across different ports and boards is has been a project of years ( happy with the recent fixes to MPremote that made that possible)

Just one random sample out of the many
I have been looking at ways to compare and publish these - but the challenge is that this is an N dimensional matrix - and that most screens only support 2

There are a few options that I explored for publication - but set aside due due to too-little-time
Github Blocks
Extend your codebase with custom, interactive blocks.
Build rich documentation, enhance your workflows, and bring your repository to life

Flat Data
Essentially a rich file viewer , that is Got aware and can track changes / and tags over time for a data file.
Simple demo on Micropython Data
adding more fileds to the json / csv / whatever data source would allow a simple search

[rkompass]

To understand what you did:
You collected the info which modules were compiled in or freezed into certain builds of MP for the different platforms?
Together with checksums?
Is this from tests of importing those modules. Or from checking dir(module-name)?
Did you test the functionality of those modules?