What are the fields returned by mpremote devs?

[DavesCodeMusings]

When I run py.exe -m mpremote devs on my Windoze machine with an ESP32 devkit plugged in, I get output like the following:

COM3 0001 10c4:ea60 Silicon Labs None

I'd like to parse the output and use it in a program, so I'm trying to figure out what the fields mean.

From right to left, I can take a guess:

• COM3 is pretty obvious. It's the port path.
• 0001 is probably the ID of the serial port. (i.e. it's the first one detected.)
• 10c4:ea60 is likely some USB plug and play info.
• Silicon Labs is the manufacturer of the USB to UART bridge.
• None is ??? I don't know.

Is this close? Is there any documentation to confirm my guesses and explain what the None indicates? I searched, but came up empty.

[scruss]

• None is ???

… what you get when it's a serial device that mpremote doesn't recognize. Here's what I get on my Linux box with three different devices plugged in:

/dev/ttyACM0 397539583033 1209:abd1 OpenMV OpenMV Virtual Comm Port in FS Mode
/dev/ttyACM1 0ecb4f7dccb7027b 2e8a:0005 MicroPython Board in FS mode
/dev/ttyACM2 54F926C95259545020312E39262705FF 239a:80cc Adafruit Industries LLC QT Py M0

The second field is the serial number of the board. It can be used to tell identical boards apart. The 0001 that you're seeing is your system's guess at a device with no serial number

[DavesCodeMusings]

Thanks for the explanation. After seeing your Linux examples, I decided to give it a try with a Raspberry Pi (Debian Linux.) The same ESP32 device plugged into the Pi shows this:

/dev/ttyUSB0 0001 10c4:ea60 Silicon Labs CP2102 USB to UART Bridge Controller

Much more complete information. Serial number is still 0001, but None has been replaced by CP2102 USB to UART Bridge Controller.

For further testing on my Windows machine, I copied and ran the list_ports_windows.py sample code from the pyserial GitHub page. The resulting output is much more complete.

COM3: Silicon Labs CP210x USB to UART Bridge (COM3) [USB VID:PID=10C4:EA60 SER=0001 LOCATION=1-1]

So if the example code can identify the USB to UART Bridge Controller part on my machine instead of spitting out None, it makes me wonder why mpremote is not able to figure it out. Assuming it's also using pyserial.

Maybe I'll dig into the mpremote code and see if I can figure it out.

[DavesCodeMusings]

Looking at the mpremote code vs. pyserial example code, the difference seems to be this:

The list_ports_windows.py example uses from serial.tools import list_ports_common

mpremote's commands.py uses import serial.tools.list_ports

list_ports_common vs. just list_ports

Interestingly, the pyserial example list_ports_linux.py also uses from serial.tools import list_ports_common so it would seem list_ports_common works regardless of the O.S.

The comments in list_ports_common.py say:

# This is a helper module for the various platform dependent list_port
# implementations.

[Josverl]

it would seem list_ports_common works regardless of the O.S.

Yes It works , but the platform specific implementations provide different results depending on the OS , on Mac it can get really interesting.

Not to cut your exploration short, but thusfar I have not found a reliable way to identify a board across ports and host OSes just though USB IDs. - but I hope you may find something new that I have not considered.

It is possible by running some micropython code on the MCU ( and that calls for identifying , connecting to the repl ( assuming it is there) , interrupting any running app , running your own script and interpreting the output.
and then there are several chips that do not actually have an identifier stamped in.

if you want that, then mpflash.MPRemoteBoard may be of interest to you.

As of 1.24.x mpremote has a somewhat basic API - so less need for subprocess.run ,
but it is rather limited - and needs work to fully avoid needing to wrap in a process , with all headaches wrt to reliability, portability and stability from that.

my initial testing and comments : https://gist.github.com/Josverl/afe2cb1527a72772c86f355cba7de86d#file-api-test-ipynb

[DavesCodeMusings]

If you're up for it, I wonder what would be the result if you ran the example list_ports_osx.py on a Mac. It uses list_ports_common as well.

Just wondering aloud if simply changing the mpremote code to use list_ports instead of list_ports_common might be an improvement. I'll see if I can get it going on my Windows machine.

Not to cut your exploration short, but thusfar I have not found a reliable way to identify a board across ports and host OSes just though USB IDs.

Mostly I'm interested in the COM port (path) portion of the output. The rest of the fields had me curious. Now it's turned into a bit of a quest. Seeing None in the output had me wondering what information I was missing out on. Now that I know it should look like the Linux output, and that the pyserial example code can do it, I'm wondering if mpremote be modified to give that missing bit of info too.

[DavesCodeMusings]

Just wondering aloud if simply changing the mpremote code to use list_ports instead of list_ports_common might be an improvement. I'll see if I can get it going on my Windows machine.

Didn't work. Just gave errors.

[scruss]

I agree with Jos that trying to parse this is going to be hard. His mpflash tool does a decent job of finding what's there.

As installing pyserial on a Mac can be a real chore, and I have a Mac M2 right here, here's list_ports_osx.py output from my machine:

/dev/cu.Bluetooth-Incoming-Port: n/a [n/a]
/dev/cu.debug-console: n/a [n/a]
/dev/cu.usbmodem2101: Seeeduino XIAO [USB VID:PID=2886:002F SER=2A9766BF5136575020312E301E2F17FF LOCATION=2-1]
/dev/cu.usbmodem83201: Board in FS mode [USB VID:PID=1B4F:0026 SER=e46030f803531924 LOCATION=8-3.2]
/dev/cu.usbmodemNextDraw85111: EiBotBoard,NextDraw8511 [USB VID:PID=04D8:FD92 SER=NextDraw8511 LOCATION=0-1.4.2]
/dev/cu.usbserial-0001: CP2102 USB to UART Bridge Controller [USB VID:PID=10C4:EA60 SER=0001 LOCATION=8-3.1.2]

mpremote devs output:

/dev/cu.Bluetooth-Incoming-Port None 0000:0000 None None
/dev/cu.debug-console None 0000:0000 None None
/dev/cu.usbmodem2101 2A9766BF5136575020312E301E2F17FF 2886:002f Seeed Seeeduino XIAO
/dev/cu.usbmodem83201 e46030f803531924 1b4f:0026 MicroPython Board in FS mode
/dev/cu.usbmodemNextDraw85111 NextDraw8511 04d8:fd92 SchmalzHaus EiBotBoard,NextDraw8511
/dev/cu.usbserial-0001 0001 10c4:ea60 Silicon Labs CP2102 USB to UART Bridge Controller

and for completeness, mpflash list output:

Error: Failed to get mcu_info for /dev/cu.usbmodemNextDraw85111
Error: Failed to get mcu_info for /dev/cu.usbserial-0001

                                Connected boards                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃Ser.                         ┃Fam.┃Board                              ┃Version┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│/dev/cu.usbmodem2101         │cpy │UNKNOWN_BOARD                      │v9.2.3 │
│                             │    │Seeeduino XIAO with samd21g18      │       │
│/dev/cu.usbmodem83201        │upy │SPARKFUN_PROMICRO                  │v1.24.0│
│                             │    │SparkFun Pro Micro RP2040 with     │       │
│                             │    │RP2040                             │       │
│/dev/cu.usbmodemNextDraw85111│?   │                                   │       │
│/dev/cu.usbserial-0001       │?   │                                   │       │
└─────────────────────────────┴────┴───────────────────────────────────┴───────┘

Mac OS typically has /dev/cu.* serial devices and /dev/tty.* serial devices. There is a minor difference, but I don't know what.

My Linux mpremote output was a little misleading. My Ubuntu box has taken to listing 31 nonexistent ports on /dev/ttyS{01..31} as being present, so I'd been filtering out all entries containing "None". This machine has a real (non-USB) serial port on /dev/ttyS0 with an ancient 12-bit computer hanging off it, and also a WCH USB serial port connected to a pen plotter. So the full mpremote listing, edited to include real active serial ports, should have read:

/dev/ttyACM0 397539583033 1209:abd1 OpenMV OpenMV Virtual Comm Port in FS Mode
/dev/ttyACM1 0ecb4f7dccb7027b 2e8a:0005 MicroPython Board in FS mode
/dev/ttyACM2 54F926C95259545020312E39262705FF 239a:80cc Adafruit Industries LLC QT Py M0
/dev/ttyS0 None 0000:0000 None None
/dev/ttyUSB0 None 1a86:7523 None None

Pyserial's list_ports.py merely lists the device names.

[Josverl]

Mpflash has a number of built-in filters to remove devices deemed non-relevant, such as bluetooth related ports.
As I don't have a Mac, it took some back and forth to find what to filter on.

mpflash -VVV list will reveal some of the filter's efforts .

[DavesCodeMusings]

It seems like I was looking in the wrong place for the info. The example code from pyserial is printing description, not manufacturer and product. There's a lot of redundancy in the description. For my device, description contains Silicon Labs CP210x USB to UART Bridge (COM3), which is why I thought it was somehow magically detecting the manufacturer and product. I was wrong.

This little snippet illustrates it:

import serial.tools.list_ports

ports = serial.tools.list_ports.comports()

print("Using product...")
for port in sorted(ports):
    print(port.device, port.manufacturer, port.product)

print("Using fallback to description when product is None...")
for port in sorted(ports):
    if (port.product):
        print(port.device, port.manufacturer, port.product)
    else:
        print(port.device, port.description)

Giving output like this:

Using product...
COM3 Silicon Labs None
Using fallback to description when product is None...
COM3 Silicon Labs CP210x USB to UART Bridge (COM3)

So it is an O.S. thing and not an mpremote thing as I originally suspected.

[scruss]

But also a device thing. That script returns

/dev/ttyUSB0 ttyUSB0

on my Linux box. It's a QinHeng Electronics (WCH) CH340 serial converter. Lots of ESP8266/ESP-32 boards use them as their serial to USB connection. So there may be MicroPython running behind one of those and you wouldn't be able to tell. Not without something like what Jos is doing.

[DavesCodeMusings]

It seems it's a much more complex problem than I anticipated at first glance. Though I do feel like I have a better understanding of what the fields mean in the mpremote devs output. Thanks to both of you for helping me to get there!

[Josverl]

So it is an O.S. thing

It's a per OS device driver thing.

You'll get a different description and details based on the device driver installed by default, or manually, or perhaps through Zadig.

[DavesCodeMusings]

In case anyone is interested...

My goal for figuring out what's represented in the fields returned by mpremote devs was to get a list of serial ports for my MPRemote VS Code extension. I really only need the first field (device path), but wanted to understand the others for completeness.

Previously I'd been using the NodeJS library SerialPort to do the detection. But when trying run the VS Code extension on Raspberry Pi, I found that SerialPort has some architecture dependencies and ARM didn't seem to be supported.

Now I have a function that uses mpremote devs to get the same information. I can eliminate the SerialPort library dependency. Theoretically, any device that can run mpremote and VS Code can now use the extension.