PICO and PICO2 crash when unplugging and replugging USB

[GitHubsSilverBullet]

With the PICO(2) plugged into a powered hub, it should be possible to cut the connection between PC and Hub. That works fine with a lot of devices, doesn't play well with PICO(2) running Micropython.
When cutting the connection between PC and Hub, it has been verified (measured) that the µC still gets the 5V from the hubs power supply.

Sometimes Micropython crashed upon cutting the connection, sometimes not.
If it survives the disconnect and continues blinking, then it'll get killed upon reconnection.

Testprogram includes ASCII drawing and a dmesg log.
Even without the print (ie. not using the USB CDC) it still crashes.

#!/micropython
# -*- coding: UTF-8 -*-
# vim: fileencoding=utf-8: ts=4: sw=4: expandtab:

#╭────────────────────────────────────────────────────────────────────────╮
#│   MicroPython v1.25.0 on 2025-04-15; Raspberry Pi Pico2 with RP2350    │
#╰────────────────────────────────────────────────────────────────────────╯
#                            ╭───────╮
#                            │ Power │
#                            ╰───┰───╯
#   ╭────────────╮          ╭────┸────╮          ╭──────╮
#   │     PC     ├────╳─────┤ USB-Hub ├──────────┤ Pico │
#   ╰────────────╯          ╰─────────╯          ╰──────╯

#━━━━━[ Plug-in Pico2 with MicroPython ]━━━━━
#[ 3459.883560] usb 1-1.3.3: new full-speed USB device number 58 using ehci-pci
#[ 3459.994235] usb 1-1.3.3: New USB device found, idVendor=2e8a, idProduct=0005, bcdDevice= 1.00
#[ 3459.994239] usb 1-1.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
#[ 3459.994241] usb 1-1.3.3: Product: Board in FS mode
#[ 3459.994243] usb 1-1.3.3: Manufacturer: MicroPython
#[ 3459.994245] usb 1-1.3.3: SerialNumber: 8946bf8bd11a23da
#[ 3459.994885] cdc_acm 1-1.3.3:1.0: ttyACM0: USB ACM device

#━━━━━[ loading and running testprogram ]━━━━━

#━━━━━[ Unplugging Hub OK, Pico2 and Micropython continue to run ]━━━━━
#[ 4090.387778] usb 1-1.3: USB disconnect, device number 71
#[ 4090.387783] usb 1-1.3.3: USB disconnect, device number 72

#━━━━━[ Replug Hub crashes MicroPython, testprogram is gone ]━━━━━
#[ 4096.240259] usb 1-1.3: new high-speed USB device number 73 using ehci-pci
#[ 4096.350276] usb 1-1.3: New USB device found, idVendor=05e3, idProduct=0608, bcdDevice=77.64
#[ 4096.350280] usb 1-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
#[ 4096.350282] usb 1-1.3: Product: USB2.0 Hub
#[ 4096.350836] hub 1-1.3:1.0: USB hub found
#[ 4096.351266] hub 1-1.3:1.0: 4 ports detected
#[ 4096.640218] usb 1-1.3.3: new full-speed USB device number 74 using ehci-pci
#[ 4096.750659] usb 1-1.3.3: New USB device found, idVendor=2e8a, idProduct=0005, bcdDevice= 1.00
#[ 4096.750664] usb 1-1.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
#[ 4096.750666] usb 1-1.3.3: Product: Board in FS mode
#[ 4096.750668] usb 1-1.3.3: Manufacturer: MicroPython
#[ 4096.750669] usb 1-1.3.3: SerialNumber: 8946bf8bd11a23da
#[ 4096.751165] cdc_acm 1-1.3.3:1.0: ttyACM0: USB ACM device

from machine import Pin
from time import sleep_ms, ticks_ms

led = Pin(25, Pin.OUT, 0)
ledblink = True
counter = 0
while True:
    while ticks_ms() % 100 < 70:
        sleep_ms(20)
    while ticks_ms() % 100 >= 70:
        pass
    if ledblink:
        led.toggle()
    try:
        print((counter:=counter+1), end=' ')
        # pass
    except Exception as e:
        ledblink = False    # Stop blinking
        led(1)              # LED permanently on

[peterhinch]

I think this is to be expected. I have encountered similar behaviour with other MicroPython targets when powered from an external supply.

For reliable debugging direct the REPL to a UART and use an FTDI adaptor. This approach also enables debugging applications which use deepsleep.

[GitHubsSilverBullet]

I think this is to be expected.

That's a, hmmm, how shall I put it, a surprising statement. ;-)

One would assume that once the CDC times out, Micropython would simply try to reestablish a reliable comm channel. A hard device reset isn't necessarily what comes to mind first. And the fact that it sometimes crashed on disconnect, other times on reconnect does not instill confidence.

So for a system that needs to be connected and disconnected while running, it would be best to rip out the entire USB subsystem from firmware and reprogram that CDC functionality in a loadable module. That way one could catch error conditions and handle them in a graceful manner.

[Josverl]

I'm pretty sure that I had a simple demo running on a rp2 makerboard that kept running reliably while plugging/unplugging the board.

It used async, or it could be that the USB CDC hardware was just better buffered on that board
I don't have the board anymore, but I'll try to repro with something else from the grabbox

[GitHubsSilverBullet]

Thanks @Josverl

Actually I tried it with original PICO, PICO2 boards as well as cheap china clones.
All were external powered (5.04V…5.05V (including 1000µF Elko || 1µF ceramic cap.))
They all crash, as soon as either a disconnect or a reconnect occurs.
And they even crash when there's no USB CDC communication going on at all.
Of course, the USB stack remains active in the background and responds to the incoming polling from the PC.

Here's the interesting part, which I noticed later.
A regular shutdown down followed by a regular boot of the PC didn't disturb the µC.
I therefore grabbed an old trusty RasPi with Debian Linux and set it up in a specific way.
Also connected the aforementioned powered hub and PICOs to it.

An external system engages a relay which in turn powers the RasPi for exactly five minutes.
Enough to boot into Debian, automagically start a terminal, verify that data from the PICO is still incoming, then shutting down.
After five minutes the relay disengages, thus removing any power from the RasPi for one minute.
(of course, the powered hub remains powered )
This six minute cycle repeats. Ten boots per hour…

Had it running for more than a day and more that 250 boot cycles and not a single problem occurred.
Then, just as a confirmation I disconnected the PICO and it crashed within a second.

That leads to one and only one conclusion:
Some USB status changes are handled properly, others are not handled and will crash and hard reset the µC.

[Josverl]

Not my experience at all.

• MicroPython v1.25.0 (PIMORONI_PICOLIPO_16MB )
• USB & LiPo powered PicoLipo 16
• NP strip to show activity
• bolt_np.py script with a clear RED pattern that shows restart
• main.py imports the linked script

Runs rock solid through the following:

• Device USB connect / Disconnect ,
• USB Power Up / Power Down. ( Switch on USB hub)
• Cable -> MCU unplug / re-plug
• PC Cable to USB hub plug / unplug
• Power to hub disconnect / connect

It just keeps running 🏃‍➡️🏃‍♀️‍➡️🏃‍♂️‍➡️, as long as there is power from either source

Looking at the schematics the PicoLipo actually has a bit more hardware to make sure there is not just power, but it is stable/conditioned, and that seems to pays off in stability

I think your problem may be due to your boards getting spikes and powerdips on the VCC while connecting / disconnecting that you probably wont see them using just a multimeter.

[GitHubsSilverBullet]

@Josverl
thanks a lot for taking the time.
Disconnecting and reconnecting the PICO directly doesn't cause the problem.

#                            ╭───────╮
#                            │ Power │
#                            ╰───┰───╯
#   ╭────────────╮          ╭────┸────╮          ╭──────╮
#   │     PC     ├────╳─────┤ USB-Hub ├──────────┤ Pico │
#   ╰────────────╯          ╰─────────╯          ╰──┰───╯
#                                                   ┃
#                                                   LiPo

It's disconnecting the hub and reconnecting it.
But in order to make sure that power spikes don't cause problems, I soldered a BAT54 to Pin39 (Vsys) and attached a LiPo directly. The RT6150 can handle 1.8V to 5.5V and so the 3.7V are in the cozy range. Now we have two independent power sources (both through Schottkys)

Disconnecting and reconnecting the PICO into the hub is OK.
Cutting the connection between Hub and PC (the large X) still causes the hard reset.
Didn't even need more than a simple machine.PWM(25,freq=10,duty_u16=2**15) to show it.

So it seems that this introduces some USB status packets which are not handled gracefully at all.

[Josverl]

that still does not explain why I cant reproduce the issue, with the same disconnection of the hub from the Win11 host
I did the same , though my host is Windows 11 and I know that there is a difference in DTR/RTS handling between *nix and Windows
I have seen these trip ESP32 into resets when mpremote was still using the default/naive *nix approach on Windows.
that was fixed quite some time ago but it may be an indication that perhaps the before/after states of the DTR/RTS signals may be worth looking onto

It's disconnecting the hub and reconnecting it.

other hubs , does the hub have power

[GitHubsSilverBullet]

there is a difference in DTR/RTS handling between *nix and Windows
before/after states of the DTR/RTS signals

Hmm, there isn't any RS232 involved here at all.
There's only a plain USB-FS integrated on-chip solution.
It's USB all the way from PC to hub to various devices including the PICOs.
And USB is per definition plug and play and not plug and pray.

[Josverl]

And USB is per definition plug and play and not plug and pray.

well , that has surely always been the intent

[shariltumin]

I have a Pico running a simple test program when it boots up. The test script is frozen into the firmware and started by the main.c before the REPL loop.

Pulling the USB hub out of the Linux PC did not cause the Pico board to perform a hard reset.

$ tio /dev/ttyACM0
...
My app running at: 722
My app running at: 723
My app running at: 724
My app running at: 725

[tio 19:31:10] Disconnected
[tio 19:32:35] Connected
My app running at: 743
My app running at: 744
My app running at: 745
My app running at: 746
...

The dmesg log shows that the MicroPython USB device recovered gracefully.

$ dmesg
...
[33256.059909] usb 1-7: USB disconnect, device number 12
[33256.059920] usb 1-7.3: USB disconnect, device number 13
[33256.060127] usb 1-7.4: clear tt 1 (9192) error -71
[33256.379165] usb 1-7.4: USB disconnect, device number 14
[33256.379168] usb 1-7.4.1: USB disconnect, device number 15
[33256.379862] usb 1-7.4.2: USB disconnect, device number 16
[33256.380017] cp210x ttyUSB0: cp210x converter now disconnected from ttyUSB0
[33256.380028] cp210x 1-7.4.2:1.0: device disconnected
----
[33111.077301] usb 1-7: new high-speed USB device number 12 using xhci_hcd
[33111.225344] usb 1-7: New USB device found, idVendor=1a40, idProduct=0101, bcdDevice= 1.11
[33111.225360] usb 1-7: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[33111.225366] usb 1-7: Product: USB 2.0 Hub
[33111.227363] hub 1-7:1.0: USB hub found
[33111.228146] hub 1-7:1.0: 4 ports detected
[33111.617278] usb 1-7.3: new full-speed USB device number 13 using xhci_hcd
[33111.821301] usb 1-7.3: New USB device found, idVendor=248a, idProduct=8367, bcdDevice= 1.00
[33111.821316] usb 1-7.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
...
[33340.838473] usb 1-7.4.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[33340.838475] usb 1-7.4.1: Product: Board in FS mode
[33340.838476] usb 1-7.4.1: Manufacturer: MicroPython
[33340.838477] usb 1-7.4.1: SerialNumber: e6625887d3335c32
[33340.846732] cdc_acm 1-7.4.1:1.0: ttyACM0: USB ACM device
[33341.024451] usb 1-7.4.2: new full-speed USB device number 21 using xhci_hcd
[33341.255961] usb 1-7.4.2: New USB device found, idVendor=10c4, idProduct=ea60, bcdDevice= 1.00

Yes, I can also connect and get output with mpremote. Of course; mpr a0 ls : will result in "mpremote.transport.TransportError: could not enter raw rep".

Ctrl+C and Ctrl+D do not work.

Can the hard-reset happen due to the REPL loop?

[shariltumin]

A simple script from the REPL works too. So nothing to do with REPL

$ tio /dev/ttyACM0 
[tio 21:47:34] tio v1.32
[tio 21:47:34] Press ctrl-t q to quit
[tio 21:47:34] Connected
MicroPython v1.25.0-preview.4.g9ea8d2a03.dirty on 2024-11-13; WaveShare RP2040 Zero with RP2040
>>> 
>>> 
>>> import time
>>> cnt =0
>>> while True:
...     print(f'Hello at {cnt}')
...     cnt += 1
...     time.sleep(5)
... 
Hello at 0
Hello at 1
Hello at 2
Hello at 3
Hello at 4
Hello at 5
Hello at 6
Hello at 7
Hello at 8

[tio 21:49:24] Disconnected
[tio 21:49:43] Connected
Hello at 9
Hello at 10
Hello at 11
Hello at 12
Hello at 13
Hello at 14
Hello at 15
...