Support dynamic USB reconfiguration

[earlephilhower]

Now that the USB descriptor and HID reports are handled by data, it is possible to change them on-the-fly from an application and customize the USB vendor/product/etc.

It is also possible to add and remove HID and other devices, at runtime.

Serial (USB) now properly handles Serial.end() and completely removes the USB CDC descriptor for the Serial port when called. That makes it possible to build sketches which only export HID (Keyboard, Joystick, Mouse) or MSC (FatFSUSB or SingleFileDrive) devices and no Serial port.

All USB devices now register themselves on xxx.begin() and not at the start of the sketch. This could be breaking for sketches which (incorrectly) forget to call Keyboard.begin() before running, for example. In most cases this will be a non-issue because most scripts (all examples here!) so call begin() as appropriate.

Updates the documentation with the new USB calls for end users. Advanced users can check out the shipped libraries for more complicated customization (adding/removing custom devices).

[madias123]

too sad, that this USB stack doesn't include USB-MIDI, because it looks like a real milestone in the Arduino world.

[earlephilhower]

I think I'm reinventing (unintentionally!) the Adafruit TinyUSB lower levels. Not really a milestone, just some convergent evolution. 😆

MIDI takes a large amount of RAM as a buffer when enabled in TinyUSB. The way TinyUSB is architected it really can't dynamically allocate it as needed (similar to LWIP, they want to avoid malloc in an IRQ context). If we enabled MIDI in libpico then all sketches would lose that space, even the vast majority which would never use MIDI.

You might be well served by the Adafruit TinyUSB library, though, since I believe you can configure exactly what's running in the TinyUSB they include there.

[madias123]

This is the funny, groundbreaking part for me:
"Now that the USB descriptor and HID reports are handled by data, it is possible to change them on-the-fly from an application and customize the USB vendor/product/etc." - I don't think this is possible with TinyUSB.

Interesting background with the memory usage of MIDI - I think the large buffer is (historically "was") needed by (massive) Sysex strings. There was a Clavia Nord modular synthesizer that had a real-time editor on a PC/Mac. This communicated exclusively in real time via Sysex! As far as I could remember, there were exactly three MIDI interfaces on the market that could handle this (Most of them failed due to the cheap&slow optocouplers used)

In the meantime, I've become "friends" with the Adafruit TinyUSB stack and write all MIDI-USB routines myself, since all additional third-party MIDI libraries have serious disadvantages/limitations.

[earlephilhower]

I thought Adafruit allowed you to ::end devices, but on further inspection this morning I do see there's only a ::begin.

I went back because I couldn't remember exactly how large a RAM hit using MIDI caused. In #627 it looks like only 400 bytes...for some reason I thought it was on the order of 4K bytes. The flash size comparison there isn't really comparable because of tie -Os that was added.

So I went and tried adding MIDI again. TinyUSB now lets us specify the buffer size (back in #627 it must have had some hardcoded defaults), so I picked 64 bytes for RX/TX because I saw someone else used that for their own bare-SDK code.

@madias123 , Is 64B each way a reasonable FIFO size? I really have no clue...

Blink.ini with this PR and no MIDI

Sketch uses 56472 bytes (5%) of program storage space. Maximum is 1044480 bytes.
Global variables use 9636 bytes (3%) of dynamic memory, leaving 252508 bytes for local variables. Maximum is 262144 bytes.

Blink.ini with this PR and MIDI w/ 64b RX/TX sizes

Sketch uses 57224 bytes (5%) of program storage space. Maximum is 1044480 bytes.
Global variables use 9964 bytes (3%) of dynamic memory, leaving 252180 bytes for local variables. Maximum is 262144 bytes.

So adding MIDI but not using it costs

Flash: +752 bytes
RAM:   +328 bytes

Those numbers don't really bother me so I think I'm fine enabling MIDI in the core if there's some library support. OTW it's basically not usable by folks who want to use it unless they can work with TinyUSB callbacks/etc. That'd be a different PR, but not really a big one. Possibly when the next SDK comes out since it requires a new libpico binary build.

[madias123]

Hi, this would be great news!
about the buffer size:
It really depends and it's all about how to implement sysex. There is (and was!) no real limit for Sysex. In reality it`s something between 8 and 256b.
https://midi.org/midi-1-0-universal-system-exclusive-messages
There is an IEEE Computers abstract, talking about 128b:
IEEE Computer
I searched a little bit in a Steinberg developer forums they are about 256 bytes (talking about professional products with KEIL).
But: The legendary (if it doesn't work on that, it won't work anywhere) Emagic-MT4 MIDI Interface used a 8bit Cypress CY7C64013 -— 256 bytes of RAM— 8 KB of PROM) - so the most common buffer must be under 256b.

There are some additional notes in the original TinyUSB thread, but more about the behavior and not the size:
hathach/tinyusb#377

IMHO: 64 should be more than ok for 99,9% of all applications and there is no use wasting more than this. If more buffer is needed, people will open an issue (or changing the buffer temporary) For all other real time MIDI data it's all about processing time / handling (like MTC or MIDI clock and drop data , like active sensing data) - the pico should more than fast enough for this .

I'm planning in a very next project using the second core of the RP2040 for handling MIDI I/O stream data only - I think this would be a interesting approach