usb: device_next: add new MIDI 2.0 device class

[titouanc]

Add implementation for the 2nd revision of the USB MIDI device class (the MIDIStreaming subclass of the Audio device class), based on the new USB device stack. Moreover, add a sample application to demonstrate usage of this new device class.

This implementation is based on the following documents:

• midi20: Universal Serial Bus Device Class Definition for MIDI Devices Release 2.0
• ump112: Universal MIDI Packet (UMP) Format and MIDI 2.0 Protocol With MIDI 1.0 Protocol in UMP Format Document Version 1.1.2

In this initial implementation, the device class only supports USB-MIDI2.0 (which can convey MIDI1 and MIDI2 data). However, the USB-MIDI2.0 specification requires to define a valid USB-MIDI1.0 interface before the 2.0 one for backward compatibility. Therefore a single interface defined in the device tree will yield 2 USB interfaces (with altsetting 0 and 1). The first one is always a "dummy" one (the minimal interface without any input/output), and the inputs/outputs of the second one are the ones defined in the device tree. As such, data exchange is only possible if the MIDI2.0 (altsetting 1) has been enabled by the host.

Zephyr application code can exchange MIDI data with the host in the form of Universal MIDI Packets through "Group Terminals", as illustrated in midi20: 4. Operational model:

[tmon-nordic]

Are you planning on supporting MIDI 1.0 later? The specification strongly recomments, but does not mandate the MIDI 1.0 backwards compatible interface.

Do you have idea about how to support System Exclusive commands? Should "slicing" the SysEx message be application duty or class?

[titouanc]

Thank you @tmon-nordic for your first comments !

Are you planning on supporting MIDI 1.0 later? The specification strongly recomments, but does not mandate the MIDI 1.0 backwards compatible interface.

Do you have idea about how to support System Exclusive commands? Should "slicing" the SysEx message be application duty or class?

I decided to go for USB-MIDI2.0, because it transports Universal MIDI Packets (UMPs). The intended benefit is that this is an externally defined spec, independent of the transport layer, supporting both MIDI1 and MIDI2, and carrying "routing" (MIDI group) metadata. I believe these packets could be readily used on other transports such as Bluetooth or IP (but this goes beyond my knowledge). On the contrary, USB-MIDI1.0 has a packet format that is specific to this transport, and therefore somehow "leaks" some implementation details that are specific to USB-MIDI.

Moreover, I decided to NOT use a stream-based API (midi_write/read(some_data_bytes) like a serial port), because it avoids the burden of chunking MIDI (application) data into USB-MIDI packets. Such an implementation would require a MIDI parser and some kind of internal state; which I believe is out of scope of a USB class driver. As you noted, this implies that application code has to handle the extra complexity of splitting up long SysEx commands into valid UMPs. If there's a real need here, we could add the following function to the API, that would be in charge of chunking & sending a larger buffer into individual UMPs:

int usbd_midi_send_sysex(const struct device *dev,
                         const uint8_t *data, size_t size);

Overall, sending/receiving simple MIDI events (note on/off, control changes, system messages, short sysex, etc...) is straightforward with this proposal. Long SysEx are already some kind of "advanced" use cases, so I wouldn't try to do more about it here.

Even though USB-MIDI1.0 transport is not supported, I noticed that I needed to add that "dummy" backward compatible interface before the USB-MIDI2.0 one, otherwise the USBD MIDI Sample wouldn't enumerate properly. I tested both with Linux 6.11/alsa 1.2.12 and Mac OS X 15.1, but I'd be glad to hear about any experience with other hosts (like Android, iOS or Windows). For the record, here is how it looks on Mac OS:

And on Linux:

$ amidi -l
Dir Device    Name
IO  hw:2,1,0  Group 1 (USBD MIDI Sample)

I haven't planned on implementing a USB-MIDI1.0 stack, because 2.0 works for me, and its features are a superset of 1.0. Moreover, I haven't a clear idea on how it would look like for application code (are the 1.0/2.0 altsetting distinguably usable from dt/code ? How does the application code know which is selected, and how to format packets ? Otherwise do we have a translation layer 1.0<->2.0 depending on the selected altsetting ?).

Finally, I will dig into the full-speed/high-speed descriptors and amend the PR accordingly.

Thank you again for looking at this !

[tmon-nordic]

Even though USB-MIDI1.0 transport is not supported, I noticed that I needed to add that "dummy" backward compatible interface before the USB-MIDI2.0 one, otherwise the USBD MIDI Sample wouldn't enumerate properly.

It wouldn't enumerate properly because it is mandatory for the MIDI 2.0 interface to be on alternate setting 1. Take a look at USB MIDI 2.0 Specification and note where "should" (recommended but optional) is used and where "shall" (mandatory) is used.

If there's a real need here, we could add the following function to the API, that would be in charge of chunking & sending a larger buffer into individual UMPs

I don't quite know about MIDI 2.0 but in MIDI 1.0 the most significant bit is reserved for Real Time messages. All SysEx commands (and responses) do have the most significant bit cleared to enable having the Real Time message while the SysEx is being transmitted. To what degree this is a problem is unknown to me, but USB-MIDI 1.0 chunks everything into 4 byte USB-MIDI Event Packets that can be freely interleaved.

While this API would be useful for applications that use SysEx messages exclusively (e.g. Digitech effect pedals, see https://github.com/desowin/gdigi for open-source host-side implementation), I have no idea if would introduce problems when chunk interleaving is necessary.

Moreover, I haven't a clear idea on how it would look like for application code (are the 1.0/2.0 altsetting distinguably usable from dt/code ? How does the application code know which is selected, and how to format packets ? Otherwise do we have a translation layer 1.0<->2.0 depending on the selected altsetting ?).

This is something that would have to be solved if we wanted proper backwards compatibility. The alt setting is only known at runtime and is entirely host-dependent. The class would have to somehow provide the information about selected protocol version to the application. About the format conversion I have no idea.

[kartben]

Pxl.20241119.152833728.mp4

@titouanc this is really cool :)

[tmon-nordic]

Wrap macro values in parentheses to avoid potential issues on macro use.

[kartben]

@titouanc wondering if you missed some of my comments regarding the code sample? Thanks!

[jfischer-no]

It looks like you used markdown formatting(?) in the commit message, what is the intent behind that?

[titouanc]

Because I initially had one commit, and because Github uses the commit message by default for the PR description, this made for a nice description.

Will rewrite the commit message with the bare links only 👍

[jfischer-no]

Thanks for adding it. usb_midi_ump should be just midi_ump, if I am correct there are no USB dependencies in "Universal MIDI Packet definitions". IMO still okay to link it here.

[diegoherranz]

I think a similar reasoning can be applied to the recently renamed include/zephyr/audio/midi.h. There are many references to USB there, which I think could be removed since that could be used for transports other than USB.

[jfischer-no]

No, what you call a sample is half of the driver. It is not an application or sample code just because it is placed in samples. That is part of the MIDI driver. I put it there as a requirement to get it in.

[jfischer-no]

I have expected to see int usbd_midi_send(const struct device *dev, const uint32_t ump[static 4]); here. struct midi_ump is just unnecessary dependency. Not sure why it is different now, and how it helps to mark unresolved conversation resolved or in general in review.

[titouanc]

As per your suggestion, I had in an intermediate version const uint32_t ump[static 4] (which is also my favorite version by the way). However, because C++ does not understand T [static N], this would prevent any C++ application to use the USB-MIDI class, because it would fail to compile any file that includes usbd_midi.h.

While I agree this adds a layer of indirection (struct midi_ump->data), this at least allows both C and C++ application to work with the USB-MIDI class

[jfischer-no]

#81197 (comment)

[titouanc]

Sorry to over-write on this, but I just want to make sure that I fully understand your intent. When you say

what you call a sample is half of the driver.

this is because usbd_midi_send is not a syscall, and therefore the caller is not isolated from kernelspace ? Is there any other reason ?

Like I said, I prefer the simplicity of uint32_t [static 4], but this is functionally equivalent to struct {uint32_t [4]}, and they have the same memory layout. Besides esthetic considerations, is there any other advantage of the former over the struct form ?

In my project using usb-midi, I implement UMP Stream (discovery and dynamic function block configuration). I do the following, which just works fine (in plain C), so I don't need to bother with structs at all:

uint32_t reply[4];
/* build reply */
usbd_midi_send(dev, (const struct midi_ump *) reply);

If we are to go for the static array form, how are C++ users supposed to do ? If I understand correctly, they would need to add a some boilerplate (to compile in C) that understands uint32_t [static 4] and expose a C++ friendly interface, which sounds like an extra burden without clear advantage to me. Do you have any other suggestion ?

Thanks again for the review, and sorry for the lengthy discussion.

[jfischer-no]

No, because the sample is actualy the driver. You are using the Zephyr driver model and DT to implement a MIDI driver with a USB interface. In Zephyr, the drivers are written in C. If someone for whatever reason decides to write them in C++ or some other language in downstream work, it is their responsibility to provide the proper interface. I do not see why the code here should suffer if someone uses a different language. There are no real dependencies on UMP in the USB MIDI2 spec, other than determining the length. The macro to get the length can be defined in usbd_midi.c, and there would be no need to include midi.h. Together with int usbd_midi_send(const struct device *dev, const uint32_t ump[static 4]);, this implementation would provide a tiny, clean and safe interface, independent of any future changes to midi.h.

[stemschmidt]

Feel free to correct me if I am on a complete wrong track, but an application will send midi data with usbd_midi_send. Therefore it has to include include/zephyr/usb/class/usbd_midi.h

According to this information from the zephyr doc page (https://docs.zephyrproject.org/latest/develop/languages/cpp/index.html#header-files-and-incompatibilities-between-c-and-c) some care has to be taken to allow headers to be included from C and C++.

If "uint32_t [static 4]" is not available in C++ it should not appear in a header file that has to be included in order to use a certain functionality. This is not writing a driver in another language, this is simply ensuring that the driver can be used from C and C++.

[jfischer-no]

@stemschmidt It might help if you read my comments where I said that it is still the driver that calls usbd_midi_send(). That is the design behind using the Zephyr device model and devicetree. Limitations of the language you use must not weaken the upstream code.

[stemschmidt]

@jfischer-no I read it, but I did not understand it. :-) And still do not understand it: How should an application send MIDI messages?

[titouanc]

Thanks again for you latest review @jfischer-no

I just applied your comments, and rebased on top of main:

• Rename ALT_USB_MIDI_x into MIDIx_ALTERNATE
• Pass ump by value (instead of by pointer) in usbd_midi_send and rx_packet_cb. Also adapted the macros in midi.h to take a struct (and not a pointer) as parameter for consistency
• Remove all references to USB in midi.h and attach it to the audio group in Doxygen
• Rewrite the first commit message to remove Markdown-like syntax

[jfischer-no]

There are a few inconsistencies, please fix them.

[jfischer-no]

Do you still need to include it?

[titouanc]

Yes, there are a handful of defines that are reused from there (to name a few):

• AUDIO class code https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/usb/device_next/class/usbd_uac2_macros.h#L51
• MIDISTREAMING subclass code https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/usb/device_next/class/usbd_uac2_macros.h#L57
• AUDIOCONTROL subclass code https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/usb/device_next/class/usbd_uac2_macros.h#L55

[titouanc]

Hello @jfischer-no ! I just applied your comments and rebased on top of main:

• rename (usbd_)midi -> midi2
• Improve consistency of comments/documentation relative to USB-MIDI/USB_MIDI/MIDI2 etc...
• Change ops.status_change_cb(dev, enum status) -> ops.ready_cb(dev, const bool)
• Use n instead of inst in DT macros
• Clean up LOG_DBG messages to remove unnecessary USB-MIDI prefix
• Rename DT bindings
• Return -ENOBUFS instead of -EAGAIN in usbd_midi_send if there isn't enough space in the Tx buffer

[jfischer-no]

LGTM, thank you @titouanc, happy FOSDEM time.

[titouanc]

Thanks a lot for getting this ready just on time @jfischer-no 🙌

[kartben]

w00t! Thanks @jfischer-no for your help!

[kartben]

Suggested change

	* @return 0 on success
	* -EIO if MIDI2.0 is not enabled by the host
	* -EAGAIN if there isn't room in the transmission buffer
	* @retval 0 success
	* @retval -EIO MIDI 2.0 is not enabled by the host
	* @retval -EAGAIN there isn't room in the transmission buffer

[kartben]

was left in a pending review of mine, apparently -- will be good to address as a follow-up

[diegoherranz]

Many thanks for your work here, guys. Really keen on giving this a try.

[ho-ho-ho]

I don't know if I should open a new issue for that or not, but I've just managed to test it under arch linux (works flawlessly) and Windows 10 (22H2) ... and I had to find out that windows does not have a MIDI 2.0 driver yet, so it doesn't work at all. It looks like they are in the process adding it with a preview version out last week, but until that goes live this driver won't work on windows :(