multi-midi: Introducing a new multi-port hardware + USB MIDI library for RP2 boards (and a question for the community)

[HLammers]

I’m happy to announce a new MicroPython library for RP2 boards, providing an interface for UART and PIO based hardware MIDI and USB MIDI 1.0: https://github.com/HLammers/multi-midi/

This is a follow up to #17640

Features

• Supports multiple MIDI IN and OUT ports (UART, PIO and USB)
• Handles real-time SysEx and standard MIDI messages
• Handles running status for hardware MIDI IN and OUT
• Manages the prioritization of MIDI Real-Time messages
• Configurable MIDI message filtering (message type, channel and CC value)
• Configurable fast-track MIDI Real Time message routing (one source IN port to many destination OUT ports)
• Asynchronous, non-blocking event processing
• Modular structure for integration into custom hardware projects
• Highly optimized code to minimize latency
• Example code for quick testing and adaptation

Question For the Community

As I describe in much detail in the PROCESS_MAP.md file (with all the pros and cons), I’m still considering whether I’ve followed the right approach, or I should try a different one:

Is it really necessary and worth the considerable effort to refactor the current asyncio based approach into an approach based on dedicating the second core to a viper code and direct register access based low-level approach?

To answer that question I probably first need to get a better understanding of how well the asyncio based implementation performs: try it out with real MIDI hardware (putting my collection of drum computers to work) and find a way to actually measure its current performance.

On the one hand I’m feeling drawn to taking the challenge of building a near-low-level version of multi-midi, but on the other hand I’m afraid it’s going to consume too much time – this project already took a little bit more time and effort than I expected…

Perhaps it’s also possible to look for hybrids of the current asyncio based approach and a complete low-level refactoring:

• Only replacing the current IRQ + event based approach to I/O with a polling approach based on direct register access.

  Disadvantage: This goes against the asyncio philosophy and it can be tricky to maintain the flow of the event loop, but it can be done…

• Only move the lowest level MIDI processing (merging data streams and I/O) to the second core.

  Disadvantage: In such case the second core could not be used anymore for off-loading heavy tasks, introducing a risk of more jitter because some MIDI processes depend on the main core.

I’d love to read the expert opinion of others who have more experience writing (semi-)low-level code than I have (and who can perhaps help me here and there if I decide to go that route…): what do you recommend? To do or not to do?

[GitHubsSilverBullet]

Is it really necessary and worth the considerable effort to refactor the current asyncio based approach into an approach based on dedicating the second core to a viper code and direct register access based low-level approach?

asyncio acts as a cooperative and not a preemptive scheduler.
And it is magnitudes slower than a real core (when comparing task switches, for instance).
Which in turn means, the whole cardhouse of asyncio tasks, but certainly not you, controls the timing.

I presume that timing is of critical importance when dealing with MIDI messages. While it might be possible to achieve something acceptable, all pieces have to fit perfectly and yield as fast as possible. And timing still wouldn't always be reproducible.
Just my 2ct.

[HLammers]

That's exactly my reasoning

[sosi-deadeye]

Maybe you want to see this: https://youtu.be/02CLD-42VdI?si=XRl9gpXMgzuOIG1l&t=4507

[HLammers]

Thank you, I will!

[peterhinch]

As a very general comment my approach would be to start with asyncio (as you did), then thoroughly test in real-world situations, trying to identify any shortfalls. Assuming some are found, try to address the shortfalls by optimising the existing code. If that proves impossible, then look at other approaches to concurrency.

Multi-core work is difficult and can harbour subtle, hard to find, bugs. While powerful, it's something of a last resort (IMHO).

[HLammers]

Thank you for your advise.

The multi-core stuff is not what scares me off: I’m using it in Cybo-Drummer, so I know how to get it working and anyway: the amount of communication between the second core and the main core would be very limited (sending MIDI messages from the application and MIDI message callbacks to the main code when receiving messages), the easiest way to do that is probably using the micropython.RingIO class (which in my testing turned out to be pretty fast).

The challenge is more that I’d have to do all the scheduling of parallel(ish) tasks manually.

But you’re right: lets focus first on benchmarking what I’ve already written and indentifying any bottlenecks I could find. I’ve already done a lot of unit testing and micro-optimization, but there’s always more…

But for now I’m going to let it rest a little (if I can control myself, haha), because finishing this has dominated my spare time a little too much recently…