MIDI over (WiFi) TCP

[espruino-discuss]

Posted at 2019-08-04 by indianajones

Looking at the Midi support for Espruino, it appears as though the only ports you can send MIDI using this module is via Serialx. I want to send MIDI over EspruinoWifi module. Is there any support for that (yet)?

[espruino-discuss]

Posted at 2019-08-05 by Robin

Sun 2019.08.04

Agree @IndianaJones, not much excitement on the MIDI topic yet.

Looking over:

http://www.espruino.com/Midi
http://www.espruino.com/BLE+MIDI
http://www.espruino.com/modules/ble_midi.js
and for WiFi
http://www.espruino.com/Interactive+Web+UI

couldn't a server be created, and mimic the same methods to communicate as in the source for the BLE version? Although I don't know of a well done example, others did help me a while back. Start at the last entry and work backwards with these examples to get an idea of what would be needed.

STA or AP mode browser unable to connect to ESP8266-12

Sidebar: Interesting you bring up MIDI. During the mid seventies, nearly a decade before Bill Gates would be on the radar and a household name, I cobbled together (a full 8 x 10in wire wrap project - no soldering) a sequencer with an 8088 and 8255 running commands written in assembler, along with whopping 2K memory chips to a Roland Juno-60, one of the last analog synths without MIDI, all over serial RS232! This was several years before MIDI would take hold. The Yamaha DX-7 comes to mind that exploited the MIDI benefits.

---

Posted at 2019-08-05 by indianajones

Wow, you predate me! I remember the iconic DX-7, but I went with the Roland D-50, which came out a few years later I believe. Still have it.

---

Posted at 2019-08-05 by Robin

Wow, you predate me!

Is that the new 2019's way of being polite? ;-) Yeah, where did that forty years go?

You should see my 45's and album collection and rotary dial corded phone! The Juno-60 still works, along with that micro, but is packed away after my last move. Still play my Fender Rhodes Mark I suitcase piano.

What are your thoughts on creating a WiFi server to solve your issue?

I'm guessing you have the MIDI command set for the audio mixer? Shouldn't be that much of a diversion from the BLE example.

---

Posted at 2019-08-05 by indianajones

The target mixer manufacturer (Allen & Heath) published their MIDI spec. It uses NRPN, so minor additional complications, but it's all there.

Re: Wifi server, I don't know yet. Coming from the world of cloud-based software, which is my day job, I think of server as a separate entity with an API and all that. All I need is a listener for MIDI (over TCP) messages from the mixer, and a sender to send the messages. Maybe your definition of server is different from mine. Feel free to expound. I'm checking out the links you provided.

You have a suitcase Rhodes? I'm jealous! Hey you might like my son's music, he's the only kid I know making real music these days - the best of 60's and 70's sound. Check him out: https://open.spotify.com/artist/4ybii90Ofz5HRJyYoTLfiq

---

Posted at 2019-08-05 by Robin

I'm certainly not qualified to provide a definitive answer as I only dabbled with that two years ago, but as it is possible to have the server running on EspruinoWiFi for instance, build and deploy web page content in the browser, listen to button click events and send data back-n-forth using query string name/value pairs, I don't see why it wouldn't be possible to perform similar tasks using send and recv functions in the same manner.

I was hoping others with more experience here could help out with links and ideas, but surprised no one has chimed in yet.

Will check out the 'real music' link tomorrow

---

Posted at 2019-08-05 by @gfwilliams

Hi! As you say, for MIDI you just need a Serial transmit pin.

You're totally fine to use Espruino WiFi for that. One serial port (Serial2) is used for the Wifi, but the other one (Serial1) is totally free to use and is available on B6 - so you should be sorted.

More info on pins here: https://www.espruino.com/WiFi#pinout

Hope that's some help?

---

Posted at 2019-08-05 by indianajones

@gfwilliams wait, confusion still reigns, my fault not yours. Are you saying that Serial2 is the wifi connection on the Espruino wifi, by default? I don't understand how that would work. I have to send TCP packets over wifi that contains MIDI commands in them, because the target device is listening on TCP for MIDI data. If the target device was a serialport-listening device like normal MIDI, serial port makes sense. Please (try to) clear the fog in my brain. Thanks.

Currently, my plan is to roll my own MIDI implementation using require('Wifi') and require('net'). And by "roll my own" I mean "just enough to get it to work".

---

Posted at 2019-08-05 by @gfwilliams

Ahh ok, so you're basically saying:

• There's some input device which is wired up to an Espruino WiFi (not via MIDI)
• You want the Espruino WiFi to create MIDI data based on the input from that device and then send it over WiFi?

The existing MIDI module only receives MIDI commands by the look of it, so if you're sending them you'll have to roll your own, but that's not too hard... And I assume that MIDI over TCP really is just sending the same raw MIDI data but over TCP not Serial?

All you need to do is follow the code at https://www.espruino.com/Internet#sockets to create a socket connection, then when you're connected you send the raw MIDI data with client.write(); - you just want to make sure you send the data as binary, so something like:

function sendNote(channel, on, pitch, velocity) {
  socket.write(E.toString([
    (on?0b10010000:0b10000000)|channel,
    pitch,
    velocity]))
}

---

Posted at 2019-08-05 by indianajones

Yes, MIDI over TCP is just the same MIDI command sequence, only over a TCP connection instead of serial connection. I'm sure there's some kind of "hey this is a MIDI packet" header or something, which I'm still researching, but the MIDI part is typical MIDI.

Thanks for the info, I'll start working on it. And I'd be happy to submit the module to the Espruino project when I'm finished.

---

Posted at 2019-08-06 by @gfwilliams

And I'd be happy to submit the module to the Espruino project when I'm finished.

That'd be great - thanks! If your module were just to use .write (and assume a connection was already established) then nothing stops it from being used for Serial MIDI devices too, because Serial also has a write method.

---

Posted at 2019-08-07 by indianajones

@gfwilliams it looks like the MIDI device (Allen & Heath mixer) communicates using UDP, not TCP. I didn't see a way to specify protocol (TCP, UDP, etc) for the .net library, but I see the choice in NetworkJS. Unless you know something I don't.

---

Posted at 2019-08-07 by @gfwilliams

Hmm, looks like there aren't specific docs on it - I should add that. UDP is handled in the same way as is done for Node.js.

As an example: https://github.com/espruino/Espruino/blob/master/tests/test_dgram_socket.js

let client = dgram.createSocket('udp4');
client.on('message', function(msg, info) {
  console.log(">"+JSON.stringify(msg));
  console.log(">"+JSON.stringify(info));
  // you probably don't care about this
});
// to send a midi note, you do:
client.send('your data', port, 'servername');

---

Posted at 2019-08-07 by indianajones

I'll give that a go, thanks!

---

Posted at 2019-08-07 by indianajones

Turns out I'm mostly wrong -- the mixer sends out a UDP packet containing only its name ('QU-SB'). Apparently that's merely for identification, not sure. But real communication happens on TCP. So back to TCP.

---

Posted at 2019-08-08 by AkosLukacs

Saw this on Nordic's site: : MIDI over Bluetooth LE
Not Espruino specific, not over Wifi, but might contain some useful info

---

Posted at 2019-08-08 by indianajones

Thanks, @AkosLukacs, I'm sure there will be some helpful info in there.

---

Posted at 2019-08-08 by indianajones

@gfwilliams, to establish and maintain connection to my MIDI device over TCP, I have to send packets with various TCP flags set. In particular, I have to initiate with a SYN, and then respond to future requests with ACK. So it goes like this:

Me: SYN
MIDIDevice: PSH, ACK
Me: ACK
MIDIDevice: PSH,ACK
Me: ACK

That goes on forever. I assume this is just normal keep-alive stuff for TCP connections (I haven't played with TCP at this level for a while).

I got this data by using wireshark and watching how the A&H TCP MIDI driver connected to the mixer. The ACK back-and-forth is what goes on after they connect. When I connect with the Espruino Wifi, it connects and sits there. I assume because I have to send some establishing packet like the A&H driver did (the SYN packet).

Any thoughts on what I'm missing?

---

Posted at 2019-08-08 by @gfwilliams

Are you sure that doesn't just happen automatically with TCP? Did you try basically just using the example I'd posted?

---

Posted at 2019-08-08 by indianajones

Exactly! Yeah I'm using your code, tweaked for connecting to the particular mixer, but pretty much the same. I prolly need to do some more digging before I publicly whine about it.

---

Posted at 2019-08-08 by maze1980

It should go like this:

Me: SYN
MIDIDevice: SYN, ACK
Me: ACK
MIDIDevice: PSH,ACK
Me: ACK

And having access to Wireshark you can look at the data sent.

---

Posted at 2019-08-08 by indianajones

@maze1980 yeah I forgot to add the data, which is super-important, and I've made it a little bit further because I forgot to check wireshark for broadcast packets. So the real A&H driver comms go like this:

Me: UDP 'Qu Find' on 255.255.255.255:51320
MidiDevice: UDP 'Qu-SB' and { address: 10.11.12.13, port: 51320, size: 6 }
Me: TCP connect to 10.11.12.13:51325 // note that 51325 is the TCP port
MidiDevice: TCP 0xEF // <- this is MIDI ActiveSense bit, sent every 300ms forever
Me: TCP ACK

So the data you were asking for is the 0xEF byte, which is what's sent (and ACK'd by 'me') forever. Here's how my Espruino code runs:

Me: UDP 'Qu Find' on 255.255.255.255:52320
(nothing happens, so instead...)
Me: UDP 'Qu Find' on 10.11.12.13:51320 // because I know the IP address already
MidiDevice: UDP 'Qu-SB' and { address: 10.11.12.13, port: 51320, size: 6 }
(cool! so far, so good)
Me: TCP connect to 10.11.12.13:51325
Me: (I get the 'connected' callback, and client.on('data', ...) is never called

So I'm doing something wrong still, because I don't get the 0xEF message ever.

---

Posted at 2019-08-08 by indianajones

Hey, why don't I also include my code?

		let udpClient = this.dgram.createSocket('udp4');
		udpClient = udpClient.bind(51320, function() {
			udpClient.on('message', function (msg, info) {
				console.log(">"+JSON.stringify(msg)); // <- here is where I get 'Qu-SB ...' from the mixer
				console.log(">"+JSON.stringify(info));

				let tcpClient = require('net');

				tcpClient.connect({port: 51325, host: "10.1.64.22"}, function() {
					console.log("connected");
				});
		
				tcpClient.on('data', function(chunk) {
					console.log("tcp data is " + JSON.stringify(chunk));
				});

				tcpClient.on('end', function() {
					console.log('client disconnected');
				  });
			});
			udpClient.send("QU Find", 0, 7, 51320, "10.1.64.22");
		});

10.1.64.22 is the Qu Mixer, which is the Midi device I'm trying to reach.

---

Posted at 2019-08-08 by indianajones

Here is a forum posting from an arduino project that is connecting to a Qu-16 (same series as mine), and I'm doing basically what he's doing: https://forum.arduino.cc/index.php?topic=230326.0

---

Posted at 2019-08-08 by indianajones

One more item...the blue LED on the Espruino wifi starts, and continues to, flash/flicker after I get the 'connected' message. So I might actually be getting the 0xEF message, but I'm never getting the tcpClient.on('data', ...) call.

And how do I send a message with tcpClient? tcpClient.write() and tcpClient.send() throw 'undefined' errors.

@gfwilliams, thoughts?

---

Posted at 2019-08-08 by Robin

Thr 2019.08.08

I'm guessing here, so if this sounds far fetched, it's okay to blast me ;-)

Could it be that the tcpClient instance creation L7 needs to be global, inserted between L1 and L2, rather than defined inside the udpClient bind object? e.g. msgs sent, but no object present to receive them?


Did you catch this post?

Out of office, 9th to 26th Aug

Not sure which time zone you are in, but it's late in the day by Gordon, and I'm sure packing/getting ready is on the agenda. I'm in CST 6hrs behind GMT

---

Posted at 2019-08-08 by maze1980

https://www.espruino.com/Reference#l_net_connect
You must place the .on functions inside of the connect callback using the returned socket, e.g.:

tcpClient.connect({port: 51325, host: "10.1.64.22"}, function(socket) {
    console.log("connected");
   socket.on("close", function() { 
      console.log("end");
   });
});

How and if your UDP code really works is also questionable, it should be similar:
https://www.espruino.com/Reference#l_dgram_createSocket
While the example on https://www.espruino.com/Internet is exactly as you did it just doesn't correspond to the documentation.

And I'd change let tcpClient = require('net'); to const NET = require('net');. It's just a shortcut to the net library, not a tcpClient.

---

Posted at 2019-08-08 by indianajones

@robin, I tried your suggestion...seems like a good idea. No difference in results though.

I missed the post about time off, I guess I'll work on other things for a couple weeks, I have plenty of other things to do on this project.

Thanks for your ideas.

---

Posted at 2019-08-08 by indianajones

holy. freaking. crap. I didn't know there was an argument to the connect callback - the sample code I found didn't have an arg. And yes, putting them inside the callback and using 'socket' made it work. I am now seeing the 0xEF packets being sent. Solved all of the confusion.

As for the UDP code, it does work, but I don't claim to have written it correctly, I'll change it to match the link you provided.

So thanks a million for the corrections!

---

Posted at 2019-08-09 by Robin

Thr 2019.08.08

@IndianaJones I think there is a word missing from that first sentence. Shouldn't it have read

" holy. freaking. crap. . . Batman! " ?

at least that's how I thought I've heard that!!

Nice observation maze1980 on the inclusion of the on() method inside the callback.

As I had no experience with sockcts and Espruino, I could only guess, but I'm glad you now have a solution.

I don't want to get into a discussion that hijacks this thread, but feel the following statement will cause confusion that are new to Espruino:

While we are all learning the nuaunces of Espruino Javascript, @maze1980 what justification are you using to suggest changing the var declaration to "const NET = require('net');. It's just a shortcut to the net library"?

A year ago, I was educated on the syntax to create a local constant copy of a module.

How to define and reference an array of CONSTANTS in deployed module

A special syntax for the exports statement inside the module is required. In the past, I have looked up each module to see if there are any comment tid-bits:

http://www.espruino.com/modules/

The 'net' module, sadly is not deployed in this manner, as it appears to be part of the build, but did find this one that is quite similar.

http://www.espruino.com/modules/WebServer.js
Note that syntax is not used as the last exports line here either.

• SIDEBAR: Gordon showed me this little trick to find comments in the source. Using the hardware reference documentation

  https://www.espruino.com/Reference#l_net­_connect

  there is a right facing arrow at the right of the heading 'net.connect' and clicking on it takes one to the source:
  https://github.com/espruino/Espruino/blob/master/libs/network/jswrap_net.c#L367

Additionally, looking over the examples show that each had a connection object assigned to a var declaration.

https://www.espruino.com/Internet

Assigning as a constant, although may work under some module designs, prevents making a modification to the underlying copy of that module. What if one would want to change the port designation or the baud rate or something of the sort?

While I'm not an expert on all the nuances with Espruino, as there is a connection() and on() function along with a few other supporting module functions, I don't believe assigning this particular module to a constant assignment is a smart idea nor was it the authors intended design.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/const
Let for block scope - maybe depends
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/let
It should be var
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/var

---

Posted at 2019-08-09 by maze1980

@robin: You don't want to modify the library or module itself.

const MODULE = require('module');
MODULE.foo();
//is the same as
require('module').foo();

You can use a variable if you want to code like this

MODULE = require('module1');
MODULE.foo();
MODULE = require('module2');
MODULE.foo();

Happy debugging.

---

Posted at 2019-08-10 by indianajones

Final update on this issue. I have it all working now - I can control my Allen & Heath Qu series mixer via MIDI over TCP using the Espruino Wifi. My high-level understanding was accurate, but you guys helped me figure out how to use the dgram and net libraries properly to make it work. Thanks @gfwilliams for steering me in the right direction, and @robin and especially @maze1980 for helping me see the errors in my code. Many thanks.

---

Posted at 2019-08-27 by @gfwilliams

tcpClient.write() and tcpClient.send() throw 'undefined'

Is it because of let tcpClient = require('net');? You'll have defined tcpClient as a local variable so it won't be available in the global scope

---

Posted at 2019-08-27 by indianajones

@gfwilliams yes I was completely forgetting some of the rules of scope in javascript.