Using socket.io with Bun

[ehuff700]

Is anyone else successfully transmitting BINARY messages with socket.io using the Bun runtime? I've spent hours debugging and I seriously think it's an issue with Bun at this point given the code was working with npm.

[ehuff700]

engine:ws received "42/agent,["test",{"test":true}]" +5ms
engine:socket received packet message +5ms
socket.io-parser decoded 2/agent,["test",{"test":true}] as {"type":2,"nsp":"/agent","data":["test",{"test":true}]} +5ms
socket.io:socket got packet {"type":2,"nsp":"/agent","data":["test",{"test":true}]} +6ms
socket.io:socket emitting event ["test",{"test":true}] +0ms
socket.io:socket dispatching an event ["test",{"test":true}] +0ms

 engine:ws received "451-/agent,["log_event",{"_placeholder":true,"num":0}]" +0ms
engine:socket received packet message +0ms
socket.io-parser decoded 51-/agent,["log_event",{"_placeholder":true,"num":0}] as {"type":5,"attachments":1,"nsp":"/agent","data":["log_event",{"_placeholder":true,"num":0}]} +0ms
engine:ws received Ok(())2023-09-15T04:27:58.977Z"agent" +0ms
engine:socket received packet error +0ms
engine:socket readyState updated from open to closed +0ms
engine:transport readyState updated from open to closing (websocket) +10ms
engine:ws closing +1ms
engine:transport readyState updated from closing to closed (websocket) +0ms
socket.io:client client close with reason parse error +8ms
socket.io:socket closing socket - reason parse error +1ms

Above is attached debug logs, "test" event is plaintext JSON and "log_event" was serialized as binary (but the engine received trimmed plaintext??). Very weird

[ehuff700]

Another example, this was sent via postman with the binary hex data "48656C6C6F20576F726C64" (Hello World).

socket.io-parser decoded 51-/agent,["test",{"_placeholder":true,"num":0}] as {"type":5,"attachments":1,"nsp":"/agent","data":["test",{"_placeholder":true,"num":0}]} +5s
 engine:ws received "Hello World" +1ms
 engine:socket received packet error +1ms
 engine:socket readyState updated from open to closed +0ms
 engine:transport readyState updated from open to closing (websocket) +5s
 engine:ws closing +0ms
 engine:transport readyState updated from closing to closed (websocket) +0ms
 socket.io:client client close with reason parse error +5s
 socket.io:socket closing socket - reason parse error +5s

[darrachequesne]

I think I found the culprit!

We expect an isBinary argument in the WebSocket "message" handler:

this.socket.on("message", (data, isBinary) => {
  const message = isBinary ? data : data.toString();
  debug('received "%s"', message);
  super.onData(message);
});

Reference: https://github.com/socketio/engine.io/blob/c6bf8c0f571aad7a5917f43860c8c3d74a9b429b/lib/transports/websocket.ts#L19-L23

If we take the following code:

import WebSocket, { WebSocketServer } from 'ws';

const wss = new WebSocketServer({
  port: 3000
});

wss.on('connection', ws => {
  ws.on('message', (data, isBinary) => {
    console.log('received', data, isBinary);
  });
});

const ws = new WebSocket('ws://www.localhost:3000');

ws.on('open', function open() {
  ws.send('hello');
  ws.send(Buffer.from([1, 2, 3]));
});

Output with Node.js:

$ node test.js
received <Buffer 68 65 6c 6c 6f> false
received <Buffer 01 02 03> true

Output with bun:

$ bun -v
1.0.2
$ bun test.js
received Buffer(5) [ 104, 101, 108, 108, 111 ] undefined
received Buffer(3) [ 1, 2, 3 ] undefined

[ehuff700]

Hi @darrachequesne! It's a pleasure to get to talk with you, thanks for all your work on this project. I forgot to update this thread, but yes, I came to the same conclusion.

It appears that when using bun, the "ws" module that as used as the default web socket engine is getting replaced with Bun's "drop-in" version during import resolution time. That drop-in version does not support that second callback for isBinary...

I tried using a different engine, including eiows, but wasn't able to successfully get it working (had some weird compilation issues for uWebSockets I couldn't figure out). There are many issues with Bun's web socket implementation and their team is working through them. Sadly although I was very happy with Bun's perf, I think it's at a stage where it's not useable in any serious project yet (I ran into a multitude of other issues with big libraries as well, see #2036 for Bun. Again thanks for looking into this! Hopefully anyone that had this same problem will be able to find this.

[darrachequesne]

I've created an issue in the Bun's repository: oven-sh/bun#5741

Updating the current API would be a breaking change though, not sure how they could handle that.

As a workaround, I think we should be able to create a specific engine, which would use Bun native WebSocket module.