socket.io failed connections when testing with Artillery

[DouglasLapsley]

I'm setting up a node.js socket.io server and need to support around 10k users. This guide says it should be possible to get up to 55k concurrent connections.

I'm hosting on an Ubuntu 20.04 EC2 with 4 vcpus and 16b RAM. Socket.io JavaScript (reduced for clarity) for node.js is:

import * as express from "express";
import { createServer } from "http";
import * as fs from 'fs';
import * as https from 'https';
import * as path from 'path';
import { Server } from "socket.io";
import * as os from 'os';
import * as sticky from 'sticky-session'; // Required to make use of multiple processors

let httpServer;
const app = express();

port = 443;

// Certificate
const privateKey = fs.readFileSync('MYPATH/privkey.pem', 'utf8');
const certificate = fs.readFileSync('MYPATH/cert.pem', 'utf8');
const ca = fs.readFileSync('MYPATH/chain.pem', 'utf8');

const credentials = {
  key: privateKey,
  cert: certificate,
  ca: ca
};

httpServer = https.createServer(credentials, app);

const io = new Server(httpServer, {
  cors: {
    origin: [
      MYORIGINS
    ]
  }
});


io.on('connection', (socket) => {
  // MY ENDPOINTS
});


if (!sticky.listen(httpServer, port)) {
  // Master code
  httpServer.once('listening', function() {
    console.log('server started on port port');
  });
} else {
  // Worker code
}

I've followed the steps in the guide above doing:

sudo nano /etc/security/limits.d/custom.conf and adding the following lines:

* soft nofile 1048576
* hard nofile 1048576

And sudo nano /etc/sysctl.d/net.ipv4.ip_local_port_range.conf adding:

net.ipv4.ip_local_port_range = 10000 65535

I've also read through and tried the settings from this article.

I am testing with Artillery with the following YAML config:

config:
  target: "MYURL"
  phases:
    - duration: 20
      arrivalRate: 250
  engines:
   socketio-v3: {
     transports: ["websocket"]
   }

scenarios:
  - name: My load testing
    engine: socketio-v3
    flow:
      - think: 20

I am getting really varied results. With exactly those settings I am getting 0 vusers.failed which is good, but that only gives me 5000 users.

If I increase either the duration or the arrival rate to give a higher overall number of users I start to get significant numbers of vusers.failed. e.g. Creating 20k users I get 9k failed. Creating 50k users I get 49k failed!

I'm not quite sure exactly what vusers.failed means (I couldn't find documentation).

I'm using sticky-session on advice from this article as I want all 4 CPU cores to be used. That seems to run 4x instances of the httpServer. I am not 100% sure if this is needed as I am never seeing CPU use go above 20% or so however hard I hit it with users.

What does vusers.failed mean exactly?

Why am I getting vusers.failed errors like this?

What do I need to change?

Do I need the sticky sessions and is that what's required to use all 4 processor cores on the server?

Thanks very much!

[darrachequesne]

Hi!

• which version of artillery are you using? It seems there was a lot of updates recently. Does it run in multi-threaded mode?

• did you try with manual client creation?

• which arrivalRate are you using? Sometimes a high arrivalRate can lead to issues from artillery.

I was able to happily go to 25k clients without any problem on my machine with one single Socket.IO server ([email protected]):

config:
  target: "http://localhost:3000"
  phases:
    - duration: 480
      arrivalRate: 200

Result:

All virtual users finished
Summary report
  Scenarios launched:  96000
  Scenarios completed: 96000
  Requests completed:  0
  Mean response/sec: NaN
  Response time (msec):
    min: NaN
    max: NaN
    median: NaN
    p95: NaN
    p99: NaN
  Scenario counts:
    My load testing: 96000 (100%)

Please note that sticky sessions are not needed if you are only using WebSockets. In that case, using sticky-session will make it worse, because it load balances the incoming connections based on the IP address, so every connection will end up on a single node.

If you also need HTTP long-polling, please check the @socket.io/sticky module: https://socket.io/docs/v4/using-multiple-nodes/#using-nodejs-cluster

[darrachequesne]

Will the server use more than one virtual CPU if sticky sessions are not enabled?

No, Node.js uses 1 CPU by default, you will need to use the Node.js built-in cluster module:

import cluster from 'cluster';
import http from 'http';
import { cpus } from 'os';
import process from 'process';
import { Server } from 'socket.io';

const numCPUs = cpus().length;

if (cluster.isPrimary) {
  console.log(`Primary ${process.pid} is running`);

  // Fork workers.
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} died`);
  });
} else {
  const io = new Server(3000);

  console.log(`Worker ${process.pid} started`);
}

Reference: https://nodejs.org/api/cluster.html

[DouglasLapsley]

Thank you very much. Most helpful.

[DouglasLapsley]

I'm able to get a cluster working with the cluster, @socket.io/sticky and @socket.io/cluster-adapter modules.

However, I'm using https on the socket.io URL and I get an error because apparently @socket.io/sticky is "not compatible with an HTTPS server".

I'm assuming that I need to use https to ensure a fully secure connection.

I'm assuming that I need @socket.io/sticky and @socket.io/cluster-adapter or similar to sync the workers.

Is there a way to run multiple cores AND https at the same time please?

Thanks so much.

[darrachequesne]

If you only need WebSocket connections, then @socket.io/sticky is not needed and you can use @socket.io/cluster-adapter and cluster directly (you don't need sticky sessions).

If you also need HTTP long-polling, then in that case you can use @socket.io/sticky, though it does not currently support HTTPS. As a workaround, you can have each worker listen to a different port, and have proxy in front of them that distributes the load and handles sticky sessions like HAProxy: https://socket.io/docs/v4/using-multiple-nodes/#haproxy-configuration

[DouglasLapsley]

Amazing thank you. You've been so very helpful and it's greatly appreciated.