Multi threading with hyper-express

[flaxeneel2]

Hi there, I wanted to run multiple child processes which each handle requests by their own to parallelize the workload, however no matter what i do (used workerpools, used child processes), i can't seem to do it. Has anyone else done it before and would be willing to help me?

[kartikk221]

I'm not sure what exact problem you are having but you can simply use the cluster module in Node.js to multi-thread HyperExpress instances to have them handle their own payloads similar to all other webservers. Below is an example code snippet to do this with some usage comments:

// NOTE: Any global code/variables will be executed for both the master and forked worker processes
const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
const HyperExpress = require('hyper-express');

// Check if the current process is the master process
if (cluster.isPrimary) {
   // NOTE: Code inside of this block will only run ONCE in the master/primary process

   // Spawn the initial set of worker instances which will be decided based on the number of cpu cores available
   for (let i = 0; i < numCPUs; i++) cluster.fork();
} else {
   // NOTE: Code inside of this block will run for each time a worker is forked with the fork() method from the master process.

   // Initialize the HyperExpress server instance
   const server = new HyperExpress.Server();

   // Bind a test route which will return the process id in which it is serving the request from
   server.get('/id', (request, response) => {
      response.send('Served this request from ' + process.pid);
   });

   // Listen the server on the desired port
   server.listen(3000);
}

If you make a bunch of requests to the /id route above, you will notice that you will receive different process ids up to the number of cpu cores you have available on your test machine.

[flaxeneel2]

Hi, thanks for replying. I tried it the way you said, but even though its spawning multiple threads, it doesn't seem to be actually spreading the load across them.

In this screenshot here, when the server is hit with a load (Using wrk), only one of the threads seems to load up and the others continue to do nothing.

This is my code.
Also, no matter how many times i refresh the /id page, it seems to return the same ID

[kartikk221]

This is most likely because you are simply not stressing the worker hard enough to have the Windows scheduling switch and use other workers. Try the following:

• Either specify the scheduling policy as round robin with cluster.schedulingPolicy = cluster.SCHED_RR at the beginning of your code file.
• Performing a CPU heavy synchronous operation such as hashing on each request.

Performing either of the above should give you more apparent results as the traffic should be distributed by the Windows OS.
You can read more about cluster's scheduling here: https://nodejs.org/api/cluster.html#clusterschedulingpolicy

[flaxeneel2]

After failing on windows for whatever reason (round robin didn't make a difference), I switched over to a linux machine, and testing with the exact same setup on that worked flawlessly. I guess there was some other bottleneck or node issue on my laptop.

Running 1m test @ http://localhost:3000/id
  16 threads and 2000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.92ms  472.71us  46.72ms   93.02%
    Req/Sec    68.01k    26.00k  127.40k    61.70%
  64961020 requests in 1.00m, 5.32GB read
Requests/sec: 1082474.28
Transfer/sec:     90.84MB

(This test was done on a 5950x with 16 threads dedicated to the server and 16 to the benchmark)
It scaled very well on the machine, which means the cluster works properly on it. Thanks for your help!

[ImLunaHey]

I have a feeling you're hitting this issue with Windows https://stackoverflow.com/a/17213054

[electrovir]

I'm having the same issue (but on macOS). If I start a plain express server in my cluster script, the requests get distributed perfectly. If I start a hyper-express server, all requests always go to the same thread.

[electrovir]

@kartikk221 your example code does not work: the process pid is always the same, even when I introduce a huge synchronous task. However, even with your sample code, if I switch to express then it works fine.

[electrovir]

https://www.npmjs.com/package/@ionited/mesh doesn't work either.
https://www.npmjs.com/package/elysia does work.

I'm guessing this is a limitation within µWebSockets.js itself.