Performance of async functions of HTTP server

[andriichyzh]

Hi,

I tested your great library and got amazing results with WebSockets. But in case of HTTP + Redis (some cache service) I got unexpected results:

OS: Mac OS 10.14.4 and Ubuntu 18.04
Node.js: v10.15.2
Lib: uWebSockets.js#v15.10.0

Native HTTP + Redis

const http = require('http');
const { promisify } = require('util');
const redis = require('redis');

const client = redis.createClient();

client.on('error', (err) => {
  console.log('Redis error', err);
});

const getAsync = promisify(client.get).bind(client);

const server = http.createServer(async (req, res) => {

  const data = await getAsync('a');

  res.statusCode = 200;
  res.setHeader('Content-Type', 'application/json');
  res.end(data);
});

server.listen(3000, () => {
  console.log('Listening to port 3000');
});

Result:

$ wrk -c 50 -d 30s -t 50 http://127.0.0.1:3000/
Running 30s test @ http://127.0.0.1:3000/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.06ms  556.61us  22.01ms   93.74%
    Req/Sec   489.99     64.84     1.17k    91.70%
  732858 requests in 30.11s, 102.74MB read
Requests/sec:  24343.35
Transfer/sec:      3.41MB

uWebSockets.js HTTP + Redis

const { promisify } = require('util');
const uws = require('uWebSockets.js');
const redis = require('redis');

const app = uws.App();

const client = redis.createClient();

client.on('error', (err) => {
  console.log('Redis error', err);
});

const getAsync = promisify(client.get).bind(client);

app.get('/', async (res, req) => {
  res.onAborted(() => {
    res.aborted = true;
  });

  const data = await getAsync('a');

  if (!res.aborted) {
    res.writeHeader('Content-Type', 'application/json');
    res.end(data);
  }
});

app.listen(3000, () => {
  console.log('Listening to port 3000');
});

Result:

$ wrk -c 50 -d 30s -t 50 http://127.0.0.1:3000/
Running 30s test @ http://127.0.0.1:3000/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.65ms  358.20us  12.52ms   89.49%
    Req/Sec   275.07     15.13   616.00     91.73%
  411773 requests in 30.07s, 41.63MB read
Requests/sec:  13693.56
Transfer/sec:      1.38MB

Total:

Native HTTP     24343.35 RPS
uWebSockets.js  13693.56 RPS

Is it a bug or can you suggest some fix for it?

[ghost]

Hi,

This is a good test. It hits an issue not yet fixed https://github.com/uNetworking/uWebSockets/issues/816

I get 16k with Node.js, 11k with uws.js and 34k with the fix.

Basically in v16 there will be some kind of res.begin() function you have to call before writing anything (as an opening matching pair to res.end). It's not yet added, so performance is lacking in this case.

[andriichyzh]

Thank you for your answer!
And BIG thanks for your work on this library! ;)

I very want to use it as HTTP / HTTPS server and I guess your fix will help :)

Do you have some ETA of v16? ;)

[ghost]

Could you try npm install uNetworking/uWebSockets.js#binaries and replace:

    res.writeHeader('Content-Type', 'application/json');
    res.end(data);

with

    res.experimental_cork(() => {
      res.writeHeader('Content-Type', 'application/json');
      res.end(data);
    });

[ghost]

I get with and without experimental_cork:

alexhultman@alexhultman-N56VZ:~/wrk$ ./wrk http://localhost:3000
Running 10s test @ http://localhost:3000
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.86ms  159.91us   7.81ms   90.65%
    Req/Sec     5.84k   528.45    10.28k    86.07%
  116731 requests in 10.10s, 10.46MB read
Requests/sec:  11558.04
Transfer/sec:      1.04MB

alexhultman@alexhultman-N56VZ:~/wrk$ ./wrk http://localhost:3000
Running 10s test @ http://localhost:3000
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   297.16us   93.35us   4.49ms   92.49%
    Req/Sec    16.94k     1.68k   31.49k    95.52%
  338747 requests in 10.10s, 30.37MB read
Requests/sec:  33542.07
Transfer/sec:      3.01MB

[andriichyzh]

This method fixed performance! Thank you for so quick experimental changes ;)

Native HTTP + Redis

$ wrk -c 50 -d 30s -t 50 http://127.0.0.1:3000/
Running 30s test @ http://127.0.0.1:3000/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.98ms  284.05us   6.96ms   88.10%
    Req/Sec   505.41     54.41     2.32k    94.40%
  755337 requests in 30.10s, 105.89MB read
Requests/sec:  25090.91
Transfer/sec:      3.52MB

uWebSockets.js HTTP + Redis

$ wrk -c 50 -d 30s -t 50 http://127.0.0.1:3000/
Running 30s test @ http://127.0.0.1:3000/
  50 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.07ms  143.56us   6.64ms   89.31%
    Req/Sec     0.93k   110.09     6.16k    99.12%
  1391357 requests in 30.10s, 140.65MB read
Requests/sec:  46222.96
Transfer/sec:      4.67MB

Total:

Native HTTP     25090.91 RPS
uWebSockets.js  46222.96 RPS

Thank you one more time! You are a performance monster! :)
We’ll really wait for these great changes in the next releases ;)

[ghost]

Even if you're getting worse performance right now, you probably want to first make it work and then just update what you have with the corking fix when it's done?

[andriichyzh]

Yes, it will be easy to change after the release of this fix :)
Thank you for amazing conversation and for quick cool answers! ;)

[ghost]

Now you just need a faster Redis client 😉

[RJ4100]

@alexhultman which one would you recommend? ioredis?

[abenhamdine]

Hello,

Anyone knows if it's still relevant to use experimental_cork with version 0.16.0 ?

[ghost]

Yes you still need it, maybe it's time to rename it to "cork" soon?

[aadityataparia]

Does this work with res.write and res.tryEnd if yes then how to use it?

[ghost]

Yes you call those write functions while inside experimental_cork

[yovanoc]

Yep I think its time to rename it to cork and add it to the typings too, good job

[ghost]

There is now docs for res.cork:

https://unetworking.github.io/uWebSockets.js/generated/interfaces/httpresponse.html#cork