Skip to content
Merged
60 changes: 60 additions & 0 deletions frameworks/JavaScript/ultimate-express/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# UltimateExpress Benchmarking Test

The Ultimate Express. Fastest http server with full Express compatibility, based on [µWebSockets](https://github.com/uNetworking/uWebSockets.js).

## Important Libraries

The tests were run with:

- [ultimate-express](https://github.com/dimdenGD/ultimate-express)
- [postgres](https://github.com/porsager/postgres)
- [mariadb](https://github.com/mariadb-corporation/mariadb-connector-nodejs)
- [lru-cache](https://github.com/isaacs/node-lru-cache)

## Database

There are individual handlers for each DB approach. The logic for each of them are found here:

- [Postgres](database/postgres.js)
- [MySQL](database/mysql.js)

There are **no database endpoints** or drivers attached by default.

To initialize the application with one of these, run any _one_ of the following commands:

```sh
$ DATABASE=postgres npm start
$ DATABASE=mysql npm start
```

## Test Endpoints

> Visit the test requirements [here](https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview)

```sh
$ curl localhost:8080/json
$ curl localhost:8080/plaintext

# The following are only available with the DATABASE env var

$ curl localhost:8080/db
$ curl localhost:8080/fortunes

$ curl localhost:8080/queries?queries=2
$ curl localhost:8080/queries?queries=0
$ curl localhost:8080/queries?queries=foo
$ curl localhost:8080/queries?queries=501
$ curl localhost:8080/queries?queries=

$ curl localhost:8080/updates?queries=2
$ curl localhost:8080/updates?queries=0
$ curl localhost:8080/updates?queries=foo
$ curl localhost:8080/updates?queries=501
$ curl localhost:8080/updates?queries=

$ curl localhost:8080/cached-worlds?count=2
$ curl localhost:8080/cached-worlds?count=0
$ curl localhost:8080/cached-worlds?count=foo
$ curl localhost:8080/cached-worlds?count=501
$ curl localhost:8080/cached-worlds?count=
```
149 changes: 149 additions & 0 deletions frameworks/JavaScript/ultimate-express/app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
import express from 'ultimate-express';
import { LRUCache } from 'lru-cache';
import cluster, { isWorker } from 'node:cluster';
import { maxQuery, maxRows } from './config.js';
import { sjs, attr } from 'slow-json-stringify'

const { DATABASE } = process.env;
const db = DATABASE ? await import(`./database/${DATABASE}.js`) : null;

const jsonSerializer = sjs({ message: attr("string") });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fast-json-steingify is faster than slow-json-stringify with "unsafe" string type https://www.npmjs.com/package/fast-json-stringify

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find a benchmark for that

Copy link
Contributor

@nigrosimone nigrosimone Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made it here https://stackblitz.com/edit/stackblitz-starters-rlrggo

slow-json-stringify: 13.105ms
fast-json-stringify: string: 0.86ms
fast-json-stringify: unfafe: 0.815ms
JSON.stringify: 0.86ms

for small json JSON.stringify is faster, slow-json-stringify is always slow. Side note "fast-json-stringify: unfafe" is raw fast-json-stringify with no special chars support on string (\n, \r, \t, ", \), Hello, World! no need escape.

Use JSON.stringify, this libraries (fast-json-stringify and slow-json-stringify) works better on complex/huge json

slow-json-stringify last update is 4 years ago (maybe is abbandoned)
fast-json-stringify is active and mantained fy Fastify team

The better option is send as raw/stringified string

app.get('/json', (req, res) => {
  res.setHeader('Server', 'UltimateExpress');
  res.setHeader('Content-Type', 'application/json');
  res.end('{"message":"Hello, World!"}');
});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You must serialize JSON in the request as per test rules. Your benchmark has a bug and doesnt serialize object properly, I fixed and tested it and it seems that fast-json-stringify is still fastest, so I switched it over.


const generateRandomNumber = () => Math.floor(Math.random() * maxRows) + 1;

const parseQueries = (i) => Math.min(parseInt(i) || 1, maxQuery);

const escapeHTMLRules = { '&': '&#38;', '<': '&#60;', '>': '&#62;', '"': '&#34;', "'": '&#39;', '/': '&#47;' };

const unsafeHTMLMatcher = /[&<>"'\/]/g;

const escapeHTMLCode = (text) => unsafeHTMLMatcher.test(text) ? text.replace(unsafeHTMLMatcher, function (m) { return escapeHTMLRules[m] || m; }) : text;

const cache = new LRUCache({
max: maxRows
});

const app = express();
app.set("etag", false);
app.set("x-powered-by", false);

app.get('/plaintext', (req, res) => {
res.setHeader('Server', 'UltimateExpress');
res.setHeader('Content-Type', 'text/plain');
res.end('Hello, World!');
});

app.get('/json', (req, res) => {
res.setHeader('Server', 'UltimateExpress');
res.setHeader('Content-Type', 'application/json');
res.end(jsonSerializer({ message: "Hello, World!" }));
});

if (db) {
app.get('/db', async (req, res) => {
res.setHeader('Server', 'UltimateExpress');

try {
const world = await db.find(generateRandomNumber());
res.json(world);
} catch (error) {
throw error;
}
});

app.get('/queries', async (req, res) => {
res.setHeader('Server', 'UltimateExpress');

try {
const queries = parseQueries(req.query.queries);
const worldPromises = new Array(queries);

for (let i = 0; i < queries; i++) {
worldPromises[i] = db.find(generateRandomNumber());
}

const worlds = await Promise.all(worldPromises);

res.json(worlds);
} catch (error) {
throw error;
}
})

app.get('/updates', async (req, res) => {
res.setHeader('Server', 'UltimateExpress');

try {
const queries = parseQueries(req.query.queries);
const worldPromises = new Array(queries);

for (let i = 0; i < queries; i++) {
worldPromises[i] = db.find(generateRandomNumber());
}

const worlds = await Promise.all(worldPromises);

for (let i = 0; i < queries; i++) {
worlds[i].randomNumber = generateRandomNumber();
}

await db.bulkUpdate(worlds);

res.json(worlds);
} catch (error) {
throw error;
}
})

app.get('/fortunes', async (req, res) => {
res.setHeader('Server', 'UltimateExpress');

try {
const fortunes = await db.fortunes()

fortunes.push({ id: 0, message: 'Additional fortune added at request time.' });

fortunes.sort((a, b) => (a.message < b.message) ? -1 : 1);

const n = fortunes.length

let i = 0, html = ''
for (; i < n; i++) html += `<tr><td>${fortunes[i].id}</td><td>${escapeHTMLCode(fortunes[i].message)}</td></tr>`

res
.header('Content-Type', 'text/html; charset=utf-8')
.end(`<!DOCTYPE html><html><head><title>Fortunes</title></head><body><table><tr><th>id</th><th>message</th></tr>${html}</table></body></html>`);
} catch (error) {
throw error;
}
})

let isCachePopulated = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be optimized by populating on test init and not on first call, eg:

  let isCachePopulated = false;
  const populate = async() => {
      if (!isCachePopulated) {
        const worlds = await db.getAllWorlds();
        for (let i = 0; i < worlds.length; i++) {
          cache.set(worlds[i].id, worlds[i]);
        }
        isCachePopulated = true;
      }
      return isCachePopulated;
  });
  populate().then(console.log).catch(console.error);

  app.get('/cached-worlds', async (req, res) => {
    res.setHeader('Server', 'UltimateExpress');

    try {
      await populate();
      const count = parseQueries(req.query.count);
      const worlds = new Array(count);

      for (let i = 0; i < count; i++) {
        worlds[i] = cache.get(generateRandomNumber());
      }

      res.json(worlds);
    } catch (error) {
      throw error;
    }
  });
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark has a warm-up period, so it doesn't matter.

app.get('/cached-worlds', async (req, res) => {
res.setHeader('Server', 'UltimateExpress');

try {
if (!isCachePopulated) {
const worlds = await db.getAllWorlds();
for (let i = 0; i < worlds.length; i++) {
cache.set(worlds[i].id, worlds[i]);
}
isCachePopulated = true;
}
const count = parseQueries(req.query.count);
const worlds = new Array(count);

for (let i = 0; i < count; i++) {
worlds[i] = cache.get(generateRandomNumber());
}

res.json(worlds);
} catch (error) {
throw error;
}
});
}

app.listen(8080, () => {
console.log(`${isWorker ? `${cluster.worker.id}: ` : ''}Successfully bound to http://0.0.0.0:8080`);
});
70 changes: 70 additions & 0 deletions frameworks/JavaScript/ultimate-express/benchmark_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
{
"framework": "ultimate-express",
"tests": [
{
"default": {
"json_url": "/json",
"plaintext_url": "/plaintext",
"port": 8080,
"approach": "Realistic",
"classification": "Micro",
"database": "None",
"framework": "ultimate-express",
"language": "JavaScript",
"flavor": "None",
"orm": "Raw",
"platform": "NodeJS",
"webserver": "µws",
"os": "Linux",
"database_os": "Linux",
"display_name": "ultimate-express",
"notes": "",
"versus": "nodejs"
},
"postgres": {
"db_url": "/db",
"fortune_url": "/fortunes",
"query_url": "/queries?queries=",
"update_url": "/updates?queries=",
"cached_query_url": "/cached-worlds?count=",
"port": 8080,
"approach": "Realistic",
"classification": "Micro",
"database": "Postgres",
"framework": "ultimate-express",
"language": "JavaScript",
"flavor": "None",
"orm": "Raw",
"platform": "NodeJS",
"webserver": "µws",
"os": "Linux",
"database_os": "Linux",
"display_name": "ultimate-express-postgres",
"notes": "",
"versus": "nodejs"
},
"mysql": {
"db_url": "/db",
"fortune_url": "/fortunes",
"query_url": "/queries?queries=",
"update_url": "/updates?queries=",
"cached_query_url": "/cached-worlds?count=",
"port": 8080,
"approach": "Realistic",
"classification": "Micro",
"database": "MySQL",
"framework": "ultimate-express",
"language": "JavaScript",
"flavor": "None",
"orm": "Raw",
"platform": "NodeJS",
"webserver": "µws",
"os": "Linux",
"database_os": "Linux",
"display_name": "ultimate-express-mysql",
"notes": "",
"versus": "nodejs"
}
}
]
}
15 changes: 15 additions & 0 deletions frameworks/JavaScript/ultimate-express/clustered.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import cluster, { isPrimary, setupPrimary, fork } from 'node:cluster'
import { cpus } from 'node:os'

if (isPrimary) {
setupPrimary({
exec: 'app.js',
})
cluster.on('exit', (worker) => {
console.log(`worker ${worker.process.pid} died`)
process.exit(1)
})
for (let i = 0; i < cpus().length; i++) {
fork()
}
}
8 changes: 8 additions & 0 deletions frameworks/JavaScript/ultimate-express/config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export const maxQuery = 500
export const maxRows = 10000
export const clientOpts = {
host: 'tfb-database',
user: 'benchmarkdbuser',
password: 'benchmarkdbpass',
database: 'hello_world',
}
15 changes: 15 additions & 0 deletions frameworks/JavaScript/ultimate-express/database/mysql.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { createPool } from 'mariadb'
import { cpus } from 'node:os'
import { clientOpts } from '../config.js'

const pool = createPool({ ...clientOpts, connectionLimit: cpus().length })

const execute = (text, values) => pool.execute(text, values || undefined)

export const fortunes = () => execute('SELECT id, message FROM fortune')

export const find = (id) => execute('SELECT id, randomNumber FROM world WHERE id = ?', [id]).then(arr => arr[0])

export const getAllWorlds = () => execute('SELECT id, randomNumber FROM world')

export const bulkUpdate = (worlds) => pool.batch('UPDATE world SET randomNumber = ? WHERE id = ?', worlds.map(world => [world.randomNumber, world.id]).sort((a, b) => (a[1] < b[1]) ? -1 : 1))
14 changes: 14 additions & 0 deletions frameworks/JavaScript/ultimate-express/database/postgres.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import postgres from 'postgres'
import { clientOpts } from '../config.js'

const sql = postgres({ ...clientOpts, max: 1 })

export const fortunes = async () => sql`SELECT id, message FROM fortune`

export const find = async (id) => sql`SELECT id, randomNumber FROM world WHERE id = ${id}`.then((arr) => arr[0])

export const getAllWorlds = async () => sql`SELECT id, randomNumber FROM world`

export const bulkUpdate = async (worlds) => await sql`UPDATE world SET randomNumber = (update_data.randomNumber)::int
FROM (VALUES ${sql(worlds.map(world => [world.id, world.randomNumber]).sort((a, b) => (a[0] < b[0]) ? -1 : 1))}) AS update_data (id, randomNumber)
WHERE world.id = (update_data.id)::int`;
Loading