[regression] dart 3.x server performance regression against dart 2.x and 1.x

[iapicca]

problem

benchmark show dart3 regression in json serialization and plain text performance against dart2 and dart

proposed solutions

1. investigate and solve performance regression
2. lose the "speed battle" and win the "resource battle" with wasi support*

*slower but more portable and infinitely more lightweight than containers ( even better on k8s)

[iapicca]

source

cc @kevmoo

[kevmoo]

I've talked to @mraleph about this, too. He may have thoughts.

CC @brianquinlan

Thanks for flagging @iapicca !

[kevmoo]

FYI @iapicca – Dart 1 and 2 DON'T use AOT compilation.

AOT is great for small binaries and quick startup, but JIT mode will almost certainly be faster for long-running (server) apps. The JIT learns. Might be worth either switching (back) to JIT or adding in JIT as another configuration.

[iapicca]

FYI @iapicca – Dart 1 and 2 DON'T use AOT compilation.

AOT is great for small binaries and quick startup, but JIT mode will almost certainly be faster for long-running (server) apps. The JIT learns. Might be worth either switching (back) to JIT or adding in JIT as another configuration.

@kevmoo

1. I admit my ignorance, in my head sounded like "AOT good, JIT bad",
   maybe the JIT theory would explain the jump in performance of dart1...
   they just threw more RAM and CPU at the server and AOT couldn't fully take advantage of it

2. Maybe I was a bit hasty calling it a regression, it would be interesting to compare the memory footprint and resources used,
   indeed faster isn't always better

[kevmoo]

AOT is GREAT if you care about (1) small binaries (2) fast, predictable initial load

But JIT will often win on overall performance for long-running processes.

[mraleph]

But JIT will often win on overall performance for long-running processes.

This is not true. JIT vs AOT is a complicated topic. In reality we have been extremely focused on AOT performance and JIT performance has been mostly neglected. AOT does a bunch of optimizations not available in JIT. So AOT often is better than JIT for performance, with exception of cases where AOT fails to specialize the code either due to conservative inlining (JIT is very aggressive with inlining) or imprecise type information (JIT can have better type feedback).

What else comes to mind:

• There were some known regressions in Dart 1 -> Dart 2 transition
• There are known regressions in the networking stack - improve socket read throughput #48210 documents most important ones I am aware of, but we might have since accumulated more of those.
• There were changes to isolate model which most certainly explain some of Dart 1 vs Dart 2 numbers.

About the last one. The code seems to spawn as many isolates (within a single group) as there are CPU cores. Hardware used in the benchmark is documented to have 56 cores. With default VM settings you can't have that many isolates running concurrently (within a group) because we limit the number of mutators based on the size of the new space, which effectively means that you underutilize cores.

[kevmoo]

My knowledge about AOT vs JIT is more than a couple years old. Good to know!

@mraleph – concrete suggestions about how we should optimize the isolate count? Some min(cores/2, MAX) or similiar?

[iapicca]

My knowledge about AOT vs JIT is more than a couple years old. Good to know!

@mraleph – concrete suggestions about how we should optimize the isolate count? Some min(cores/2, MAX) or similiar?

wouldn't that keep underutilizing the cores?

With default VM settings you can't have that many isolates running concurrently (within a group) because we limit the number of mutators based on the size of the new space, which effectively means that you underutilize cores.

can we have more "groups", I'd be happy to submit a PR for the next round if you point me toward some docs I can follow

[mraleph]

We would need to benchmark but it might be worth doing something like this (pseudo-code, not fully tested):

final numIsolatesPerGroup = 16;  // Need some experiments, maybe 8 is better.
if (!args.contains("g")) {
  final numGroups = numCpus ~/ numIsolatesPerGroup;  
  for (var i = 0; i < numGroups - 1; i++) {
    // Note: need to use dart compile aot-snapshot instead of dart compile exe
    Isolate.spawnUri('main.aot', ['g']);  // spawn a new group
  }
}

for (var i = 0; i < min(numIsolatesPerGroup, numCpus) - 1; i++) {
  Isolate.spawn(...);
}

await runWebServer(...); 

[a-siva]

@iapicca could you let us know if the above suggestion helps ?

[iapicca]

@a-siva
I really appreciate @mraleph and @kevmoo insights
but I cannot really test on my own as I don't have that kind of hardware available
(and the core count is very relevant to the issue)

Intel Xeon Gold 6330 CPU @ 2.00GHz (56 cores), 64 GB of memory

even assuming this approach works there are a (at least) 2 points to address:

• dev ex
• no exe support!