Merge pull request #14 from lightpanda-io/lpd-cdp

Update becnhmark for Lightpanda CDP

Author: krichprollsch

README.md

@@ -39,7 +39,7 @@ $ apt install time hyperfine
 ```
 
 You have also to install [Google Chrome](https://www.google.com/chrome/) and
-Lightpanda browser, but the code is not publicly available yet.
+[Lightpanda browser](https://github.com/lightpanda-io/browser/releases/tag/nightly).
 
 ### Demo web page
 
@@ -65,28 +65,30 @@ This bench is a very basic test to compare the two software.
 We start the browser and request the fake web page once with full JS execution. The final DOMTree is
 rendered in stdout.
 
-We use Google Chrome version 122.0.6261.94.
+We use Google Chrome version 130.0.6723.58.
 
 ```console
 $ google-chrome --version
-Google Chrome 122.0.6261.94
+Google Chrome 130.0.6723.58
 ```
 
+And Lightpanda commit [1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2](https://github.com/lightpanda-io/browser/commit/1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2).
+
 ### Execution time
 
 ```console
-$ hyperfine --warmup 3 --runs 20 --shell=none "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/" "./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/"
-Benchmark 1: google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/
-  Time (mean ± σ):     556.7 ms ±  10.2 ms    [User: 360.8 ms, System: 170.6 ms]
-  Range (min … max):   538.2 ms … 571.6 ms    20 runs
+$ hyperfine --warmup 3 --runs 20 --shell=none "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/" "./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/"
+Benchmark 1: google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/
+  Time (mean ± σ):     618.9 ms ±  12.8 ms    [User: 363.0 ms, System: 162.0 ms]
+  Range (min … max):   598.2 ms … 659.2 ms    20 runs
 
-Benchmark 2: ./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/
-  Time (mean ± σ):       8.6 ms ±   0.2 ms    [User: 5.0 ms, System: 3.2 ms]
-  Range (min … max):     8.3 ms …   9.0 ms    20 runs
+Benchmark 2: ./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/
+  Time (mean ± σ):       9.9 ms ±   0.3 ms    [User: 5.8 ms, System: 3.7 ms]
+  Range (min … max):     9.3 ms …  10.4 ms    20 runs
 
 Summary
-  './browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/' ran
-   64.48 ± 1.74 times faster than 'google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/'
+  './lightpanda --dump http://127.0.0.1:1234/campfire-commerce/' ran
+   62.58 ± 2.32 times faster than 'google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/'
 ```
 
 ![aws.m5 hyperfine](./img/aws_m5_hyperfine.png)
@@ -96,47 +98,48 @@ Summary
 ```console
 $ /usr/bin/time -v google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/
         Command being timed: "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/"
-        User time (seconds): 0.38
-        System time (seconds): 0.14
-        Percent of CPU this job got: 96%
-        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.55
+        User time (seconds): 0.34
+        System time (seconds): 0.19
+        Percent of CPU this job got: 94%
+        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.57
         Average shared text size (kbytes): 0
         Average unshared data size (kbytes): 0
         Average stack size (kbytes): 0
         Average total size (kbytes): 0
-        Maximum resident set size (kbytes): 169924
+        Maximum resident set size (kbytes): 174096
         Average resident set size (kbytes): 0
-        Major (requiring I/O) page faults: 5
-        Minor (reclaiming a frame) page faults: 20535
-        Voluntary context switches: 2664
-        Involuntary context switches: 1655
+        Major (requiring I/O) page faults: 17
+        Minor (reclaiming a frame) page faults: 20609
+        Voluntary context switches: 2563
+        Involuntary context switches: 1618
         Swaps: 0
-        File system inputs: 0
-        File system outputs: 1624
+        File system inputs: 1048
+        File system outputs: 4576
         Socket messages sent: 0
         Socket messages received: 0
         Signals delivered: 0
         Page size (bytes): 4096
         Exit status: 0
+
 ```
 
 ```console
-$ /usr/bin/time -v ./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/
-        Command being timed: "./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/"
+$ /usr/bin/time -v ./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/
+        Command being timed: "./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/"
         User time (seconds): 0.00
         System time (seconds): 0.00
-        Percent of CPU this job got: 100%
+        Percent of CPU this job got: 92%
         Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.01
         Average shared text size (kbytes): 0
         Average unshared data size (kbytes): 0
         Average stack size (kbytes): 0
         Average total size (kbytes): 0
-        Maximum resident set size (kbytes): 14348
+        Maximum resident set size (kbytes): 21276
         Average resident set size (kbytes): 0
         Major (requiring I/O) page faults: 0
-        Minor (reclaiming a frame) page faults: 751
+        Minor (reclaiming a frame) page faults: 925
         Voluntary context switches: 6
-        Involuntary context switches: 70
+        Involuntary context switches: 11
         Swaps: 0
         File system inputs: 0
         File system outputs: 0
@@ -147,60 +150,149 @@ $ /usr/bin/time -v ./browsercore-get --dump http://127.0.0.1:1234/campfire-comme
         Exit status: 0
 ```
 
-## Multiple requests using Playwright
+## Multiple requests using Puppeteer
 
 We compare now multiple page loads and js evaluations using
-[Playwright](https://playwright.dev), which connects to the browser using CDP (Chrome Debug Protocol).
+[Puppeteer](https://https://pptr.dev/), which connects to the browser using CDP
+(Chrome Debug Protocol).
 
 ### Dependencies
 
 To run the benchmark, you need ti install [nodejs](https://nodejs.org/en/download).
 
 Once `nodejs` is installed, please run a `npm install` to install nodejs
-dependencies, mainly Playwright.
+dependencies, mainly Puppeteer.
 
 You have also to install [Google Chrome](https://www.google.com/chrome/) and
 Lightpanda browser, but the code is not publicly available yet.
 
 ### Running the benchmark
 
-The `playwright/cdp.js` benchmark accepts multiple env vars to be configured.
+The `puppeteer/cdp.js` benchmark accepts multiple env vars to be configured.
 * `BROWSER_ADDRESS` is the address of the running browser listening the CDP protocol, by default `http://127.0.0.1:9222`.
 * `BASE_URL` is the base url of the running web reser to request, by default `http://127.0.0.1:1234`.
 * `RUNS` is the number of pages loaded by the benchmark, default is `100`.
 
-`npm run bench-playwright-cdp` starts a playwright process
+`npm run bench-puppeteer-cdp` starts a Puppeteer process
 instance and load the page to extract data 100 times.
 
 ```console
-$ npm run bench-playwright-cdp
+$ npm run bench-puppeteer-cdp
 ```
 
 ### Results
 
 **Google Chrome**
 
-We use Google Chrome version 123.0.6312.105.
+We use Google Chrome version 130.0.6723.58.
 
 You have to start the browser first.
 ```console
-$ google-chrome --headless=new --disable-gpu --remote-debugging-port=9222
+$ /usr/bin/time -v google-chrome --headless=new --remote-debugging-port=9222
 ```
 
 Then you can run the benchmark.
 ```console
-$ npm run bench-playwright-cdp
+$ npm run bench-puppeteer-cdp
 
-> [email protected] bench-playwright-cdp
-> node playwright/cdp.js
+> [email protected] bench-puppeteer-cdp
+> node puppeteer/cdp.js
 
 ................................................................................
 ....................
 total runs 100
-total duration (ms) 18792
-avg run duration (ms) 184
-min run duration (ms) 168
-max run duration (ms) 323
+total duration (ms) 25218
+avg run duration (ms) 248
+min run duration (ms) 216
+max run duration (ms) 451
+```
+
+![aws.m5 Puppeteer with Google Chrome](./img/aws_m5_puppeteer_chrome.png)
+
+```console
+        Command being timed: "google-chrome --headless=new --remote-debugging-port=9222"
+        User time (seconds): 16.19
+        System time (seconds): 6.59
+        Percent of CPU this job got: 41%
+        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:54.51
+        Average shared text size (kbytes): 0
+        Average unshared data size (kbytes): 0
+        Average stack size (kbytes): 0
+        Average total size (kbytes): 0
+        Maximum resident set size (kbytes): 207828
+        Average resident set size (kbytes): 0
+        Major (requiring I/O) page faults: 138
+        Minor (reclaiming a frame) page faults: 265165
+        Voluntary context switches: 143625
+        Involuntary context switches: 82912
+        Swaps: 0
+        File system inputs: 35680
+        File system outputs: 169688
+        Socket messages sent: 0
+        Socket messages received: 0
+        Signals delivered: 0
+        Page size (bytes): 4096
+        Exit status: 0
+```
+
+**Lightpanda browser**
+
+We use Lightpanda commit [1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2](https://github.com/lightpanda-io/browser/commit/1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2).
+
+You have to start the Lightpanda Gateway.
+```console
+./gateway
 ```
 
-![aws.m5 Playwright with Google Chrome](./img/aws_m5_playwright_chrome.png)
+And Lightpanda browser itself.
+```console
+/usr/bin/time -v ./lightpanda
+```
+
+Then you can run the benchmark.
+```console
+$ npm run bench-puppeteer-cdp
+
+> [email protected] bench-puppeteer-cdp
+> node puppeteer/cdp.js
+
+................................................................................
+....................
+total runs 100
+total duration (ms) 3456
+avg run duration (ms) 32
+min run duration (ms) 23
+max run duration (ms) 105
+```
+
+![aws.m5 Puppeteer with Lightpanda browser](./img/aws_m5_puppeteer_lightpanda.png)
+
+```console
+        Command being timed: "./lightpanda"
+        User time (seconds): 41.78
+        System time (seconds): 14.15
+        Percent of CPU this job got: 99%
+        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.14
+        Average shared text size (kbytes): 0
+        Average unshared data size (kbytes): 0
+        Average stack size (kbytes): 0
+        Average total size (kbytes): 0
+        Maximum resident set size (kbytes): 100940
+        Average resident set size (kbytes): 0
+        Major (requiring I/O) page faults: 0
+        Minor (reclaiming a frame) page faults: 29326
+        Voluntary context switches: 235
+        Involuntary context switches: 4099
+        Swaps: 0
+        File system inputs: 0
+        File system outputs: 0
+        Socket messages sent: 0
+        Socket messages received: 0
+        Signals delivered: 0
+        Page size (bytes): 4096
+        Exit status: 0
+```
+
+---
+
+Console images generated with [Carbon](https://carbon.now.sh).