Skip to content

Conversation

@JoviDeCroock
Copy link
Member

@JoviDeCroock JoviDeCroock commented Nov 28, 2024

While researching the numbers on v15 vs v16 I realised that our printer became much slower, the reason for this is two-fold.

  • We started leveraging a really expensive join function that filters even when it's not needed
  • During visit we always recreate the full tree

Now that second point we should discuss, as changing this might be considered a breaking change.

The way visit currently behaves is that it is a reducer, whether or not the user returns a value it will always fully recreate the AST without any attempt at reusing parts of it, or as is the issue with our printer, it will recreate the full AST even if we aren't even remotely returning anything AST-like, we are returning strings 😅

This PR can serve as an exploration of how we can improve things in v17 - I've seen issues flying around about refactoring the visitor, which this seems like a good trigger judging from how the performance is.

@JoviDeCroock JoviDeCroock requested a review from a team as a code owner November 28, 2024 08:00
@github-actions
Copy link

Hi @JoviDeCroock, I'm @github-actions bot happy to help you with this PR 👋

Supported commands

Please post this commands in separate comments and only one per comment:

  • @github-actions run-benchmark - Run benchmark comparing base and merge commits for this PR
  • @github-actions publish-pr-on-npm - Build package from this PR and publish it on NPM

@JoviDeCroock

This comment has been minimized.

@github-actions
Copy link

@github-actions run-benchmark

@JoviDeCroock

Benchmark output

[email protected] benchmark
node benchmark/benchmark.js "--revs" "HEAD" "BASE"

🍳 Preparing HEAD...
🍳 Preparing BASE...
⏱ Build Schema from AST
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m45.51�[0m ops/sec �[90m±�[0m�[33m2.21�[0m�[36m%�[0m�[90m x �[0m2.18 MB/op�[90m (8 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m45.63�[0m ops/sec �[90m±�[0m�[32m0.71�[0m�[36m%�[0m�[90m x �[0m2.18 MB/op�[90m (8 runs sampled)�[0m

⏱ Build Schema from Introspection
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m54.13�[0m ops/sec �[90m±�[0m�[32m0.67�[0m�[36m%�[0m�[90m x �[0m1.11 MB/op�[90m (11 runs sampled)�[0m
BASE�[90m x �[0m�[32m53.82�[0m ops/sec �[90m±�[0m�[32m1.55�[0m�[36m%�[0m�[90m x �[0m1.12 MB/op�[90m (11 runs sampled)�[0m

⏱ Execute Introspection Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m49.44�[0m ops/sec �[90m±�[0m�[33m2.37�[0m�[36m%�[0m�[90m x �[0m2.48 MB/op�[90m (8 runs sampled)�[0m
BASE�[90m x �[0m�[32m47.89�[0m ops/sec �[90m±�[0m�[32m1.69�[0m�[36m%�[0m�[90m x �[0m2.49 MB/op�[90m (7 runs sampled)�[0m

⏱ Parse introspection query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m13,613�[0m ops/sec �[90m±�[0m�[32m0.16�[0m�[36m%�[0m�[90m x �[0m2.62 KB/op�[90m (35 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m13,637�[0m ops/sec �[90m±�[0m�[32m0.14�[0m�[36m%�[0m�[90m x �[0m2.62 KB/op�[90m (35 runs sampled)�[0m

⏱ Print ktichen-sink query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m2,696�[0m ops/sec �[90m±�[0m�[32m1.10�[0m�[36m%�[0m�[90m x �[0m875 Bytes/op�[90m (11 runs sampled)�[0m
BASE�[90m x �[0m�[32m2,663�[0m ops/sec �[90m±�[0m�[32m0.90�[0m�[36m%�[0m�[90m x �[0m 3.54 KB/op�[90m (11 runs sampled)�[0m

⏱ Many repeated fields
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m 99.1�[0m ops/sec �[90m±�[0m�[32m0.38�[0m�[36m%�[0m�[90m x �[0m524 KB/op�[90m (20 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m99.38�[0m ops/sec �[90m±�[0m�[32m0.18�[0m�[36m%�[0m�[90m x �[0m525 KB/op�[90m (20 runs sampled)�[0m

⏱ Validate Introspection Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m571�[0m ops/sec �[90m±�[0m�[32m0.33�[0m�[36m%�[0m�[90m x �[0m268 KB/op�[90m (14 runs sampled)�[0m
BASE�[90m x �[0m�[32m565�[0m ops/sec �[90m±�[0m�[32m0.45�[0m�[36m%�[0m�[90m x �[0m270 KB/op�[90m (14 runs sampled)�[0m

⏱ Validate Invalid Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m509�[0m ops/sec �[90m±�[0m�[32m0.19�[0m�[36m%�[0m�[90m x �[0m326 KB/op�[90m (14 runs sampled)�[0m
BASE�[90m x �[0m�[32m508�[0m ops/sec �[90m±�[0m�[32m0.19�[0m�[36m%�[0m�[90m x �[0m326 KB/op�[90m (14 runs sampled)�[0m

⏱ Validate SDL Document
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m67.11�[0m ops/sec �[90m±�[0m�[32m1.26�[0m�[36m%�[0m�[90m x �[0m221 KB/op�[90m (11 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m68.35�[0m ops/sec �[90m±�[0m�[32m0.66�[0m�[36m%�[0m�[90m x �[0m203 KB/op�[90m (11 runs sampled)�[0m

⏱ Visit all AST nodes
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m302�[0m ops/sec �[90m±�[0m�[32m0.15�[0m�[36m%�[0m�[90m x �[0m310 KB/op�[90m (20 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m315�[0m ops/sec �[90m±�[0m�[32m1.15�[0m�[36m%�[0m�[90m x �[0m311 KB/op�[90m (20 runs sampled)�[0m

⏱ Visit all AST nodes in parallel
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m28.95�[0m ops/sec �[90m±�[0m�[32m0.53�[0m�[36m%�[0m�[90m x �[0m1.23 MB/op�[90m (7 runs sampled)�[0m
BASE�[90m x �[0m�[32m27.85�[0m ops/sec �[90m±�[0m�[31m6.55�[0m�[36m%�[0m�[90m x �[0m1.23 MB/op�[90m (7 runs sampled)�[0m

@JoviDeCroock

This comment has been minimized.

}
}
} else {
node = Object.defineProperties(
Copy link
Member Author

@JoviDeCroock JoviDeCroock Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a massive deopt, maybe we could keep this in by i.e. rather than copying the node just returning the string if that's what the edit is symbolising

@github-actions
Copy link

@github-actions run-benchmark

@JoviDeCroock

Benchmark output

[email protected] benchmark
node benchmark/benchmark.js "--revs" "HEAD" "BASE"

🍳 Preparing HEAD...
🍳 Preparing BASE...
⏱ Build Schema from AST
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m44.56�[0m ops/sec �[90m±�[0m�[32m1.54�[0m�[36m%�[0m�[90m x �[0m2.18 MB/op�[90m (8 runs sampled)�[0m
BASE�[90m x �[0m�[32m43.08�[0m ops/sec �[90m±�[0m�[33m2.50�[0m�[36m%�[0m�[90m x �[0m2.18 MB/op�[90m (8 runs sampled)�[0m

⏱ Build Schema from Introspection
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m51.34�[0m ops/sec �[90m±�[0m�[32m1.30�[0m�[36m%�[0m�[90m x �[0m1.11 MB/op�[90m (11 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m51.97�[0m ops/sec �[90m±�[0m�[33m2.26�[0m�[36m%�[0m�[90m x �[0m1.12 MB/op�[90m (11 runs sampled)�[0m

⏱ Execute Introspection Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m49.17�[0m ops/sec �[90m±�[0m�[33m3.65�[0m�[36m%�[0m�[90m x �[0m2.48 MB/op�[90m (8 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m50.45�[0m ops/sec �[90m±�[0m�[32m1.97�[0m�[36m%�[0m�[90m x �[0m2.49 MB/op�[90m (8 runs sampled)�[0m

⏱ Parse introspection query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m13,239�[0m ops/sec �[90m±�[0m�[33m2.48�[0m�[36m%�[0m�[90m x �[0m2.62 KB/op�[90m (33 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m13,554�[0m ops/sec �[90m±�[0m�[32m0.25�[0m�[36m%�[0m�[90m x �[0m2.62 KB/op�[90m (35 runs sampled)�[0m

⏱ Print kitchen-sink query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m9,620�[0m ops/sec �[90m±�[0m�[32m0.28�[0m�[36m%�[0m�[90m x �[0m2.14 KB/op�[90m (25 runs sampled)�[0m
BASE�[90m x �[0m�[31m2,660�[0m ops/sec �[90m±�[0m�[32m0.48�[0m�[36m%�[0m�[90m x �[0m3.54 KB/op�[90m (11 runs sampled)�[0m

⏱ Many repeated fields
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m99.53�[0m ops/sec �[90m±�[0m�[32m0.35�[0m�[36m%�[0m�[90m x �[0m524 KB/op�[90m (21 runs sampled)�[0m
BASE�[90m x �[0m�[32m99.18�[0m ops/sec �[90m±�[0m�[32m0.39�[0m�[36m%�[0m�[90m x �[0m525 KB/op�[90m (21 runs sampled)�[0m

⏱ Validate Introspection Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m553�[0m ops/sec �[90m±�[0m�[33m3.91�[0m�[36m%�[0m�[90m x �[0m268 KB/op�[90m (14 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m558�[0m ops/sec �[90m±�[0m�[32m0.49�[0m�[36m%�[0m�[90m x �[0m270 KB/op�[90m (14 runs sampled)�[0m

⏱ Validate Invalid Query
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

�[32mHEAD�[0m�[90m x �[0m�[32m502�[0m ops/sec �[90m±�[0m�[32m0.70�[0m�[36m%�[0m�[90m x �[0m326 KB/op�[90m (14 runs sampled)�[0m
BASE�[90m x �[0m�[32m499�[0m ops/sec �[90m±�[0m�[32m1.64�[0m�[36m%�[0m�[90m x �[0m326 KB/op�[90m (14 runs sampled)�[0m

⏱ Validate SDL Document
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[33m61.47�[0m ops/sec �[90m±�[0m�[33m2.66�[0m�[36m%�[0m�[90m x �[0m221 KB/op�[90m (10 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m65.61�[0m ops/sec �[90m±�[0m�[33m2.29�[0m�[36m%�[0m�[90m x �[0m203 KB/op�[90m (10 runs sampled)�[0m

⏱ Visit all AST nodes
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[33m290�[0m ops/sec �[90m±�[0m�[33m4.58�[0m�[36m%�[0m�[90m x �[0m310 KB/op�[90m (19 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m314�[0m ops/sec �[90m±�[0m�[32m0.75�[0m�[36m%�[0m�[90m x �[0m311 KB/op�[90m (19 runs sampled)�[0m

⏱ Visit all AST nodes in parallel
�[36m1�[0m tests completed.
�[36m2�[0m tests completed.

HEAD�[90m x �[0m�[32m28.49�[0m ops/sec �[90m±�[0m�[32m1.99�[0m�[36m%�[0m�[90m x �[0m1.23 MB/op�[90m (7 runs sampled)�[0m
�[32mBASE�[0m�[90m x �[0m�[32m29.14�[0m ops/sec �[90m±�[0m�[32m0.71�[0m�[36m%�[0m�[90m x �[0m1.23 MB/op�[90m (7 runs sampled)�[0m

Copy link
Contributor

@yaacovCR yaacovCR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks amazing

@yaacovCR
Copy link
Contributor

looks amazing

Might want to make the PR title a bit more specific tho :)

@JoviDeCroock
Copy link
Member Author

After evaluation it doesn't look safe to remove property-descriptors 😅 this would be a breaking change

@JoviDeCroock JoviDeCroock deleted the make-gql-js-faster branch November 28, 2024 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants