CST memory optimization#2191
Open
spoenemann wants to merge 10 commits into
Open
Conversation
Contributor
|
in my measurements we see only 35% memory gain (on the total heap) but this is still a huge improvement. for pure cst it is almost 50% |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR aims at improving Langium's memory usage, which was found to be relatively high in this article. The Conrete Syntax Tree (CST) implementation is the main reason for this. These measures have been applied:
rootis no longer stored in all CST nodes, but derived by walking up the container hierarchy.rangeis no longer stored in leaf CST nodes, but created on demand from integer values._astNodeand_rangeCache) are initialized withundefinedso the V8 engine includes them in the finalized object layout (this avoids the overhead of a separatePropertyArrayfor properties added later).contentis now areadonlyarray and must be manipulated via the CstNodeBuilder so we avoid the CstNodeContainer overhead.The first two changes save memory by doing a bit more computation for the
root,text, andrangeproperties, which mostly affects LSP-related services. The overhead of these computations is virtually negligible, and the benefits of the changes outweigh it by far as shown in the benchmarks below.I created four benchmarks to measure the impact of these changes on the CST size, parsing, and document building:
cst-node-memory-benchmark.tsmeasures the memory usage of a synthetically generated CSTcst-parse-memory-benchmark.tsmeasures the memory usage of a parsed CSTparser-time-benchmark.tsmeasures the execution time of the parserworkspace-build-benchmark.tsmeasures the execution time and heap size of a full workspace build