Scopes: Spec text for the "scopes" proposal #196

szuend · 2025-04-09T09:58:47Z

Preview

Rebased on top of #193 and #194.

This is an early draft of the "scopes" proposal spec text. Sharing it already to get some early feedback and to write it out in the open :)

Done

Record types for scopes and ranges
Grammar for scopes and ranges
Decoding abstract operations/SDOs for scopes and ranges

ToDo

Add "optionally report an error" handling throughout the decoding where it makes sense.
Add a section that defines bounds of generated ranges w.r.t. to Ecma262 AST.
Add normative (or probably informative) section how to correctly utilize the scopes info (similar to Spec: add algorithm for retrieving original position #195):
- How to retrieve original function names
- How to expand stack frames with inlined functions
- How to condense stack traces with outlined functions

nicolo-ribaudo

(I only went through the data structures so far)

spec.emu

nicolo-ribaudo · 2025-07-02T10:42:27Z

spec.emu

+          </tr>
+        </table>
+      </emu-table>
+      <p>A root Decoded Generated Record describes a distinct part of the generated code. Each Generated Range Record may be associated with a Original Scope Record as indicated by the presence of the [[Definition]] field.</p>


"Decoded Generated Record" is not auto-linked.

I think the ecmarkup gets confused as there is currently a definition for both the terms "generated range" and "generated range record". Maybe we have to rename it to "range record", or remove the definition for "generated range". Wdyt?

spec.emu

* Combine 'IsStackFrame' and 'IsHidden' of generated range into a single enum field. * Get rid of Sub-Range Binding Record in favor of a list of list of binding records.

spec.emu

jridgewell · 2025-07-11T06:48:48Z

spec.emu

+          <emu-alg>
+            1. Let _relativeName_ be the VLQSignedValue of |Vlq|.
+            1. Set _state_.[[ScopeNameIndex]] to _state_.[[ScopeNameIndex]] + _relativeName_.
+            1. Set _scope_.[[Name]] to _names_[_state_.[[ScopeNameIndex]]].


Do we need to check names length?

Depends on how lists are defined. If out-of-bound reads results in *null*, this would be fine. But we probably want to "optionally report an error" in such cases anyway.

jridgewell · 2025-07-11T06:58:29Z

spec.emu

+            1. If |RangeLine| is present, perform DecodeGeneratedRangeItem of |RangeLine| with arguments _range_, _state_ and _names_.
+            1. Set _range_.[[Start]].[[Line]] to _state_.[[RangeLine]].


Ditto this creates ambiguity in the parser.

jridgewell · 2025-07-11T06:58:47Z

spec.emu

+            1. Assert: |ScopeName| is present if and only if, _flags_ & 0x1 = 0x1.
+            1. If |ScopeName| is present, perform DecodeOriginalScopeItem of |ScopeName| with arguments _scope_, _state_ and _names_.


I'm not sure we can assert this because ScopeName and ScopeKind are ambiguous. We could pass different args based on the value, though, if both were defined as Vlq instead of special named productions.

Yeah, the grammar currently is not context-free. If we define both as Vlq, how do I reference them from the SDO?

You can reference them as the first Vlq or the second Vlq(eg https://tc39.es/ecma262/multipage/abstract-operations.html#sec-runtime-semantics-stringnumericvalue:~:text=Let%20a%20be%20the%20MV%20of%20the%20first). We'd need to call a differently named SDO instead of reusing DecodeOriginalScopeItem. Or we could define an AO that operates on Parse Nodes.

I gave it a shot. See commit f15b006 for the diff. I'm not a huge fan as it makes the production/decoding more complicated. Does it really need to be unambiguous?

Note that we should probably turn these assertions into "optionally report an error".

spec.emu

Co-authored-by: Justin Ridgewell <[email protected]>

* Include 'scopes' in the overall source map example * Clarify pre-order for scope/range start/end items * Clarify ~none~ for StackFrametype * Introduce 'BindingExpression' as a dedicated name for sub-range binding VLQs * Don't reserve 'A' for as an item tag

nicolo-ribaudo

Grammar review

nicolo-ribaudo · 2025-08-21T14:50:42Z

spec.emu

+            Tag
+            Tag UnknownVlqList


Suggested change

Tag

Tag UnknownVlqList

Tag UnknownVlqList?

nicolo-ribaudo · 2025-08-21T15:02:12Z

spec.emu

+        Scopes :
+          OriginalScopeTreeList
+          TopLevelItemList
+          OriginalScopeTreeList `,` TopLevelItemList


It would be helpful to have either a comment saying that OriginalScopeTreeList is guaranteed to only start with ,/B or to be empty, and that TopLevelItemList is guaranteed to not be empty and not start with ,/B.

It's not obvious that there is no ambiguity here otherwise, since it requires going down into the grammar multiple levels across different sections.

In addition to that, I don't really like that an eager parser cannot parse this Scopes production. When parsing the string ,,X, a parser would:

parse ,, as a OriginalScopeTreeList

see that ,, ends with a , is followed by something that is a valid TopLevelItemList, and backtrack by 1 so that now the OriginalScopeTreeList is only ,

Consume the now available ,

Parse TopLevelItemList

I also slightly dislike that in ,,X we have two original scope trees, while in ,, we have three.

What if the last trailing comma for OriginalScopeTreeList was always ignored, like it is inside array literals in JS, so that the grammar for Scope would become

Scope : OriginalScopeTreeList ScopeCommas? OriginalScopeTreeList ScopeCommas TopLevelItemList OriginalScopeTreeList : [empty] OriginalScopeTree OriginalScopeTreeList ScopeCommas OriginalScopeTree ScopeCommas : `,` ScopeCommas `,`

and:

would fall in the first case, with an empty OriginalScopeTreeList and no ScopeCommas

, would have an empty OriginalScopeTreeList followed by a ScopeCommas with one comma

,, would have an empty OriginalScopeTreeList followed by a ScopeCommas with two commas

,,X would have an empty OriginalScopeTreeList followed by a ScopeCommas with two commas

B...,,B...,,X would have a B...,,B... OriginalScopeTreeList followed by a ScopeCommas with two commas and then TopLevelItemList

When decoding, for we would first decode OriginalScopeTreeList and then add as many nulls as the number of commas in Scope's ScopeCommas minus 1.

nicolo-ribaudo · 2025-08-21T15:28:52Z

spec.emu

+        <p>In grammar form:</p>
+        <emu-grammar type="definition">
+          OriginalScopeTree :
+            OriginalScopeStart OriginalScopeVariablesItem? OriginalScopeItemList? `,` OriginalScopeEnd


Very slight preference here for doing OriginalScopeStart `,` OriginalScopeVariablesItem? OriginalScopeItemList? OriginalScopeEnd and giving OriginalScopeVariablesItem and OriginalScopeItemList a trailing rather than leading comma, so that the disambiguation for whether they are present or not is on the first character and not on the second.

nicolo-ribaudo · 2025-08-21T15:30:02Z

spec.emu

+          ScopeVariableList :
+            ScopeVariable
+            ScopeVariableList ScopeVariable


I haven't checked yet the SDOs, but it might make sense to have a generic VlqList production that we can reuse in multiple places.

nicolo-ribaudo · 2025-08-21T15:33:39Z

spec.emu

+        <p>In grammar form:</p>
+        <emu-grammar type="definition">
+          GeneratedRangeTree :
+            GeneratedRangeStart GeneratedRangeBindingsItem? GeneratedRangeCallSiteItem? GeneratedRangeItemList? `,` GeneratedRangeEnd


Same here about using trailing rather than leading commas

nicolo-ribaudo · 2025-08-21T15:51:17Z

spec.emu

+        </h1>
+        <dl class="header"></dl>
+        <emu-alg>
+          1. Let _scope_ be a new Original Scope Record with all fields default initialized.


Things like [[Start]] do not have a clear default, it'd be better to explicitly initialize the record.

nicolo-ribaudo · 2025-08-21T16:23:43Z

I find the decoding algorithms very hard to follow, with multiple mutable data structures used at the same time. I'm gong to submit a PR to your branch with a potential alternative, if I can find a way to make it clearer.

nicolo-ribaudo · 2025-08-21T16:41:10Z

For ScopeFlags and ScopeNameOrKind/ScopeKind, what if instead of having a bit that says "is the first one present" and one that says "is the second one present" (leading to potentially invalid states, such as saying that both are present but then they are not), we had a single bit that says "if both are present, then it's the name/kind".

By doing so if neither or both are present we don't even need to check the bit.

nicolo-ribaudo · 2025-08-21T15:56:27Z

spec.emu

+            1. Set _scope_.[[Start]].[[Column]] to _state_.[[ScopeColumn]].
+            1. Let _flags_ be the VLQUnsignedValue of |ScopeFlags|.
+            1. If _flags_ & 0x3 = 0x1, then
+              1. Assert: |ScopeNameOrKind| is present and |ScopeKind| is not present.


Is this actually guaranteed? What if I set 0x1 in the flags, and then put both |ScopeNameOrKind| and |ScopeKind| in it? Should we optionally report an error and return instead? Or maybe have an "early error" in the grammar that ensures it?

nicolo-ribaudo · 2025-08-21T15:56:37Z

spec.emu

+            1. Let _flags_ be the VLQUnsignedValue of |ScopeFlags|.
+            1. If _flags_ & 0x3 = 0x1, then
+              1. Assert: |ScopeNameOrKind| is present and |ScopeKind| is not present.
+              1. DecodeOriginalScopeName(|ScopeNameOrKind|, _scope_, _state_, _names_).


Suggested change

1. DecodeOriginalScopeName(|ScopeNameOrKind|, _scope_, _state_, _names_).

1. Perform DecodeOriginalScopeName(|ScopeNameOrKind|, _scope_, _state_, _names_).

(and the others)

szuend force-pushed the scopes branch 2 times, most recently from 26d4baf to 115f0b0 Compare April 24, 2025 09:27

Scopes: Initial draft of scope structure and decoding

261a3f8

szuend force-pushed the scopes branch from 115f0b0 to 261a3f8 Compare June 30, 2025 09:34

nicolo-ribaudo reviewed Jul 2, 2025

View reviewed changes

Adress first round of review comments for data structures

3bc364b

* Combine 'IsStackFrame' and 'IsHidden' of generated range into a single enum field. * Get rid of Sub-Range Binding Record in favor of a list of list of binding records.

jridgewell reviewed Jul 11, 2025

View reviewed changes

szuend and others added 3 commits July 14, 2025 07:36

Smaller typo fixes and suggestions from code review

f79b426

Co-authored-by: Justin Ridgewell <[email protected]>

Make OriginalScopeStart production context-free

f15b006

hbenl mentioned this pull request Aug 12, 2025

Scopes: Algorithm for computing original stack frames #219

Open

nicolo-ribaudo reviewed Aug 21, 2025

View reviewed changes

This was referenced Aug 21, 2025

Refactor scopes tree decoding szuend/source-map#1

Closed

Scopes: Refactor scopes tree decoding #220

Merged

nicolo-ribaudo added the Proposal: Scopes label Aug 28, 2025

		1. If \|RangeLine\| is present, perform DecodeGeneratedRangeItem of \|RangeLine\| with arguments _range_, _state_ and _names_.
		1. Set _range_.[[Start]].[[Line]] to _state_.[[RangeLine]].

		1. Assert: \|ScopeName\| is present if and only if, _flags_ & 0x1 = 0x1.
		1. If \|ScopeName\| is present, perform DecodeOriginalScopeItem of \|ScopeName\| with arguments _scope_, _state_ and _names_.

	1. DecodeOriginalScopeName(\|ScopeNameOrKind\|, _scope_, _state_, _names_).
	1. Perform DecodeOriginalScopeName(\|ScopeNameOrKind\|, _scope_, _state_, _names_).

Scopes: Spec text for the "scopes" proposal #196

Are you sure you want to change the base?

Scopes: Spec text for the "scopes" proposal #196

Uh oh!

Conversation

szuend commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Done

ToDo

Uh oh!

nicolo-ribaudo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nicolo-ribaudo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nicolo-ribaudo commented Aug 21, 2025

Uh oh!

nicolo-ribaudo commented Aug 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

szuend commented Apr 9, 2025 •

edited

Loading