Scope Provider Help

[snarkipus]

Tags: @msujew

Alright, I'm giving in and asking/pleading for help since after several weeks of banging my head against it, I think there's something I'm fundamentally missing (more likely a flaw in my understanding). Unless requested otherwise, I'll just post links to code rather than giant code blocks out of context.

I'm about halfway done with the final section of this reference on Xtext scoping and it has proven to be incredibly difficult for me to work through. Thus far, most of Langium loosely follows the basic Xtext way of doing things. Obviously, the default scoping implementation is quite different with the onus placed on the dev to tailor it to their needs.

• Example Language Repo: Small Java

Question 1: Default Scoping Testing

To me, the Xtext IScope getScope(EObject context, EReference Reference) method is a very 'reference-type'-centric view of the scoping situation. From that perspective, I eventually thought that determining scopes would be quite similar to resolving completions so I took inspiration from that.

• Is this a valid/sound testing approach?

Question 2: Scope Computation Customization

The DomainModel DSL example implementation is easy enough to follow by virtue of the fact that 'Packages' and the 'Domain Model' are all comprised of commonly named elements thereby making it somewhat trivial to iterate through. This example language is admittedly more complex with more flexibility in nesting. Unfortunately, the Xtext method (which relies on niceties like takeWhile) is quite different from the DomainModel solution so I've hit a brick wall on this. I'm stuck in the middle of extending the Default Scope Computation class and need a fresh set of eyes if possible.

• Small Java Scope Computation
• Test Cases

Thanks in advance.

[msujew]

Hey @snarkipus,

I haven't looked to deep into your language repo, but instead provided a small example with the type-system branch of the langium-lox project, for which I've just added class based scoping. Please check it out and take a look at it. I believe my code speaks louder and more clearly than any of my verbose explanations are able to. Also, note the following:

1. Member based scoping (i.e. chaining members of classes/types such as a.b.c) happens on the ScopeProvider level, not the ScopeComputation level. Both the indexing (exporting of nodes) and the scope pre-computation needs to be already done before we can actually start performing member based scoping, as names of elements, classes, etc. need to be available during that time. However, Java style fully-qualified-name handling (package names) can be added to the indexing phase.
2. Member scoping also requires some sort of type system in place. To scope to a member, we first need to find the type of the previous member call. This also needs to happen recursively until we reach the root of the member call. Having some type system in place helps tremendously for this task. See the linked branch for an example.
3. For all of this to work well, the grammar needs to be structured in ways that allow us to easily extract the information that we need. This is quite non-trivial, as seemingly minor changes in the grammar can have a huge influence on this. Make sure to follow the approach outlined in the lox language for this to work correctly.

Note that there's still some stuff missing in the branch, notably constructors and function overriding, among other minor issues. I'll try to add those soon, but I'm quite busy this month between vacation (which just ended), the flu, and preparations for multiple presentations such as meetups and EclipseCon 2022. Afterwards I'll probably have some more time, which I plan to dedicate on some more Langium documentation, especially regarding scoping.

[snarkipus]

Thanks @msujew for the very detailed and helpful response!

Member based scoping (i.e. chaining members of classes/types such as a.b.c) happens on the ScopeProvider level, not the ScopeComputation level. Both the indexing (exporting of nodes) and the scope pre-computation needs to be already done before we can actually start performing member based scoping, as names of elements, classes, etc. need to be available during that time. However, Java style fully-qualified-name handling (package names) can be added to the indexing phase.

I finally arrived at this conclusion, although admittedly in the blind. It turns out that when Wallaby tells you 'this code never gets used' it actually means it. Your explanation provides the context I was looking for as to 'why'.

Member scoping also requires some sort of type system in place. To scope to a member, we first need to find the type of the previous member call. This also needs to happen recursively until we reach the root of the member call. Having some type system in place helps tremendously for this task. See the linked branch for an example.

Thank you for the typing system example! I implemented a 'snarkipus best-effort' type system using the Xtext example from this book. Admittedly, I bailed out when it got to deeper conformance testing since, at the time, I assumed it was purely academic and my actual DSL implementation doesn't require it. Hopefully, I can grok your example code for using the type system.

• Small Java Type Computer and tests
• Small Java Type Conformance and tests

For all of this to work well, the grammar needs to be structured in ways that allow us to easily extract the information that we need. This is quite non-trivial, as seemingly minor changes in the grammar can have a huge influence on this. Make sure to follow the approach outlined in the lox language for this to work correctly.

I've come to discover (the hard way) that almost none of this is non-trivial. :) I feel like you should have warned me months ago! If nothing else, I learned TypeScript I suppose.

I'm quite busy this month between vacation (which just ended), the flu, and preparations for multiple presentations such as meetups and EclipseCon 2022.

A well earned vacation I'm sure ... although, if you had the flu on vacation then you're doing it wrong. :) Exciting stuff with the presentations - hopefully they'll be available on YouTube for those of us on the other side of the pond.

FWIW, once I'm done with this 'Small Java' exercise my intent is just to hand over a somewhat faithful, albeit possibly flawed, academic treatment of Langium (including test code). Perhaps it could be helpful to folks looking for a Xtext-to-Langium comparison (Chapters 9-10) that incorporates all of the concepts presented in the first 8 chapters.