Strange behavior when goto-definition is triggered in VSCode

[ymc9]

Hi there,

I'm making good progress on my DSL and now tackling some remaining issues inside of VSCode.

Just a bit of background, my DSL involves a standard library, which I ship with the VSCode extension as a separate text file. To load it, I extended WorkspaceManager and included the document in loadAdditionalDocuments. After that, users can write code that references declarations in the stdlib. I suppose it's the right way of doing the job.

This all works fine until I Cmd+Hover a reference to stdlib (which seems to trigger DefinitionProvider). It seems the document is reparsed, relinked, and revalidated, but without including my standard library, so now all references become unresolved. I double-confirmed this by subclassing DefinitionProvider and always returning undefined for references to stdlib, then the errors are gone (of course, goto-definition doesn't work anymore).

I'm on version 0.5.0. Is this a known issue? Let me know if a repro is needed, but hopefully, you already have something in your mind 😄. Thanks in advance!

[msujew]

Hey @ymc9,

I haven't seen that behavior yet, so nothing that comes to mind on how to easily fix that. I'm kind of confused by the behavior, as builtin libraries shouldn't be reparsed at all. A reproducible example would be great, I'd be interested in looking into that.

Before that, you can take a look at this change which introduces a builtin library guide to our website. Maybe there's already something in there which fixes your issue.

[spoenemann]

builtin libraries shouldn't be reparsed at all

Actually we reparse any document, regardless whether built-in or not, as soon as it's being opened in a text editor, because vscode sends a didOpen notification with the full text document, which is then processed by our DocumentBuilder.

My thoughts:

• Debug what's happening in this case
• Consider adding options to mark LangiumDocuments as unchangeable to suppress processing through DocumentBuilder

[ymc9]

Hey,

Thanks for following up. Unfortunately, I wasn't able to reproduce it with a simple repro 😢. I'll keep digging and see what I can get. I will update here then.

[ymc9]

Hi guys,

I'm continuing to dig into the issue, and I believe it has something to do with how I export global declarations. My language has "enum" constructs and enum fields are exported as global and can be directly resolved to without using a qualified name.

E.g. (conceptually):
// in my standard library
enum Foo { X }

// in user-defined source file
y = X

My standard library is loaded via a customized WorkspaceManager by overriding "loadAdditionalDocuments".

export default class ZModelWorkspaceManager extends DefaultWorkspaceManager {
    protected async loadAdditionalDocuments(
        _folders: WorkspaceFolder[],
        _collector: (document: LangiumDocument) => void
    ): Promise<void> {
        await super.loadAdditionalDocuments(_folders, _collector);
        const stdLibUri = URI.file(
            path.join(__dirname, '../res', STD_LIB_MODULE_NAME)
        );
        console.log(`Adding stdlib document from ${stdLibUri}`);
        const stdlib = this.langiumDocuments.getOrCreateDocument(stdLibUri);
        _collector(stdlib);
    }
}

The enum fields are exported by customizing ScopeComputation by overriding computeExports:

export class ZModelScopeComputation extends DefaultScopeComputation {
    constructor(private readonly services: LangiumServices) {
        super(services);
    }

    async computeExports(
        document: LangiumDocument<AstNode>,
        cancelToken?: CancellationToken | undefined
    ): Promise<AstNodeDescription[]> {
        const result = await super.computeExports(document, cancelToken);

        // add enum fields so they can be globally resolved across modules
        for (const node of streamAllContents(document.parseResult.value)) {
            if (cancelToken) {
                await interruptAndCheck(cancelToken);
            }
            if (isEnumField(node)) {
                const desc =
                    this.services.workspace.AstNodeDescriptionProvider.createDescription(
                        node,
                        node.name,
                        document
                    );
                result.push(desc);
            }
        }

        return result;
    }
}

This worked all fine for CLI, and for VSCode when the user document was initially loaded. However, when I cmd+hover an enum field reference in the user's doc (the enum is defined in the standard library), it seems that VSCode triggers DefaultDocumentBuilder.update method (with changed argument set to the standard library), which results in rebuilding both documents (but with a different order as they were initially built), and in this build pass, the exported global scopes seems gone.

I guess this is related to if the standard library is built before or after the user's doc. I'm not sure how this works internally... is it that scope exports are computed across all documents, and then they're parsed and linked? Or actually, do the full export->parse->link phases happen sequentially per document? If that's the case, I believe a change in build order can result in the problem.

Do you see any anti-pattern in the way how I export global definitions? Also, is my guess above correct? If so, should documents that're loaded via loadAdditionalDocuments always be built first?

Sorry, the explanation is a bit long. I wish I could better understand some assumptions of Langium. Thank you!

[ymc9]

I finally found the cause ... I was storing resolution result on AST nodes and not refreshing them during linking, so had stale reference to old documents. I had the (wrong) assumption that a fresh new AST tree is created for when document is rebuilt :D.

Refreshing the cached resolution fixed the issue. I hope I responded fast enough so you didn't bother to check my previous note. This issue can be closed now.

[msujew]

Ah, yes. To explain that in short: Unless the content has changed (i.e. a text edit happened) the AST is never reparsed. References might be unlinked due to related changes in other files, but the AST is not touched by the framework code.