Resolving cross references in Worker Threads

[vibsid0986]

Hi Team,

I'm currently engaged in the development of a code generator for a Langium-based DSL. In the code generation process, we leverage worker threads in Node.js to dynamically instantiate new Langium services. Subsequently, we create the Abstract Syntax Tree (AST) model by transmitting the file URI from the main thread. Despite the overall success of the implementation, I'm encountering a challenge related to resolving cross-references.

Specifically, I've observed that the cross-reference of a particular object model is readily accessible when examined in the main thread. However, when attempting to access the same cross-reference in the worker thread, it appears to be undefined.

Any insights or guidance on resolving this issue would be greatly appreciated
Please check ref objects for same model in both worker and main threads.

In worker thread

In main thread

Thanks in advance.

[msujew]

Hey @vibsid0986,

Are you only parsing the document in the worker, or are you also building it? The reference in question neither has a ref nor an error, so it seems like it didn't go through the linking process yet. It might be a good idea to share the code that you're using to create the document. Also, please make sure that the worker that builds the document has all necessary documents for linking available in memory - the index for resolving references cannot be shared across workers.

[vibsid0986]

Hi @msujew ,

Thank you for your prompt response. Initially, I focused on parsing the document. However, even when transitioning to the document-building phase, I encountered an obstacle. The build() function, crucial to the construction process, returns a Promise. Unfortunately, worker threads are unable to seamlessly process Promises.

Below is a snippet of the code utilized for reference. Kindly inform me if additional information is required for a more comprehensive understanding of the issue.

const langDocs: LangiumDocument<AstNode>[] = []

documentsURI.forEach(file => {
    const langDocument = services.shared.workspace.LangiumDocuments.getOrCreateDocument(URI.parse(file));
    langDocs.push(langDocument);
});

services.shared.workspace.DocumentBuilder.build(langDocs);

Thanks in advance.

[vibsid0986]

Thank you for pointing out the correction. Currently, I'm using the code snippet provided below. Unfortunately, I've encountered an error, and a screenshot depicting the issue is attached for reference

const langDocs: LangiumDocument<AstNode>[] = [];
const getBuildLangiumDocuments = async (): Promise<void> => {
    documentsURI.forEach(file => {
        const langDocument = services.shared.workspace.LangiumDocuments.getOrCreateDocument(URI.parse(file));
        langDocs.push(langDocument);
    });
    return services.shared.workspace.DocumentBuilder.build(langDocs);
};
await getBuildLangiumDocuments();

[msujew]

Unfortunately, the screenshot helps very little, aside from the info that the cross reference cannot be resolved. Non resolving references can have any number of causes:

• Is there even an element in the documents with the given name?
• Have the documents all been parsed correctly?
• Is the scope provider working correctly?

[vibsid0986]

Thanks @msujew for your help,

I have refined my solution strategy by exclusively passing the workspace folder path to the worker thread. Within the worker thread, I am re-invoking the initializeWorkspace() function. This modification proves effective, seamlessly resolving all cross-references automatically, obviating the need to traverse each stage of the document lifecycle individually. I would appreciate your insights on whether this is an advisable approach.

[msujew]

Since workers don't share memory, documents loaded in one worker/thread cannot be accessed by any other thread. If you need access to the whole workspace in the worker, loading it via initializeWorkspace is recommended.

obviating the need to traverse each stage of the document lifecycle individually

Note that each document is still going through each step of the document lifecycle via the DocumentBuilder. There's no real way to prevent this (nor would that be advisable).

[vibsid0986]

Thanks @msujew for the help.