consecutive fireDocumentUpdate

[cdietrich]

hi, what will happen if a second fireDocumentUpdate is emitting while the first is still running.
will the 2nd one wait? the first one cancelled?

[msujew]

Hey @cdietrich,

if you call it in the context of a WorkspaceLock#write operation, the first one will be cancelled. If you don't use the WorkspaceLock, then it depends on whether you call it inside of a DocumentBuilder#onBuildPhase call. If so, the inner will complete first, then the outer one. If you don't then both will run in parallel.

I would recommend you to call the WorkspaceLock#write method though.

[cdietrich]

potential workaround (helps in 98% of the cases)

  protected override async runCancelable(
    documents: LangiumDocument[],
    targetState: DocumentState,
    cancelToken: CancellationToken,
    callback: (document: LangiumDocument) => Promise<unknown>
  ): Promise<void> {
    const filtered = documents.filter(e => e.state < targetState)
    for (const document of filtered) {
      await interruptAndCheck(cancelToken)
      await callback(document)
    }
    await this.notifyBuildPhase(filtered, targetState, cancelToken)
    for (const document of filtered) {
      document.state = targetState
    }
    this.currentState = targetState
  }

  protected override async notifyBuildPhase(
    documents: LangiumDocument[],
    state: DocumentState,
    _cancelToken: CancellationToken
  ): Promise<void> {
    if (documents.length === 0) {
      // Don't notify when no document has been processed
      return
    }
    const listeners = this.buildPhaseListeners.get(state)
    for (const listener of listeners) {
      // await interruptAndCheck(cancelToken);
      await listener(documents, CancellationToken.None)
    }
  }

(just doing the state change at the end does not seem to be sufficient in all cases, i assume this is cause the references are resolved and thus no revalidation will be happening anyway)

[msujew]

I'm not sure this is the right approach:

1. This is way to broad, as that will also affect phases like parsing, and indexing, which should not be repeated for the same file. In fact, this mechanism would require us to "reset" the AST/index for a given file in case the build was interrupted on a different file for the same phase.
2. I'm still not sure why this actually fixes the issue. Invalid files (i.e. those with missing references) are supposed to be relinked anyway due to the isAffected computation. I'm not sure how preventing to advance the state in case of cancellation improves on this.

[cdietrich]

The problem is that the notification (that there are no markers anymore) won’t arrive . If cancel is in between

I can provide the Java program to ease reproduction if it helps

[cdietrich]

package fuse_simulator;

import java.nio.charset.StandardCharsets;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.StandardOpenOption;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

import com.google.common.collect.Lists;

public class FuseSimulator {
	
	public static void main(String[] args) throws Exception {
		String dir = "/Users/dietrich/Downloads/all_end_june/fuse";
		List<String> fileList = new ArrayList<>();
	    Files.walkFileTree(Paths.get(dir), new SimpleFileVisitor<Path>() {
	        @Override
	        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
	            if (!Files.isDirectory(file)) {
	                fileList.add(file.toString());
	            }
	            return FileVisitResult.CONTINUE;
	        }
	    });
	    Collections.shuffle(fileList);
	    List<List<String>> partitions = Lists.partition(fileList, 50);
	    for (List<String> part : partitions) {
	    	System.out.println(part);
	    	for (String file : part) {
	    		Path oldP = Paths.get(file);
	    		Path newP;
	    		if (file.endsWith("_bak")) {
	    			newP = Paths.get(file.substring(0,file.length()-4));
	    		} else {
	    			newP = Paths.get(file+"_bak");
	    		}
	    		List<String> allLines = Files.readAllLines(oldP);
	    		Files.delete(oldP);
	    		Files.createFile(newP);
	    		boolean ls = newP.toString().endsWith("_bak");
				List<List<String>> halves = Lists.partition(allLines, Math.max(1, (int)(allLines.size()/4)));
				for (List<String> linePart : halves) {
					for (String line : linePart) {
		    			Files.write(newP, (line+"\n").getBytes(StandardCharsets.UTF_8), StandardOpenOption.APPEND);
		    		}
					if (!ls && Math.random()<0.05) {
						Thread.sleep((int)(Math.random()*250));
					}
				}
	    		
	    	}
	    }
	}

}

this basically renames all *.extension to *.extension_bak and back to *.extension
assuming in the workspace there is only .extension files

[msujew]

The problem is that the notification (that there are no markers anymore) won’t arrive . If cancel is in between

Ooooh, now I see. Yeah, that makes perfect sense. I know what to do now, thanks a lot for investigating this 👍