Debug: log to show difference in vscode.executeCompletionItemProvider in vscode vs Positron

[vezwork]

Please see the comments on the modified code in this PR.

[vezwork]

For debugging purposes: re-enable completions in the visual editor in Positron

[vezwork]

In vscode, in the extension host, in the developer console:

In Positron, in the extension host, in the developer console:

[vezwork]

This is right before the place where the the visual editor calls a vscode command to gets its completions for Python and R. Let's log out all the arguments to the command.

I looked at this log during a test where I ran an extension host in both vscode and Positron. In an R block in a .qmd, I tried typing lib to get completions.

Here is the log in the debug console in vscode:

Here is the log in the debug console in Positron:

I opened the vdoc in the file system and in both cases it was an r file with lib as the content. I did wc -l to see if there was differences in new lines and there weren't.

In both cases these were logged to have the same values. So... the arguments to the following command seem to be the same in both Positron and vscode.

[vezwork]

Under the test described above (entering lib in a r code block in a qmd in the visual editor) this gives different results in vscode v.s. Positron!

In Positron:

v.s. in vscode:

[juliasilge]

Thank you so much for setting this up! Extremely helpful 🙌

I started digging into this today, and I think the differences between vscode and Positron are because of differences between the underlying R LSP, which are really different systems.

I want to highlight the differences here, just in Positron:

Regular .R file

Quarto file in source mode

Quarto file in visual mode

The good news is that the completions are the same in all situations. The bad news is that in visual mode, they are presented in a different order.

[juliasilge]

I do also see that the completions present when debugging vdoc-completion.ts in Positron are very numerous, like with ~2500 or so items. I just spent a bit of time digging into how we go from that big list to the list of 7 items that get shown in the UI but haven't found that yet.

[vezwork]

Here's an architecture diagram I made of completions in the visual editor:

If we look at the definition of getCompletions in completion.ts in the UI code, we'll see that there is a bunch of ranking and filtering of completions. Those completions ultimately, in the extension code, come from vscode.executeCompletionItemProvider which must talk to the R / Python LSP to get completions.

[vezwork]

When I was testing in Positron, I believe I noticed that no matter what the contents of the R language virtual doc were, vscode.executeCompletionItemProvider always returned the same list of 2649 completions; furthermore when I inspected the completions, almost all of them were not relevant to the contents of the virtual doc. It seemed possibly like a list of all possible completions, regardless of what is being completed.

This seems to be strong evidence that there is a bug outside of Quarto extension code (a bug that is present in Positron but not VScode) that lead to this degradation of quality of completions in the Visual Editor.

It may be possible to work around this bug in the Quarto extension by further refining the ranking and filtering code in completion.ts, but this seems like a hack to me.

[juliasilge]

I set up my many-windows-open-at-once debugging situation today 😆 to dig into this in more detail. I think it may be easier to use a Python example because then we don't get distracted by the R completion model, and to see the differences between the source and visual editor.

Here are some screenshots to get us started:

Source editor

Underlying virtual doc for the source editor

Visual editor

Underlying virtual doc for the visual editor

I did some extensive walking through of various scenarios, and I can confirm that:

• the completions provided from the Positron side are the same in all these situations
• the completions received on the Quarto extension side are the same in all these situation

To me, it looks pretty clear that the differences are happening in the visual editor itself, what you mention here in #777 (comment):

If we look at the definition of getCompletions in completion.ts in the UI code, we'll see that there is a bunch of ranking and filtering of completions.

[vezwork]

Ah okay, this helps a lot! I was getting distracted by the R completions!

I was able to reproduce your Python completion results in Positron. I also attempted to reproduce them in VSCode but got different results. I looked at the getCompletions logs in both and noticed some significant differences

left: VSCode; right: Positron

I have to look more into the sorting and filtering in the extension, but in the meantime, based off these logs, I have a theory that the sortText provided for each completion in VSCode is "better" than in Positron, leading to better sorting in VSCode than in Positron. In some brief testing, it appears that the completions in source editor and visual editor are consistent with eachother in VSCode.

[juliasilge]

We will get different completions in VS Code than in Positron because the underlying LSP is different, and providing different things.

[juliasilge]

You might want to change your VS Code setup to use Jedi instead of Pylance for Python, for a better comparison.

[vezwork]

You might want to change your VS Code setup to use Jedi instead of Pylance for Python, for a better comparison.

Thanks @juliasilge, this cleared up a lot for me!:

I changed my VSCode Python Language Server to Jedi and tested again. Now I get very similar, if not the same, results as in Positron (and similar, if not the same, completions data).

Using Jedi in VSCode, both Positron and VSCode present similar "good" completions in source mode, and present similar "bad" completions in visual mode.

Awesome! I finally feel like I understand where the differences in completions between VSCode and Positron were coming from (they were coming from different completions data, which came from different language servers); and as a result of clearing that up (and ignoring the arc stuff for now), I now see the essential issue isolated to the extension's sorting and/or filtering.

---

Regarding sortText: It does seem to be the case that pylance gives sortText that leads to better completion sorting in the visual editor than Jedi. I can see two possibilities for how this leads to completions differences between the source editor and the visual editor:

• I wonder if the source editor has a way to ask for nicer completions data from Jedi? Seems unlikely?
• Or does the source editor really have hidden fancy magic completion post-language-server-process sorting? I found some code in the Positron repo that might be doing this ...

I suppose we don't have to get to the bottom of which of these two possibilities is the. case. Though if the source editor does have magic sorting, it would be really nice to be informed by that magic! I'm sure we could already improve the sorting quite a bit without being informed though.

[vezwork]

to achieve consistency across languages and to honor different clients usually the client is responsible for filtering and sorting.

Completion - language server protocol specification 3.17

[vezwork]

After going over all this information with @juliasilge (also see posit-dev/ark#900), we have arrived at a plan for a solution to the problem of completions not being properly sorted and filtered in the VE in Positron (we now know it affects Jedi users in VSCode as well).

I will try copying&pasting from what seems to be the code where the source editor sorts completions. This will give consistency between the source editor and the visual editor; it will respect the LSP spec which says completion sorting and filtering should be handled by the client; it adds a maintenance burden (if the source editor sorting/filtering does change and we want the visual editor to remain consistent with it).