Help with writing, running, and integrating a custom language server

[HalfMillennium]

Description

I'm trying to integrate my SQL language server into jupyter-lsp, and the extension seems to recognize the LSP that I've configured. However, something about how I've set things up causes the extension to hang on "Connecting..." when I load JupyterLab, and I receive a Permission denied error in my console. I used the following configuration code for my LSP, which is free of schema errors:

{
  "LanguageServerManager": {
    "language_servers": {
      "elastic-lsp": {
        "version": 2,
        "argv": ["/Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp", "--stdio"],
        "languages": ["sql"],
        "mime_types": ["text/sql", "text/x-sql"]
      }
    }
  }
}

Could this be an issue with file path? I pointed to the root of my LSP, but should I be pointing at a specific server file?

Reproduce

1. Install my elastic-lsp with jlpm add --dev elastic-lsp
2. Restart JLab and wait for the error to appear in the console.

Context

• Operating System and version: macOS Big Sur
• Browser and version: Safari 14.1
• JupyterLab version: 3.0.16
• jupyter-lsp version:
• jupyterlab-lsp version: 3.8.0
• Language Server: elastic-lsp
• Language Server version: 1.0.14
• Language Server installed with: npm
• Language Server Spec

# jupyter_server_config.json
{
  "LanguageServerManager": {
    "language_servers": {
      "elastic-lsp": {
        "version": 2,
        "argv": ["/Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp", "--stdio"],
        "languages": ["sql"],
        "mime_types": ["text/sql", "text/x-sql"]
      }
    }
  }
}

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here.

[HalfMillennium]

This is what I see when I click on "Connecting..."

Strangely, I don't see "elastic-lsp" at all.

[krassowski]

Well "elastic-lsp" does not show up because you did not specify "display_name". It is indirectly documented via link to full schema, but feel welcome to open a pull request to mention it in the documentation.

[krassowski]

@HalfMillennium what exactly is /Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp? Something that I was hinting at in a comment to your stack overflow question was that PermissionError: [Errno 13] Permission denied shows up when you try to use a directory as if it was a file. My best guess is that /Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp is a directory rather than an executable.

[HalfMillennium]

@krassowski Ah okay, thank you for the clarification about the name.

And yes, this does seem to be the case. However, when I try running NodeJS file server.js (created as output to running tsc server.ts) with node .../vscode-extension-samples/elastic-lsp/server/src/server.js --node-ipc I receive no output or confirmation that the server is running. Here's a repo showing all the full file structure of /Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp: https://github.com/HalfMillennium/elastic-language-server.

This causes the extension to list a SQL server as running but is stuck on "connected" rather than "initialized".

[krassowski]

node ../vscode-extension-samples/elastic-lsp/server/src/server.js indeed sounds like going in the right direction, at least for the development setup (to distribute it properly you would likely want to use bin/npx). Is it running in stdio mode?

If it is in stdio mode, then I would not expect to see any messages when it is run from command line (as those would be interpreted as messages). On related note, be careful not to log anything to standard output as this will break the messaging system (using standard error is fine).

As for debugging on client side, you can got to Advanced Settings Editor → Language Server → enable logAllCommunication (this is misleading, I am migrating to API which enables logging but most endpoints were not migrated yet), and set loggingLevel to "debug".

[HalfMillennium]

Oh alright, so I don't need binaries if I'm debugging and targeting .../server.js directly.

And I was running in Node IPC mode, since that seems to be what VSCode's default code prefers. When I run in stdio mode (command line), I receive no output and the program hangs.

[krassowski]

I just saw that you are using --node-ipc but we do not support TCP mode yet (see https://github.com/krassowski/jupyterlab-lsp/pull/636). Please use --stdio instead.

[krassowski]

What do you mean by "receive no output"? If you mean that it does not print to terminal - yes, this is expected, this is what stdio mode is meant to do (listen and wait for an initialize message). You just need to update argv in the spec to use the node some/path/server.js --stdio approach and it should work just fine, but if it does not please let me now.

[HalfMillennium]

@krassowski Oh okay, I didn't realize that was expected. --stdio seems to be working now!

However, I noticed that autocomplete doesn't work? Not just for my LSP, but I also see that behavior with your fork of pyls. Is this worthy of a new issue or have you seen this bug before?

[krassowski]

However, I noticed that autocomplete doesn't work? Not just for my LSP, but I also see that behavior with your fork of pyls. Is this worthy of a new issue or have you seen this bug before?

What do you mean by autocomplete?

[HalfMillennium]

I mean hinting as you type, automatic completion

[krassowski]

Have you enabled it in settings?

[HalfMillennium]

Would this be the continuousHinting field? I have this set to true in my User preferences.

[krassowski]

Yes. Could you show a screenshot of your preferences?

[HalfMillennium]

Wait, I see now. Apparently there is an issue with how my autocomplete suggestions are delivered, as I see errors in my console when I type.

<LspStdIoReader(parent=<LanguageServerSession(language_server=elastic-lsp, argv=['node', '/Users/garrettc/src/elastic-lsp/vscode-extension-samples/elastic-lsp/server/src/server.js', '--stdio'])>)> couldn't enqueue message: None (no colon in header line)
    Traceback (most recent call last):
      File "/Users/garrettc/miniconda3/envs/jlab-py38/lib/python3.8/site-packages/tornado/httputil.py", line 179, in parse_line
        name, value = line.split(":", 1)
    ValueError: not enough values to unpack (expected 2, got 1)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/garrettc/miniconda3/envs/jlab-py38/lib/python3.8/site-packages/jupyter_lsp/stdio.py", line 87, in read
        message = await self.read_one()
      File "/Users/garrettc/miniconda3/envs/jlab-py38/lib/python3.8/site-packages/jupyter_lsp/stdio.py", line 156, in read_one
        headers.parse_line(line)
      File "/Users/garrettc/miniconda3/envs/jlab-py38/lib/python3.8/site-packages/tornado/httputil.py", line 181, in parse_line
        raise HTTPInputError("no colon in header line")
    tornado.httputil.HTTPInputError: no colon in header line

I doubt this looks familiar to you, but I figured it was worth a shot to ask. Thanks for being so helpful, by the way!