Advice for Language Servers using parsers written in C++

[gnikit]

Hi All,

We are in the process of using our own Fortran parser present in lfortran, written in C++, as the backend of a new Language Server and I was wondering if you could share some of your expertise or at least provide some comments.
Specifically, I was wondering if you could provide some input with regards to your Language Server implementations. Reading through the implementations in the LSP website I see that your C++ LSs are actually coded in C++. I assume that means that your parser/tokenizer is written in C++ and therefore your AST/ASR representation is also in C++. If that is true, I am curious to know if you have implemented a JSONRPC interface in C++ and then send these requests to your TypeScript client or if you have implemented some sort of interface between C++ and Typescript, like nbind that is capable of translating your C++ AST/ASR to Typescript to then be converted into LS requests via the vscode-languageserver/node module.

Basically, implementing the LSP interface in C++, even if that is only for the server, would be a nightmare to maintain and currently there is not enough manpower to support this. So we are looking for better solutions to communicate our AST/ASR objects into Typescript. nbind, looked like a good solution but the project appears to be dead. Do you have any recommendations with regards to this?

I know that your C++ LSs are proprietary and you are limited to what you are allowed to share. If it would be preferable we can continue this conversation in private.

Thanks in advance for any and all help!

[sean-mcmanus]

TypeScript to LSP server communication could be done with any JSON serialization library and doesn't require RPCs. Our AST isn't sent to TypeScript -- if that's what you're trying to do then we don't have any advice on that. Our LSP server just reads the JSON messages and returns JSON results. The LSP is versioned so you can stay on an older version unless you need a newer one for a particular feature so I'm not sure what maintenance nightmare you're referring to.

[sean-mcmanus]

Yeah, you are correct -- our cpptools has to parse/serialize the JSON it receives using some JSON library -- see https://github.com/microsoft/vscode-cpptools/blob/main/Extension/ThirdPartyNotices.txt#L2034 . We haven't found it to be a maintenance issue so we haven't investigated other approaches to potentially auto-generate the C++ code.

[gnikit]

Thanks @sean-mcmanus for all your time answering my questions, as always you've been extremely helpful!

[gnikit]

Out of curiosity, on the C++ side I suspect you keep the process running and communicate via IO buffer/network port the incoming/outgoing JSON to Typescript. You don't spin up a new process every time you request something from the server, right?

[sean-mcmanus]

Yes -- you can see that via installing our C/C++ extension and looking at the cpptools process in a task manager. The only time cpptools would get restarted is if it crashes.

[certik]

Thanks @sean-mcmanus for the clarifications. This has been extremely helpful and valuable for us to understand how to design our language server.