Shared Python LSP Client and Manager

[tjdevries]

@uforic, your sublime plugin got me thinking a lot about how to make the architecture more accessible to multiple editors (and the update got me quite excited about what you guys are planning on doing!), so I spent quite awhile tonight trying to think of maybe a better way to implement parts of the current neovim plugin.

My goal with this new architecture was that it would be very easy for any editor that has python support for plugins would be able to implement the lsp plugin pretty quickly. Also, a lot of the backend stuff would not have to be reimplimented a bunch of times between editors (as it seemed we were doing when I compared the nvim plugin to the sublime plugin).

Here's some example code of what I have been bouncing around in my head for a bit tonight.

In short it goes something like this:

• A LSP client is written in Python that handles startup / initialization; connection; reading, sending, and creating messages; etc.
  • This is pretty much the lsp_client,py that you built in the sublime repo
• A LSP client manager is written in Python that handles the creation of the client, managing multiple clients, etc.
  • Once again, this is pretty much the client manager in the sublime repo, except it now would contain a few items that are in your main.py in the sublime repo
• A Editor plugin that is written in Python with a plugin class that implements several important functions / attributes
  • This would include the request / response functions and utility functions that the manager could call to simplify the code in the editor plugin
  • I'm not sure whether all the requests should just be formed from the editor's helper functions in the client manager, because they should pretty much always be the same. I just don't know how you would alert your editor of the ability to request a function without making it in the editor plugin. This is something I'm still thinking about.

# This is the custom plugin one would write for an editor
# Neovim is an example in this case.
import time
from queue import Empty

import neovim

from LSP import util


def nvim_get_position(nvim):
    return {
        'line': nvim.line,
        'character': nvim.col
    }


@neovim.plugin
class NeovimLSP:
    # Not sure exactly what args you would pass into this yet
    def __init__(self, client, others):
        self.nvim = neovim.attach()
        self.client = client

        # Create a mapping in Vim for the client
        self.nvim.command('nnoremap <leader>lh :LSPHover <args>')

        self.supported_topics = [
            'textDocument/hover',
        ]

    @neovim.command("LSPHover", nargs='*')
    def textDocument_hover_request(self, args):
        callback_args = {'TODO': 'TODO'}
        self.client.hover(
            callback_args,
            'textDocument/hover',
            text_document=self.nvim.command('expand("%")'),
            position=nvim_get_position(self.nvim)
        )

    def get_textDocument_hover_args(self):
        return {'client': 'neovim'}

    def textDocument_hover_callback(self, lsp_message: dict, callback_data: dict):
        self.nvim.command('call langserver#goto#response({0}, {1}, {2})'.format(
            util.uri_to_path(lsp_message['result'][0]['uri']),
            lsp_message['result'][0]['range']['start']['line'],
            lsp_message['result'][0]['range']['start']['character'],
        ))

    # Could also implement common helper functions
    # ... which could be used in the client manager
    def get_file_name(self):
        return self.nvim.command('expand("%")')

    def get_textDocument_hover_args(self):
        pass

    def get_textDocumentPositionParams(self):
        pass

    def get_position(self):
        pass

# Implemented in the shared LSP library for all python editor plugins.

# Here's some of the new code in the Client Manager
class LspClientManager():
    def __init__(self):
        # Initializing code ...
        pass

    def get_plugin(self, plugin_class, **kwargs):
        # Register the plugin, or create it here as well
        return plugin_class(**kwargs)

    # Other code here

    # Register callbacks based on the name, so everyone uses a shared name between plugins.
    def response_handler(self, plugin, response_queue):
        while True:
            try:
                response, callback_args = response_queue.get_nowait()

                method = callback_args['request']['method']
                if method in plugin.supported_topics:
                    # Run the corresponding callback
                    getattr(plugin, method.rplace('/', '_') + '_callback')(response, callback_args)
                else:
                    print(callback_args['request']['method'])

            except Empty:
                pass
            except Exception as err:
                print("Error handling reponse %s with callback args %s, err: %s" %
                      (response, callback_args, err))
                print(err)

        time.sleep(.05)

    # More code here

    # Each item could have a request, that could be defined by having
    # the plugins define common operations.
    def textDocument_hover_request(self, plugin, client):
        try:
            hover_args = plugin.get_textDocument_hover_args()
        except AttributeError:
            hover_args = {}

        client.hover(
            hover_args,
            plugin.get_textDocumentPositionParams(),
            plugin.get_position(),
        )

I'm sure some of this won't make sense, since I'm finishing this up at basically 2 AM my time, but I wanted to get it out there.

Let me know what you think. I can ping more people on slack for them to look at it if you think it's worthwhile. I think this could really help people make plugins quickly for their editor.

[uforic]

Yo!

Couple thoughts:

My goal with this new architecture was that it would be very easy for any editor that has python support for plugins would be able to implement the lsp plugin pretty quickly.

Definitely a good goal - you'll noticed I was borrowing from the good looking lsp client over at https://github.com/tbelaire/lang-server-client . Besides Vim and Sublime, are there other editors that you know of that support Python? I'm sure many editors could, but I suspect given how important LSP is going to be it's going to make sense to do it in Java for IntelliJ, JavaScript for Atom, etc. But no reason NVim and Sublime can't share some code!

Also, for any other projects folks want to do for language servers (not involving editors), could be cool to have a well-written python library to interact with them.

Regarding the long term goal of the Sublime plugin, the "Editor plugin" thing you mention, I'm thinking about how to user a lot of the code in this project: https://github.com/Microsoft/TypeScript-Sublime-Plugin - Sublime has tricky API's, and that code is pretty epic and well done. It kinda sorta already speaks LSP, but differs in a couple key respects. I think wiring up the client and the manager code to it could be the right path forward.

I wonder - is there some similarly epic Vim plugin that is written for one language that we can just wire up to the Python libs we are working on?

Thanks for looking this over!