Merge pull request #539 from krassowski/documentation-language-servers

Improve language servers configuration documentation

Author: krassowski

CONTRIBUTING.md

@@ -322,44 +322,57 @@ python scripts/lint.py
 
 ### Specs
 
-It is convenient to collect common patterns for connecting to installed language
-servers as `pip`-installable packages that Just Work with `jupyter-lsp`.
+While language servers can be configured by the user using a simple JSON or Python [configuration file](./Configuring.ipynb#language_servers),
+it is preferable to provide users with an option that does not require manual configuration. The language server specifications (specs)
+wrap the configuration (as would be defined by the user) into a Python class or function that can be either:
 
-If an advanced user installs, locates, and configures, their own language
-server it will always win vs an auto-configured one.
+- distributed using PyPI/conda-forge and made conveniently available to users for `pip install` and/or `conda install`
+- contributed to the collection of built-in specs of jupyter-lsp by opening a PR (preferable for popular language servers, say >100 users)
 
-#### Writing a spec
+In either case the detection of available specifications uses Python `entry_points` (see the `[options.entry_points]` section in jupyter-lsp [setup.cfg]).
 
-> See the built-in [specs][] for implementations and some [helpers][].
+> If an advanced user installs, locates, and configures, their own language server it will always win vs an auto-configured one.
 
-[specs]: https://github.com/krassowski/jupyterlab-lsp/tree/master/python_packages/jupyter_lsp/jupyter_lsp/specs
-[helpers]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/specs/utils.py
+#### Writing a spec
 
-A spec is a python function that accepts a single argument, the
-`LanguageServerManager`, and returns a dictionary of the form:
+A spec is a Python callable (a function, or a class with `__call__` method) that accepts a single argument, the
+`LanguageServerManager` instance, and returns a dictionary of the form:
 
 ```python
 {
   "python-language-server": {            # the name of the implementation
-      "version": 1,                      # the version of the spec schema
+      "version":  SPEC_VERSION,          # the version of the spec schema (an integer)
       "argv": ["python", "-m", "pyls"],  # a list of command line arguments
-      "languages": ["python"]            # a list of languages it supports
+      "languages": ["python"],           # a list of languages it supports
+      "mime_types": ["text/python", "text/x-ipython"]
   }
 }
 ```
 
-The absolute minimum listing requires `argv` (a list of shell tokens to launch
-the server) and `languages` (which languages to respond to), but many number of
-other options to enrich the user experience are available in the
-[schema][] and are exercised by the current `entry_points`-based [specs][].
+The above example is only intended as an illustration and not as an up-to-date guide.
+For details on the dictionary contents, see the [schema][] definition and [built-in specs][].
+Basic concepts (meaning of the `argv` and `languages` arguments) are also explained in the [configuration files](./Configuring.ipynb#language_servers) documentation.
 
-[schema]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/schema/schema.json
+When contributing a specification we recommend to make use of the helper classes and other [utilities][] that take care of the common use-cases:
+
+- `ShellSpec` helps to create specs for servers that can be started from command-line
+- `PythonModuleSpec` is useful for servers which are Python modules
+- `NodeModuleSpec` will take care of finding Node.js modules
+
+See the built-in [built-in specs][] for example implementations.
 
 The spec should only be advertised if the command _could actually_ be run:
 
-- its runtime (e.g. `julia`, `nodejs`, `python`, `r`, `ruby`) is installed
+- its runtime/interpreter (e.g. `julia`, `nodejs`, `python`, `r`, `ruby`) is installed
 - the language server itself is installed (e.g. `python-language-server`)
 
+otherwise an empty dictionary (`{}`) should be returned.
+
+[built-in specs]: https://github.com/krassowski/jupyterlab-lsp/tree/master/python_packages/jupyter_lsp/jupyter_lsp/specs
+[setup.cfg]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/setup.cfg
+[schema]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/schema/schema.json
+[utilities]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/specs/utils.py
+
 ##### Common Concerns
 
 - some language servers need to have their connection mode specified
@@ -369,10 +382,11 @@ The spec should only be advertised if the command _could actually_ be run:
   - `LanguageServerManager.nodejs` will provide the location of our best
     guess at where a user's `nodejs` might be found
 - some language servers are hard to start purely from the command line
-  - use a helper script to encapsulate some complexity.
-    - See the [r spec][] for an example
+  - use a helper script to encapsulate some complexity, or
+  - use a command argument of the interpreter is available (see the [r spec][] and [julia spec] for examples)
 
 [r spec]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/specs/r_languageserver.py
+[julia spec]: https://github.com/krassowski/jupyterlab-lsp/blob/master/python_packages/jupyter_lsp/jupyter_lsp/specs/julia_language_server.py
 
 ##### Example: making a pip-installable `cool-language-server` spec
 
@@ -402,7 +416,8 @@ def cool(app):
         "cool-language-server": {
             "version": 1,
             "argv": [cool_language_server],
-            "languages": ["cool"]
+            "languages": ["cool"],
+            "mime_types": ["text/cool", "text/x-cool"]
         }
     }
 ```

docs/CHANGELOG.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## CHANGELOG"
+    "## Changelog"
    ]
   },
   {

docs/Configuring.ipynb

@@ -25,7 +25,7 @@
     "```\n",
     "\n",
     "They will be merged from bottom to top, and the directory where you launch your\n",
-    "`notebook` server wins, making it easy to check in to version control."
+    "`notebook` or `lab` server wins, making it easy to check in to version control."
    ]
   },
   {
@@ -66,8 +66,15 @@
     "    `mkdir \"new directory\"` should be split into `[\"mkdir\", \"new directory\"]`;\n",
     "    If you have Python installed, you can use `shlex.split(\"your command\")` to\n",
     "    get such an array.\n",
-    "- the `languages` which the server will respond to, and\n",
+    "- the `languages` which the language server will respond to, and\n",
     "- the schema `version` of the spec (currently `2`)\n",
+    "- `mime_types` by which the notebooks and files will be matched to the langauge\n",
+    "  server:\n",
+    "  - for notebooks the MIME type is derived from `language_info`/`mimetype`\n",
+    "    element of [kernel_info][] response (with fallback on to cell metadata if\n",
+    "    missing from kernel response)\n",
+    "  - for files the implemntation is frontend-specific; in JupyterLab the MIME\n",
+    "    type is obtained from the MIME type registry.\n",
     "\n",
     "```python\n",
     "# ./jupyter_server_config.json                   ---------- unique! -----------\n",
@@ -80,12 +87,16 @@
     "      \"a-language-server-implementation\": {\n",
     "        \"version\": 2,\n",
     "        \"argv\": [\"/absolute/path/to/a-language-server\", \"--stdio\"],\n",
-    "        \"languages\": [\"a-language\"]\n",
+    "        \"languages\": [\"a-language\"],\n",
+    "        \"mime_types\": [\"text/language\", \"text/x-language\"]\n",
     "      }\n",
     "    }\n",
     "  }\n",
     "}\n",
-    "```"
+    "```\n",
+    "\n",
+    "[kernel_info]:\n",
+    "  https://jupyter-client.readthedocs.io/en/stable/messaging.html#kernel-info"
    ]
   },
   {
@@ -111,13 +122,15 @@
     "        # if installed as a binary\n",
     "        \"argv\": [shutil.which(\"a-language-server\")],\n",
     "        \"languages\": [\"a-language\"],\n",
-    "        \"version\": 2\n",
+    "        \"version\": 2,\n",
+    "        \"mime_types\": [\"text/a-language\"]\n",
     "    },\n",
     "    \"another-language-implementation\": {\n",
     "        # if run like a script\n",
     "        \"argv\": [shutil.which(\"another-language-interpreter\"), \"another-language-server\"],\n",
     "        \"languages\": [\"another-language\"],\n",
-    "        \"version\": 2\n",
+    "        \"version\": 2,\n",
+    "        \"mime_types\": [\"text/another-language\"]\n",
     "    }\n",
     "}\n",
     "```"
@@ -144,7 +157,7 @@
     "> default: `True`\n",
     "\n",
     "If `True`, `jupyter-lsp` will look for all\n",
-    "[known language servers](#installing-language-servers). User-configured\n",
+    "[known language servers](./Language%20Servers.ipynb). User-configured\n",
     "`language_servers` of the same implementation will be preferred over\n",
     "`autodetect`ed ones."
    ]
@@ -199,24 +212,21 @@
    "source": [
     "### Python `entry_points`\n",
     "\n",
-    "`pip`-installable packages in the same environment as the Jupyter `notebook`\n",
-    "server can be automatically detected as providing\n",
-    "[language_servers](#language_servers). These are a little more involved, but\n",
-    "also more powerful: see more in [Contributing](Contributing.ipynb#Specs).\n",
-    "Servers configured this way are loaded _before_ those defined in\n",
-    "[configuration files](#Configuration-Files), so that a user can fine-tune their\n",
-    "available servers."
+    "`pip`-installable packages in the same environment as the Jupyter server can be\n",
+    "automatically detected as providing [language_servers](#language_servers). These\n",
+    "are a little more involved, but also more powerful: see more in\n",
+    "[Contributing](Contributing.ipynb#Specs). Servers configured this way are loaded\n",
+    "_before_ those defined in [configuration files](#Configuration-Files), so that a\n",
+    "user can fine-tune their available servers."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "source": [
-    "### Example: Scala Language Server (metals) integration with jupyterlab-lsp\n",
+    "### Example: Scala Language Server (metals) integration\n",
     "\n",
-    "Step 1: Get a Scala-based kernel installed.\n",
+    "**Step 1:** Get a Scala-based kernel installed.\n",
     "\n",
     "2 possible options: Almond kernel or the Spark magic kernel.\n",
     "\n",
@@ -241,7 +251,7 @@
     "jupyter-kernelspec install sparkmagic/kernels/sparkkernel\n",
     "```\n",
     "\n",
-    "Step 2: Install metals server in the working directory:\n",
+    "**Step 2:** Install metals server in the working directory:\n",
     "\n",
     "Metals has a coursier based installation.\n",
     "\n",
@@ -250,10 +260,12 @@
     "./coursier bootstrap org.scalameta:metals_2.12:0.7.0 --force-fetch -o metals -f\n",
     "```\n",
     "\n",
-    "(Might need to use the --force-fetch flag if you are getting dependency issues.)\n",
+    "(Might need to use the `--force-fetch` flag if you are getting dependency\n",
+    "issues.)\n",
     "\n",
-    "Step 3: Configure the metals server in jupyterlab-lsp. Enter the following in\n",
-    "the jupyter_server_config.json:\n",
+    "**Step 3:** Configure the metals server in jupyterlab-lsp. Enter the following\n",
+    "in the `jupyter/jupyter_server_config.d/metals-ls.json` (in one of the jupyter\n",
+    "configuration directories):\n",
     "\n",
     "```python\n",
     "{\n",
@@ -271,8 +283,8 @@
     "```\n",
     "\n",
     "You are good to go now! Just start `jupyter lab` and create a notebook with\n",
-    "either the Spark or the Scala kernel and you should be able to see the metals\n",
-    "server initialised from the bottom left corner."
+    "either the Spark or the Scala kernel and the metals server should automatically\n",
+    "initialise (the status indicator should show \"Fully initialized\")."
    ]
   }
  ],

docs/Language Servers.ipynb

@@ -6,22 +6,28 @@
    "source": [
     "## Language Servers\n",
     "\n",
-    "`jupyter-lsp` does not come with any Language Servers! However, we will try to\n",
-    "use them if they _are_ installed and we know about them. For the language\n",
-    "servers in the tables below, use one of the suggested package managers to\n",
-    "install them: these implementations are tested to work with `jupyter-lsp`.\n",
+    "By default `jupyter-lsp` does not come with any language servers preinstalled.\n",
+    "However, we will try to use them if they _are_ installed and we know about them\n",
+    "(i.e. someone contributed a full specification).\n",
     "\n",
-    "- _You can disable this feature by configuring_\n",
-    "  [autodetect](./Configuring.ipynb#autodetect)\n",
+    "> You can disable auto-detection by configuring\n",
+    "> [autodetect](./Configuring.ipynb#autodetect)\n",
     "\n",
-    "If you do not see a language you would like, but can find it one of these lists:\n",
+    "You can add another language server for languages that are not listed on this\n",
+    "page:\n",
     "\n",
-    "- the [official list][lsp-implementations] of language servers\n",
-    "- a [community-curated list][langserver] of language servers\n",
+    "- using a minimal JSON or Python\n",
+    "  [configuration file](./Configuring.ipynb#language_servers) (good for\n",
+    "  experimenting or configuring a niche server), or\n",
+    "- contributing a [full specification](./Contributing.ipynb#Specs) (to enable\n",
+    "  better integration and help other users of the same language)\n",
     "\n",
-    "...you might be able to add it\n",
-    "[via configuration](./Configuring.ipynb#language_servers) or\n",
-    "[build your own spec](./Contributing.ipynb#Specs) for the server in question.\n",
+    "The existing language servers are listed on the [official\n",
+    "list][lsp-implementations] and on the [community-curated list][langserver].\n",
+    "\n",
+    "For the language servers in the tables below, use one of the suggested package\n",
+    "managers to install them: these implementations are tested to work with\n",
+    "`jupyter-lsp`.\n",
     "\n",
     "[language-server]:\n",
     "  https://microsoft.github.io/language-server-protocol/specification\n",
@@ -141,6 +147,15 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The Scala language server (`metals`) is not currently auto-detected, but can be\n",
+    "configured as demonstrated in the\n",
+    "[configuration example](<./Configuring.ipynb#Example:-Scala-Language-Server-(metals)-integration>)."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

docs/Roadmap.ipynb

@@ -18,12 +18,12 @@
     "### Front End\n",
     "\n",
     "- improved code navigation when there are multiple jump targets\n",
-    "- autocompleter sorting based on LSP suggestions\n",
     "- system of settings, including options:\n",
     "  - to change the verbosity of signature documentation hints (number of lines to\n",
     "    be shown)\n",
     "  - custom foreign extractors allowing to customize behaviour for magics\n",
     "- code actions (allowing to \"quick fix\" a typo, etc.)\n",
+    "- code formatting\n",
     "- gutter with linter results\n",
     "- use the kernel session for autocompletion in FileEditor if available (PR\n",
     "  welcome)"
@@ -35,7 +35,6 @@
    "source": [
     "### Backend\n",
     "\n",
-    "- release on `conda`\n",
     "- [#49](https://github.com/krassowski/jupyterlab-lsp/issues/49) cookiecutter for\n",
     "  pip-installable specs\n",
     "- add hook system to allow serverextensions/kernels to modify, inspect and react\n",

docs/_static/css/custom.css

@@ -32,7 +32,13 @@ body div.nboutput.container div[class*='highlight'] pre {
 }
 
 .wy-nav-content {
-  max-width: 100%;
+  /* Increase the width of the content from the default 800px;
+  not using 100% as it makes reading sentences unbearable on 4K screens */
+  max-width: 1100px;
+}
+
+.wy-nav-content p {
+  max-width: 950px;
 }
 
 .nbinput .prompt,

docs/index.ipynb

@@ -56,7 +56,7 @@
    "source": [
     "## Project Information\n",
     "\n",
-    "- [CHANGELOG](./CHANGELOG.ipynb)\n",
+    "- [Changelog](./CHANGELOG.ipynb)\n",
     "- [Roadmap](./Roadmap.ipynb)\n",
     "- [Architecture](Architecture.ipynb)"
    ]