Jedi fails to get completions when Rope/Jedi race is active

[ccordoba12]

@goanpeca, to activate the race between Rope and Jedi you need to

1. Uninstall python-language-server.
2. Install this repo in develop mode.
3. Go to Preferences > Language Server Protocol in Spyder, then double-click on the Python entry, then change the value rope_completion > enabled to true (that's very cumbersome, I know, but we'll improve it in PR: Provide graphical options to configure the PyLS spyder#8647).

After that, you need to get a completion where Rope fails, e.g. import pand. If the race would be working correctly, this should be completed by Jedi. But instead I'm getting the following error in server_python.log:

Traceback (most recent call last):
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/pyls_jsonrpc/endpoint.py", line 113, in consume
    self._handle_request(message['id'], message['method'], message.get('params'))
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/pyls_jsonrpc/endpoint.py", line 182, in _handle_request
    handler_result = handler(params)
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/pyls_jsonrpc/dispatchers.py", line 23, in handler
    return method(**(params or {}))
  File "/home/carlos/Projects/spyder/spyder-pyls/pyls/python_ls.py", line 284, in m_text_document__completion
    return self.completions(textDocument['uri'], position)
  File "/home/carlos/Projects/spyder/spyder-pyls/pyls/python_ls.py", line 205, in completions
    workspace=self.workspace
  File "/home/carlos/Projects/spyder/spyder-pyls/pyls/_utils.py", line 185, in race_hooks
    first_impl, result = next(pool.imap_unordered(_apply, impls, chunksize=1))
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/multiprocessing/pool.py", line 735, in next
    raise value
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/carlos/Projects/spyder/spyder-pyls/pyls/_utils.py", line 178, in _apply
    return impl, impl.function(**kwargs)
  File "/home/carlos/Projects/spyder/spyder-pyls/pyls/plugins/jedi_completion.py", line 49, in pyls_completions
    definitions = document.jedi_script(position).completions()
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/api/__init__.py", line 179, in completions
    completions = completion.completions()
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/api/completion.py", line 98, in completions
    completion_names = self._get_context_completions()
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/api/completion.py", line 194, in _get_context_completions
    only_modules=only_modules,
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/api/completion.py", line 266, in _get_importer_names
    return i.completion_names(self._evaluator, only_modules=only_modules)
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/evaluate/imports.py", line 469, in completion_names
    names += self._get_module_names()
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/evaluate/imports.py", line 413, in _get_module_names
    for name in sub.list_module_names(search_path):
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 108, in wrapper
    kwargs=kwargs,
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 199, in run
    return self._send(id(evaluator), function, args, kwargs)
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 242, in _send
    is_exception, traceback, result = pickle_load(self._process.stdout)
  File "/home/carlos/miniconda/envs/py36/lib/python3.6/site-packages/jedi/_compatibility.py", line 469, in pickle_load
    return pickle.load(file, encoding='bytes')
_pickle.UnpicklingError: invalid load key, 'y'.

[goanpeca]

Will look into it :-)

[ccordoba12]

The race is implemented here:

https://github.com/spyder-ide/spyder-pyls/blob/develop/pyls/_utils.py#L166

Since that uses multiprocessing, my guess is that Jedi can't be run through it.

[goanpeca]

It seems unpickling stack underflow can happen when a pickle ends unexpectedly. so what is a case where rope is faster than jedi? to test something

[ccordoba12]

Rope completes things correctly. For example:

import pandas as pd
pd.

[ccordoba12]

But that means that Rope is the only one working.

[orlandini]

I am facing a similar issue using pyls on emacs via lsp-mode, so I am interested in it. By any chance, are there any updates?