Editable install doesn't provide/link `datadir`

[tobiasdiez]

We currently install the Jupyter kernel spec via

kernel_dir = get_option('datadir') / 'share/jupyter/kernels/sagemath'
install_data('ext_data/notebook-ipython/logo.svg', install_dir: kernel_dir)
install_data('ext_data/notebook-ipython/logo-64x64.png', install_dir: kernel_dir)
kernel_data = configuration_data()
kernel_data.set('SAGE_VERSION', meson.project_version())
configure_file(
  input: 'ext_data/notebook-ipython/kernel.json.in',
  output: 'kernel.json',
  install_dir: kernel_dir,
  install: true,
  configuration: kernel_data,
)

This works fine in a "normal" installation. But if you use an editable install, the kernelspec is not created in the python prefix and thus Jupyter cannot find the kernel.

What's the correct way to "install" such data files in editable mode?

[rgommers]

The usual fix for data files is to access them through importlib.resources rather than through a fixed path (probably constructed with __file__?). See for example scipy/scipy#19710.

[tobiasdiez]

Thanks, this works if you are in control of the code that reads the data file. However, in our case, we are trying to register a jupyter kernel and this really just queries the file in a predefined folder (https://github.com/jupyter/jupyter_client/blob/1f36fbf4d0e2e067129552d9e69212f7bf7c0624/jupyter_client/kernelspec.py#L43).

Our current workaround is to call jupyter kernelspec install with the meson build directory as argument (where the kernel.json is written to). Seems to work, but is a bit suboptimal.

[rgommers]

Ah yes, that's not a great interface. I think you're doing the right thing now, passing in the build directory seems preferable to adding a hack that installs kernel.json in-tree for an editable install.

[tobiasdiez]

Thanks!

Is there a way to detect in meson if a project is build in editable mode? Then one could call the jupyter command directly from meson.

[rgommers]

There is no easy/supported way to know whether an install is editable. In NumPy we need to know at install time for running some compile tests, and do it like this:

https://github.com/numpy/numpy/blob/82bd6ef31ce57859280be7591c3a20f562e69d34/numpy/testing/_private/utils.py#L59-L78

If you'd want to know during the build, you could derive it from the absolute path to the build directory, since that directory is different for editable builds:

meson-python/mesonpy/__init__.py

Line 1239 in 121339e

config_settings['build-dir'] = 'build/' + mesonpy._tags.get_abi_tag()

[tobiasdiez]

Thanks! If I'll find the time, I will look into providing the editable mode as a build option to meson.

[eli-schwartz]

Thanks! If I'll find the time, I will look into providing the editable mode as a build option to meson.

That might also require being able to configure in pyproject.toml, an additional set of setup-args which get passed specifically in editable mode.

[dnicolodi]

The fact that data files are not accessible at the install location for editable wheels is documented and it is an intrinsic property of editable installs. Is there anything actionable is this issue?