Skip to content

Latest commit

 

History

History
73 lines (54 loc) · 2.54 KB

local-dev.md

File metadata and controls

73 lines (54 loc) · 2.54 KB

Local development of kernels

Introduction

kernel-builder builds kernels in a sandbox. This has various benefits, such as building kernels for a wide range of Torch versions, compatibility with older C library versions and avoiding accidental dependencies.

However, this is not ideal during kernel development, since language servers and IDEs do not interpret the build.toml file. As a result, code completion will typically not work. kernel-builder provides the build2cmake utility to generate CMake files to build native code and setuptools files for building the kernel as a regular Python package. Since CMake and setuptools are widely supported by IDEs, this provides a much-improved development experience.

Installing build2cmake

build2cmake is available as a Rust crate. After installing Rust, it can be built and installed as follows:

$ cargo install build2cmake

Generating a Python project with build2cmake

build2cmake generates a CMake/Python project from a build.toml file. The invocation is as follows:

$ build2cmake generate build.toml -f

The -f flag is optional and instructs build2cmake to overwrite existing files.

It is recommended to do an editable install of the generated project into your Python virtual environment for development:

$ pip install wheel # Needed once to enable bdist_wheel.
$ pip install --no-build-isolation -e .

Warnings:

  • Kernels built in this way should not be published on the Kernel Hub. They do not fulfill the kernel requirements.
  • Do not add the generated files to Git. build2cmake has regular updates and you generally want to use files generated by the latest version.

Testing kernel builds before publishing

Once you have built a kernel with kernel builder, you may want to test it locally with software that uses get_kernel or LayerRepository before publishing. This can be done using the LOCAL_KERNELS variable, which maps a repository ID to a local kernel directory. For example, you could use the kernel in devel/activation for any use of the kernels-community/activation repository with:

$ LOCAL_KERNELS="kernels-community/activation=devel/activation" \
  python my_app.py

It is also possible to map multiple repositories to local kernel directories by separating the entries with a colon (:):

$ LOCAL_KERNELS="kernels-community/activation=devel/activation:kernels-community/flash-attn2=devel/flash-attn2" \
  python my_app.py