|
| 1 | +The BOO (Bag of Ops) project is intended to provide a python interface for launching individual IREE kernels within a pytorch model. |
| 2 | + |
| 3 | +Currently, forward and backward convolution kernels are supported. |
| 4 | + |
| 5 | +## Environment Variables |
| 6 | + |
| 7 | +There are a few environment variables that control BOO behavior: |
| 8 | + |
| 9 | +- `BOO_CACHE_ON=<0 or 1>`: whether to cache kernel artifacts for boo kernels (default = 1). |
| 10 | +- `BOO_CACHE_DIR=<path to cache dir>`: where to store kernel artifacts (default = `~/.cache/turbine_kernels/boo/`). |
| 11 | +- `BOO_USE_BACKWARD_KERNELS=<0 or 1>`: whether to use our backward kernels (default = 0). |
| 12 | + |
| 13 | +## Usage: |
| 14 | + |
| 15 | +### Launch a single convolution: |
| 16 | + |
| 17 | +```python |
| 18 | +import torch |
| 19 | +from iree.turbine.kernel.boo.ops import boo_conv |
| 20 | + |
| 21 | +input_shape = ... |
| 22 | +weight_shape = ... |
| 23 | +dtype = ... |
| 24 | + |
| 25 | +x = torch.randn(input_shape, dtype=dtype, device="cuda:0") |
| 26 | +w = torch.randn(weight_shape, dtype=dtype, device="cuda:0") |
| 27 | +b = None # optionally fuse with a bias-add |
| 28 | + |
| 29 | +y = boo_conv(x,w,b) # can also specify stride, dilation, padding, groups, and layouts (e.g., "NHWC") |
| 30 | +``` |
| 31 | + |
| 32 | +### Replace `Conv2d` in a model with `BooConv2d`: |
| 33 | + |
| 34 | +For a `resnet_18` boo convolution example with sample training, see [`examples/resnet_18_backward.py`](examples/resnet_18_backward.py). |
| 35 | + |
| 36 | +```python |
| 37 | +from iree.turbine.kernel.boo.modeling import replace_conv2d_with_boo_conv |
| 38 | + |
| 39 | +model = ... # torch.nn.Module |
| 40 | + |
| 41 | +replacement_kwargs = {"stride" : (1,1)} # controls which types of Conv2d are replaced. |
| 42 | +model = replace_conv2d_with_boo_conv(model, **replacement_kwargs) |
| 43 | +outputs = model(...) |
| 44 | +``` |
| 45 | + |
| 46 | +### Use a `BooConv2d` module directly: |
| 47 | + |
| 48 | +```python |
| 49 | +from iree.turbine.kernel.boo.modeling import BooConv2d |
| 50 | + |
| 51 | +conv2d = BooConv2d(...) # usual Conv2d args |
| 52 | +``` |
0 commit comments