|
1 | 1 | <div align=center> |
2 | | -<img src="images/herbie-tires.png" width=250> |
| 2 | +<img src="https://raw.githubusercontent.com/blaylockbk/herbie-plugin-tutorial/refs/heads/main/images/herbie-tires.png" width=250> |
3 | 3 |
|
4 | 4 | # Herbie Plugin Tutorial |
5 | 5 |
|
6 | 6 | </div> |
7 | 7 |
|
8 | | -This is a demonstration of writing a plugin for [Herbie](https://github.com/blaylockbk/Herbie) to add custom model templates--like giving Herbie a new set of tires. |
| 8 | +This tutorial shows how to write a plugin for [Herbie](https://github.com/blaylockbk/Herbie) to add custom model templates—like giving Herbie a new set of tires. |
9 | 9 |
|
10 | | -I’d love for you to contribute your model template to the main Herbie repository—but sometimes you might need your own: |
| 10 | +You might need your own model template when: |
11 | 11 |
|
12 | | -- You have local GRIB2 files you want to access using Herbie (e.g., a WRF or MPAS simulation). |
13 | | -- You need to handle an existing model a little differently. |
14 | | -- You have access to GRIB2 model data on a private network. |
15 | | -- You want to test a new or updated model template before contributing upstream. |
16 | | -- Other reasons? Let me know! |
| 12 | +- You have local GRIB2 files (e.g., WRF/MPAS output) you'd like to access with Herbie. |
| 13 | +- You have access to GRIB2 data on a private network. |
| 14 | +- You want to override behavior of an existing model temple. |
| 15 | +- You want to iterate on a new model template before contributing upstream. |
17 | 16 |
|
18 | | -## How it works |
| 17 | +## What is a Herbie model template? |
19 | 18 |
|
20 | | -Herbie plugins let you add custom model templates that Herbie can discover when imported. |
| 19 | +A _model template_ in Herbie is a Python class that defines where Herbie looks for weather model datasets. Herbie comes with a bunch of [model templates](https://github.com/blaylockbk/Herbie/tree/main/src/herbie/models) you can look at for reference. When you import Herbie, it loads its model templates, and then Herbie looks for any templates from installed plugins. |
21 | 20 |
|
22 | | -For example, after installing Herbie and this plugin tutorial: |
| 21 | +## Project structure |
23 | 22 |
|
24 | | -```bash |
25 | | -pip install herbie-data |
| 23 | +Here's what your plugin project should look like: |
26 | 24 |
|
27 | | -git clone https://github.com/blaylockbk/herbie-plugin-tutorial.git |
28 | | -cd herbie-plugin-tutorial |
29 | | -pip install -e . |
| 25 | +``` |
| 26 | +herbie-plugin-tutorial/ |
| 27 | +├── pyproject.toml |
| 28 | +└── src/ |
| 29 | + └── herbie_plugin_tutorial/ |
| 30 | + └── __init__.py # contains your model templates |
30 | 31 | ``` |
31 | 32 |
|
32 | | -Herbie can now use the custom templates defined in this .`herbie-plugin-tutorial` plugin. |
33 | | - |
34 | | -## Creating a plugin |
| 33 | +## Create the plugin project |
35 | 34 |
|
36 | | -I created this `herbie-data-tutorial` repository using [uv](https://docs.astral.sh/uv/) and specified Python 3.10 (the minimum required by Herbie): |
| 35 | +I used [uv](https://docs.astral.sh/uv/) to create this example plugin: |
37 | 36 |
|
38 | 37 | ```bash |
39 | 38 | uv init --lib herbie-plugin-tutorial --python 3.10 |
40 | | -``` |
41 | | - |
42 | | -Then I added `herbie-data` as a dependency (because the plugin without Herbie would not be very useful). |
43 | | - |
44 | | -```bash |
| 39 | +cd herbie-plugin-tutorial |
45 | 40 | uv add herbie-data |
46 | 41 | ``` |
47 | 42 |
|
48 | | -To register this package as a Herbie plugin, add the following endpoints to your `pyproject.toml`. The key (e.g., `herbie_plugin_demo`) should match your plugin's name. |
| 43 | +- I set `--python 3.10` because that is Herbie's minimum version |
| 44 | +- Add `herbie-data` as a dependency, because what good is a plugin without the main package. |
| 45 | + |
| 46 | +To register your plugin with Herbie, add the following to your `pyproject.toml`: |
49 | 47 |
|
50 | 48 | ```toml |
51 | 49 | [project.entry-points."herbie.plugins"] |
52 | | -herbie_plugin_demo = "herbie_plugin_demo" |
| 50 | +hrrr_analysis = "herbie_plugin_tutorial:hrrr_analysis" |
| 51 | +bmw = "herbie_plugin_tutorial:bmw" |
53 | 52 | ``` |
54 | 53 |
|
55 | 54 | ### Making a custom model template |
56 | 55 |
|
57 | | -Your model templates live in your plugin’s __init__.py file. |
| 56 | +Your model templates live in your plugin’s `__init__.py` file. |
58 | 57 |
|
59 | 58 | Model templates are a bit of a craft due to some historical quirks, a few odd conventions (Herbie's grown over time!), and support for a lot of edge cases. Still, the basic structure is simple. |
60 | 59 |
|
61 | | -Take a look at `herbie-plugin-tutorial/src/herbie_plugin_tutorial/__init__.py`, which includes two example templates: |
| 60 | +Take a look at this plugin's [`__init__.py`](https://github.com/blaylockbk/herbie-plugin-tutorial/blob/main/src/herbie_plugin_tutorial/__init__.py), which includes two example templates: |
62 | 61 |
|
63 | | -1. `hrrr_analysis` — A custom version of the HRRR model that only finds analysis fields from AWS. |
64 | | -2. `bmw` — Local GRIB2 files output from a fictional dataset: _BMW_ "Brian's Model of Weather" |
| 62 | +1. `hrrr_analysis` — A custom template for the HRRR model that only locates analysis fields from AWS. |
| 63 | +2. `bmw` — Local GRIB2 files output from a fictional dataset _BMW_ "Brian's Model of Weather" |
65 | 64 |
|
| 65 | +> [!TIP] |
| 66 | +> |
| 67 | +> 1. Class names must be lowercase! Herbie lowercases the `model=` input, so `model='HRRR'` becomes `model='hrrr'`. |
| 68 | +> |
| 69 | +> 1. Set `self.DESCRIPTION` and `self.DETAILS` for helpful metadata. |
| 70 | +> |
| 71 | +> 1. `self.PRODUCTS` must have at least one entry. If the user doesn't provide a `product=`, Herbie uses the first one by default. |
| 72 | +> |
| 73 | +> 1. `self.SOURCES` is a dictionary of key-value pairs. Herbie will try each one in order until it finds a valid file. Prefix a key with `local` if the file is on disk instead of on a remote server. |
| 74 | +> |
| 75 | +> 1. `self.LOCALFILE` is how you specify what the file name will be when it is downloaded. Setting to `f"{self.get_remoteFileName}"` simply says to keep the original name of the file. |
| 76 | +> |
| 77 | +> Look at [existing model templates](https://github.com/blaylockbk/Herbie/tree/main/herbie/models) for more examples. |
66 | 78 |
|
67 | | -### Tips for writing a template |
68 | 79 |
|
69 | | -1. The class name must be lowercase! Herbie lowercases the `model=` input, so `model='HRRR'` becomes `model='hrrr'`. |
| 80 | +## Use your Herbie plugin |
70 | 81 |
|
71 | | -1. Set `self.DESCRIPTION` and `self.DETAILS` for helpful metadata. |
| 82 | +Install the plugin `pip install -e .` in your environment to use your custom templates. |
72 | 83 |
|
73 | | -1. `self.PRODUCTS` must have at least one entry. If the user doesn't provide a `product=`, Herbie uses the first one by default. This value is stored as `self.product`. |
74 | 84 |
|
75 | | -1. `self.SOURCES` is a dictionary of key-value pairs. Herbie will try each one in order until it finds a valid file. Prefix a key with `local` if the file is on disk instead of on a remote server. |
| 85 | +Let's walk through using this plugin in a new project using uv. |
76 | 86 |
|
77 | | -Look at [existing model templates](https://github.com/blaylockbk/Herbie/tree/main/herbie/models) for more examples. |
| 87 | +```bash |
| 88 | +uv init new_project |
| 89 | +cd new_project |
| 90 | +uv add herbie-data --extra extras |
| 91 | +uv add --editable ../herbie-plugin-tutorial |
| 92 | +``` |
78 | 93 |
|
79 | | -## Install your plugin |
| 94 | +- I used `--editable` so I could debug the plugin if needed. |
80 | 95 |
|
81 | | -From the root of your new plugin (where `pyproject.toml` lives) install your package with: |
| 96 | +Then launch Python |
82 | 97 |
|
83 | 98 | ```bash |
84 | | -pip install -e . |
| 99 | +uv run python |
85 | 100 | ``` |
86 | 101 |
|
87 | | -## Use your custom model in Herbie |
88 | | - |
89 | | -Once installed, Herbie will automatically detect and register your custom templates: |
| 102 | +Importing Herbie automatically registers your templates. |
90 | 103 |
|
91 | 104 | ```python |
92 | 105 | from herbie import Herbie |
93 | 106 | ``` |
94 | 107 |
|
95 | | -You’ll see something like this in your console: |
| 108 | +You should see output like this when a plugin loads: |
96 | 109 |
|
97 | | -``` |
98 | | -Herbie: model template 'hrrr_analysis' from custom plugin was added to globals. |
99 | | -Herbie: model template 'bmw' from custom plugin was added to globals. |
100 | | -``` |
| 110 | +> ``` |
| 111 | +> Herbie: Added model 'bmw' from herbie-plugin-tutorial. |
| 112 | +> Herbie: Added model 'hrrr_analysis' from herbie-plugin-tutorial. |
| 113 | +> ``` |
101 | 114 |
|
102 | | -You can now use your custom model like any built-in Herbie model: |
| 115 | +Now you can use those models in Herbie. |
103 | 116 |
|
104 | 117 | ```python |
105 | | -H = Herbie("2025-01-01", model="hrrr_analysis") |
| 118 | +H = Herbie("2023-01-01", model="hrrr_analysis") |
| 119 | +
|
| 120 | +# This one doesn't find anything because its a fictitious model |
| 121 | +H = Herbie("2022-01-01", model="bmw", domain='hello') |
106 | 122 | ``` |
107 | 123 |
|
108 | | -```python |
109 | | -H = Herbie("2025-01-01", model="bmw", domain='3a') |
| 124 | +You can look at the possible source locations with `H.SOURCES` |
110 | 125 |
|
111 | | -# Note: This example will never find anything because BMW is a fictional model for demonstration. |
| 126 | +```python |
| 127 | +>>> from herbie import Herbie |
| 128 | +>>> H = Herbie("2022-01-01", model="bmw", domain='hello') |
| 129 | +💔 Did not find ┊ model=bmw ┊ product=default ┊ 2022-Jan-01 00:00 UTC F00 |
| 130 | +>>> H.SOURCES |
| 131 | +{'local_main': '/path/to/bmw/model/output/bmw/gribfiles/2022010100/my_file.t00z.f00_hello.grib2'} |
112 | 132 | ``` |
| 133 | + |
| 134 | +## Fin |
| 135 | + |
| 136 | +Please [tell me](https://github.com/blaylockbk/Herbie/discussions/categories/show-and-tell) if you made a useful plugin. Consider publishing it on GitHub or PyPI if you think others would like it too. |
0 commit comments