more about custom models

Author: CamDavidsonPilon

developer-guide/10-Hardware/10-custom-hardware.md

@@ -0,0 +1,83 @@
+---
+title: Customize your Pioreactor hardware
+slug: /developer-guide/custom-hardware
+description: Extend or override Pioreactor hardware definitions by layering YAML files that hardware.py reads.
+hide_table_of_contents: true
+---
+
+Pioreactor's hardware layer is intentionally data-driven. Everything in [`core/pioreactor/hardware.py`](https://github.com/pioreactor/pioreactor/blob/main/core/pioreactor/hardware.py) loads user-editable YAML files from `~/.pioreactor/hardware/` (or the folder pointed to by the `DOT_PIOREACTOR` env var). By editing these files you can rewire pins, add new peripherals, or describe an entirely new bioreactor model without touching the Python code. Pair these configs with [custom bioreactor model definitions](/custom-bioreactor-models) so the UI, safety limits, and wiring stay in sync.
+
+## How the loader works
+
+`hardware.py` deep-merges two directories for each **mod** (subsystem):
+
+1. `~/.pioreactor/hardware/hats/<hat_version>/<mod>.yaml` – physical wiring that ships with a specific HAT revision (`hardware_version_info`).
+2. `~/.pioreactor/hardware/models/<model>/<version>/<mod>.yaml` – intent for a particular bioreactor model returned by `get_pioreactor_model()`.
+
+Later layers override earlier ones. Missing files are allowed; missing keys throw `HardwareError`s when helpers such as `get_pwm_to_pin_map()` run.
+
+Common mods defined inside `hardware.py` are:
+
+- `pwm.yaml` (`get_pwm_controller`, `get_heater_pwm_channel`, `get_pwm_to_pin_map`)
+- `gpio.yaml` (`get_pcb_led_pin`, `get_hall_sensor_pin`, `get_sda_pin`, `get_scl_pin`)
+- `temp.yaml` (`get_temp_address`, used by `is_heating_pcb_present`)
+- `adc.yaml` (`get_adc_curriers`, which subsequently powers PD channel helpers)
+- `dac.yaml` (`get_dac_address`)
+
+You can add additional mods with any key/value structure—`get_layered_mod_config("my_mod")` will still combine the YAML files.
+
+## Step-by-step: add a custom hardware profile
+
+1. **Create directories** for your HAT and model version.
+   ```bash
+   mkdir -p ~/.pioreactor/hardware/hats/<hat_version>
+   mkdir -p ~/.pioreactor/hardware/models/<model_name>/<model_version>
+   ```
+   Use `pioreactor.version.hardware_version_info` (for hats) and your `pioreactor.models` entry for models.
+
+2. **Describe the wiring** (hat layer). Example `~/.pioreactor/hardware/hats/0.2/pwm.yaml`:
+   ```yaml
+   controller: hat_mcu
+   heater_pwm_channel: "5"
+   pwm_to_pin:
+     "1": 17
+     "2": 13
+     "5": 18
+   ```
+   These keys map directly to `_load_pwm_cfg()` in `hardware.py`.
+
+3. **Describe the capabilities** (model layer). Suppose you added a new auxiliary photodiode ADC:
+   ```yaml title="~/.pioreactor/hardware/models/my_model/v1.0/adc.yaml"
+   pd1:
+     driver: ADS1115
+     address: 0x48
+     channel: 0
+   aux:
+     driver: pico
+     address: 0x40
+     channel: 0
+   salty:
+     driver: ADS1115
+     address: 0x49
+     channel: 2
+   ```
+   `get_adc_curriers()` will automatically expose `.keys()` for every entry (`pd1`, `aux`, `salty`). Downstream helpers, like `get_available_pd_channels()`, treat `pd*` names as photodiode channels.
+
+4. **Reference new mods in code**. Anywhere in your plugin/automation you can import:
+   ```python
+   from pioreactor.hardware import get_layered_mod_config
+
+   stir_cfg = get_layered_mod_config("stirrer")  # merges hats/<hat>/stirrer.yaml and models/<model>/stirrer.yaml
+   if stir_cfg.get("controller") == "pico":
+       ...
+   ```
+   This keeps custom hardware logic outside Pioreactor core.
+
+## Tips for customization
+
+- Use integers for all addresses and pins; `hardware.py` casts everything via `int()` and will raise if parsing fails.
+- Keep YAML minimal—only override keys that differ from the hat defaults to take advantage of deep-merge layering.
+- The helper predicates (`is_ADC_present`, `is_heating_pcb_present`, etc.) are safe ways to gate optional hardware in your jobs.
+- Store supporting scripts, cad files, or calibration notes under `~/.pioreactor/hardware/<...>/README.md` so teammates know how to reproduce the wiring.
+
+Once you describe the hardware this way, the rest of the stack—jobs, automations, plugins—just asks `hardware.py` for the capabilities it needs, so new hardware becomes a configuration exercise instead of a fork of the core codebase.

developer-guide/10-Hardware/11-custom_models.md

@@ -0,0 +1,75 @@
+---
+title: Custom bioreactor models
+slug: /custom-bioreactor-models
+hide_table_of_contents: true
+---
+
+:::tip
+This feature is actively under development - let us know what else you would like to see!
+:::
+
+If you are building a novel bioreactor on top of our software, you can use the `~/.pioreactor/models` dir to add your own bioreactors. This metadata flows into the Inventory UI, safety interlocks, analytics, and can be consumed from Python via `pioreactor.whoami.get_pioreactor_model()`.
+
+## Directory layout
+
+1. Create the models folder if it does not exist:
+   ```bash
+   mkdir -p ~/.pioreactor/models
+   ```
+2. Add one YAML file per model+version, for example `~/.pioreactor/models/custom_100ml__v1_0.yaml`.
+3. Restart the Pioreactor UI to pick up the new definition.
+4. Assign pioreactors to this new model on the Inventory UI page or via `pio workers update-model ...` tool.
+
+## Required fields
+
+Each YAML is validated against `pioreactor.structs.Model`, so include at least:
+
+- `model_name`: slug-style identifier (`custom_100ml`).
+- `model_version`: semantic-ish version (`"1.0"`).
+- `display_name`: friendly label shown in the Inventory.
+- `reactor_capacity_ml`: physical vessel size.
+- `reactor_max_fill_volume_ml`: operational maximum before spilling.
+- `reactor_diameter_mm`: used by stirring + growth-rate heuristics.
+- `max_temp_to_reduce_heating`, `max_temp_to_disable_heating`, `max_temp_to_shutdown`: temperature safety thresholds in °C.
+
+## Example definition
+
+```yaml title="~/.pioreactor/models/custom_100ml.yaml"
+model_name: custom_100ml
+model_version: "1.0"
+display_name: "Custom 100 mL, v1.0"
+reactor_capacity_ml: 100.0
+reactor_max_fill_volume_ml: 95.0
+reactor_diameter_mm: 50.0
+max_temp_to_reduce_heating: 80.0
+max_temp_to_disable_heating: 85.0
+max_temp_to_shutdown: 90.0
+```
+
+After saving the file, the Inventory page will list “Custom 100 mL, v1.0” in each unit’s model dropdown. The same data becomes available to jobs and plugins:
+
+```python
+from pioreactor.whoami import get_pioreactor_model
+
+model = get_pioreactor_model()
+print(model.display_name)
+print(model.reactor_capacity_ml)
+```
+
+## Coordinating with hardware profiles
+
+Model definitions describe intent (volumes, limits), while the hardware loader handles wiring, pin assignments, and sensor addresses. When you introduce a new model, you usually need a matching directory under `~/.pioreactor/hardware/models/<model_name>/<model_version>/` so [`hardware.py` can layer the right YAMLs](/developer-guide/custom-hardware). Keeping the names and versions identical lets software map UI selections → hardware capabilities.
+
+If you are basing your custom model off an existing Pioreactor model, can reuse the hardware for our Pioreactor models:
+
+```
+mkdir -p ~/.pioreactor/hardware/models/<model_name>/<model_version>
+cp ~/.pioreactor/hardware/models/pioreactor_40ml/1.5/* ~/.pioreactor/hardware/models/<model_name>/<model_version>
+
+```
+
+## Validation and troubleshooting
+
+- Run `python -m pioreactor.models` to ensure your YAML parses; validation errors are also logged via the `models` logger.
+- If a model is missing from the dropdown, confirm the filename ends with `.yaml`/`.yml`, the process has read permissions, and the Pioreactor services have restarted.
+- Treat this directory like code: commit the YAMLs to version control alongside any CAD notes or lab SOPs so teammates understand the assumptions behind each model.

developer-guide/11-custom_models.md

@@ -6,17 +6,17 @@ description: Export and import a Pioreactor’s ~/.pioreactor directory from the
 hide_table_of_contents: true
 ---
 
-Each Pioreactor keeps its configuration, calibration data, and persistent state inside `~/.pioreactor`. The Inventory page now lets you export that directory as a zip for safekeeping and import it onto the same unit when you need to restore or clone a setup.
+Each Pioreactor keeps its configuration, calibration data, and persistent state inside `~/.pioreactor`. The Inventory page now lets you export that directory as a system archive (`.zip`) for safekeeping and import it onto the same unit when you need to restore or clone a setup.
 
-![Manage Pioreactor menu showing export and import actions highlighted](/img/user-guide/inventory-export-import-menu.png)
+![Manage Pioreactor menu showing Export system archive and Import system archive actions highlighted](/img/user-guide/inventory-export-import-menu.png)
 
-## Exporting system files
+## Exporting a system archive
 
-1. Open the **Inventory** page, find the Pioreactor you want to back up, and click **Manage Pioreactor → Export system files**.
-2. Confirm the export when prompted. The UI downloads a file named `<pioreactor>_dot_pioreactor.zip`.
+1. Open the **Inventory** page, find the Pioreactor you want to back up, and click **Manage Pioreactor → Export system archive**.
+2. Review the dialog and click **Export** when ready. The UI downloads a file named `<pioreactor>_dot_pioreactor.zip`.
 3. Store the archive somewhere safe. It contains every file from that unit’s `~/.pioreactor` plus a `pioreactor_export_metadata.json` descriptor (hostname, version, export time).
 
-![Export system files confirmation dialog with the export button highlighted](/img/user-guide/inventory-export-dialog.png)
+![Export system archive confirmation dialog with the Export button highlighted](/img/user-guide/inventory-export-dialog.png)
 
 After you confirm, the browser saves a zip named `<pioreactor>_dot_pioreactor.zip`.
 
@@ -25,15 +25,19 @@ After you confirm, the browser saves a zip named `<pioreactor>_dot_pioreactor.zi
 - Since the export is over http, you may need to "approve" your browser to download it (check the download tab in your browser).
 - For the leader Pioreactor, it's also exporting the sqlite database. This might take a while for very large databases.
 
-## Importing system files
+## Importing a system archive
 
 Use imports when you need to restore a backup, duplicate a Pioreactor after replacing hardware, or bring back calibration data after re-flashing an SD card.
 
-1. On the **Inventory** page, choose **Manage Pioreactor → Import system files**.
-2. Read the warning and confirm. You will be prompted to select a `.zip` generated by the export flow.
+1. On the **Inventory** page, choose **Manage Pioreactor → Import system archive**.
+2. Read the warning and click **Select system archive file**, then pick a `.zip` generated by the export flow.
 3. Wait for confirmation in the snackbar. The UI shows a spinner while uploading and tells you when the Pioreactor begins rebooting.
 
-![Import system files confirmation dialog with the import button highlighted](/img/user-guide/inventory-import-dialog.png)
+:::danger Important
+Importing wipes the existing `~/.pioreactor` on that unit before restoring the archive, so anything not inside the uploaded zip is permanently removed.
+:::
+
+![Import system archive confirmation dialog with the Select system archive file button highlighted](/img/user-guide/inventory-import-dialog.png)
 
 Important behaviour to note: