Skip to content

Commit 09a0449

Browse files
makseqcaitlinwheeless
makseq
and
caitlinwheeless
authored
docs: Add more info on how to use YOLO custom models (#638)
Co-authored-by: caitlinwheeless <[email protected]>
1 parent 7b48f35 commit 09a0449

File tree

2 files changed

+140
-15
lines changed

2 files changed

+140
-15
lines changed

‎label_studio_ml/examples/yolo/README.md‎

Lines changed: 138 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,9 @@ This tutorial uses the [YOLO example](https://github.com/HumanSignal/label-studi
6262

6363
## Quick start
6464

65-
1. Add `LABEL_STUDIO_URL` and `LABEL_STUDIO_API_KEY` to the `docker-compose.yml` file. These variables should point to your Label Studio instance and its API key, respectively. For more information about finding your Label Studio API key, [see our documentation](https://labelstud.io/guide/user_account#Access-token).
65+
1. Add `LABEL_STUDIO_URL` and `LABEL_STUDIO_API_KEY` to the `docker-compose.yml` file.
66+
These variables should point to your Label Studio instance and its API key, respectively.
67+
For more information about finding your Label Studio API key, [see our documentation](https://labelstud.io/guide/user_account#Access-token).
6668

6769
2. Run docker compose
6870

@@ -229,18 +231,141 @@ Attempts to run YOLOv5 were unsuccessful without modifications.
229231
It may be possible to run it by applying some changes, such as installing additional dependencies.
230232
The same applies to other YOLO models.
231233
232-
## Custom YOLO models
234+
## Your own custom YOLO models
233235
234236
You can load your own YOLO labels using the following steps:
235237
236-
1. Mount your model as `/app/models/<your-model>.pt` inside of your docker.
237-
2. Set `ALLOW_CUSTOM_MODEL_PATH=true` (it is true by default) in your Docker environment parameters ([`docker-compose.yml`](docker-compose.yml)).
238-
3. Add `model_path="<your-model>.pt"` to the control tag in the labeling configuration, e.g.:
238+
1. Mount your model as `/app/models/<your-model>.pt` inside of your Docker.
239+
2. Add `model_path="<your-model>.pt"` to the control tag in the labeling configuration, e.g.:
239240
240241
```xml
241242
<RectangleLabels model_path="my_model.pt">
242243
```
243244
245+
<details>
246+
<summary><b>Step by step guide</b>: Using your own custom YOLOv8 model</summary>
247+
<br/>
248+
You can integrate your own custom-trained YOLOv8 models with the YOLO ML backend for Label Studio.
249+
Follow these detailed steps to set up your custom model in the ML backend Docker:
250+
251+
### Step 1: Prepare your custom YOLOv8 model
252+
253+
Ensure that your custom YOLOv8 model is saved as a `.pt` file, which is the standard format for PyTorch models.
254+
Let's assume your model file is named `my_custom_model.pt`.
255+
256+
### Step 2: Place your model
257+
258+
1. **Create a 'models' directory:**
259+
260+
In your project root directory `yolo` (the same directory where your `docker-compose.yml` is located),
261+
create a new folder named `models` and place your `my_custom_model.pt` file inside it.
262+
263+
```
264+
yolo/
265+
├── docker-compose.yml
266+
├── models/
267+
└── my_custom_model.pt
268+
```
269+
270+
2. **Modify `docker-compose.yml` to mount the 'models' directory:**
271+
272+
Update your `docker-compose.yml` file to include a volume that maps the local `models` directory
273+
to the `/app/models` directory inside the Docker container.
274+
275+
```yaml
276+
services:
277+
yolo:
278+
container_name: yolo
279+
...
280+
volumes:
281+
- ./models:/app/models # Mount the local 'models' directory
282+
environment:
283+
- ALLOW_CUSTOM_MODEL_PATH=true
284+
...
285+
```
286+
287+
### Step 3: Ensure environment variables are set correctly
288+
289+
- **`ALLOW_CUSTOM_MODEL_PATH=true`**: Ensure this is set in your Docker environment variables to allow the ML backend to load custom model paths (it's `true` by default).
290+
- **`LABEL_STUDIO_URL`**: This is necessary to specify the external IP or domain of your Label Studio instance.
291+
If you run Label Studio and ML backend locally, you can try setting it to `LABEL_STUDIO_URL=http://host.docker.internal:8080`.
292+
- **`LABEL_STUDIO_API_KEY`**: This is necessary to specify the API key of your Label Studio instance.
293+
You can find it in Label Studio on the [User Account page](https://labelstud.io/guide/user_account#Access-token).
294+
- **`LOG_LEVEL`**: (optional) Set the logging level for the ML backend. You can use `DEBUG`, `INFO`, `WARNING`, `ERROR`.
295+
296+
Example `docker-compose.yml`:
297+
298+
```yaml
299+
environment:
300+
- ALLOW_CUSTOM_MODEL_PATH=true
301+
- LABEL_STUDIO_URL=http://host.docker.internal:8080
302+
- LABEL_STUDIO_API_KEY=your_api_key
303+
- LOG_LEVEL=DEBUG # optional
304+
```
305+
306+
### Step 4: Update your Labeling Configuration in Label Studio
307+
308+
In your Label Studio project, specify the path to your custom model in the labeling configuration
309+
by adding the `model_path` parameter to the control tag you're using
310+
(e.g., `<RectangleLabels>`, `<PolygonLabels>`, `<Choices>`, etc.).
311+
312+
**Example for Object Detection with `<RectangleLabels>`:**
313+
314+
```xml
315+
<View>
316+
<Image name="image" value="$image"/>
317+
<RectangleLabels
318+
name="label" toName="image"
319+
model_path="my_custom_model.pt"
320+
model_score_threshold="0.25">
321+
<Label value="Cat"/>
322+
<Label value="Dog"/>
323+
</RectangleLabels>
324+
</View>
325+
```
326+
327+
**Important Notes:**
328+
329+
- **`model_path`**: The **`model_path`** should match the filename of your custom model and must be located in the `/app/models` directory inside the Docker container.
330+
- **Class names mapping**: Ensure that the `value` attributes in your `<Label>` tags correspond to the **class names** your model predicts. If they differ, use the `predicted_values` attribute to map them. Example:
331+
```xml
332+
<Label value="Cat" predicted_values="feline"/>
333+
<Label value="Dog" predicted_values="canine"/>
334+
```
335+
- **Where to find class names**: See ML backend logs **to know the exact class names** your model predicts, and then
336+
you can use these names in the `predicted_values` attribute and in the `value` of Label tags directly.
337+
338+
### Step 5: Restart the ML Backend
339+
340+
After making these changes, restart the ML backend to apply the new configuration.
341+
342+
```bash
343+
docker-compose down
344+
docker-compose up --build
345+
```
346+
347+
### Step 6: Connect the ML Backend to your Label Studio project
348+
349+
1. Open your Label Studio instance.
350+
2. Go to the **Settings** of your project.
351+
3. Navigate to the **Model** tab.
352+
4. Connect to the ML backend by entering the ML backend URL
353+
(if you run it locally, it's most likely `http://localhost:9090`).
354+
355+
### Step 7: Test your setup
356+
357+
Add some tasks (images or other data) to your Label Studio project and open a task in the labeling interface.
358+
The ML backend should now use your custom model to generate predictions.
359+
360+
### Common pitfalls
361+
362+
- **Incorrect model path:** Ensure that the `model_path` in your labeling configuration exactly matches the filename of your model inside `/app/models`.
363+
- **Label mismatch:** Double-check that your labels in Label Studio match the classes your model predicts, or use `predicted_values` to map them.
364+
- **Keypoints models and model_index:** If you use a keypoints model, you should specify the `model_index` parameter in each `Label` tag.
365+
366+
</details>
367+
368+
244369
## Training
245370

246371
The current Label Studio ML backend doesn't support training YOLO models. You have to do it manually on your side.
@@ -276,10 +401,10 @@ https://github.com/user-attachments/assets/30c5ce43-2c89-4ddf-a77d-9d1d75ac3419
276401

277402
### Parameters
278403

279-
| Parameter | Type | Default | Description |
280-
|-------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
281-
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
282-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
404+
| Parameter | Type | Default | Description |
405+
|-------------------------|--------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
406+
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
407+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#your-own-custom-yolo-models). |
283408
| `choice` | string | single | Possible values: `single`, `single-radio`, `multiple`. If you use `choice="single"` (default) you can select only one label. The ML backend will return the label with the highest confidence using argmax strategy. If you use `choice="multiple"` you can select multiple labels. The ML backend will return all labels with confidence above the `model_score_threshold`. |
284409

285410

@@ -327,7 +452,7 @@ https://github.com/user-attachments/assets/413b4650-422d-43dc-809d-51c08f0ad434
327452
| Parameter | Type | Default | Description |
328453
|-------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
329454
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
330-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
455+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#your-own-custom-yolo-models). |
331456
| `model_obb` | bool | False | Enables Oriented Bounding Boxes (OBB) mode. Typically it uses `*-obb.pt` yolo models. |
332457

333458
For example:
@@ -386,7 +511,7 @@ https://github.com/user-attachments/assets/9b2447d3-392d-42be-bc7f-ef2b6c81d54c
386511
| Parameter | Type | Default | Description |
387512
|-------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
388513
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
389-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
514+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#your-own-custom-yolo-models). |
390515

391516
For example:
392517
```xml
@@ -463,7 +588,7 @@ More info: [Ultralytics YOLO Keypoint Documentation](https://docs.ultralytics.co
463588

464589
| Parameter | Type | Default | Description |
465590
|-------------------------|--------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
466-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
591+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#your-own-custom-yolo-models). |
467592
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for bounding box detections. Keypoints that are related to the detected bbox with a confidence below this threshold will be disregarded. |
468593
| `model_point_threshold` | float | 0.0 | Minimum confidence threshold for keypoints. Keypoints with confidence below this value will be ignored. |
469594
| `model_add_bboxes` | bool | True | Adds bounding boxes for detected keypoints. All keypoints will be grouped by parent bounding boxes on the region panel. See details in the tip below. |
@@ -636,7 +761,7 @@ https://docs.ultralytics.com/modes/track/?h=track#tracking-arguments
636761
| `model_conf` | float | 0.25 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
637762
| `model_iou` | float | 0.7 | Intersection Over Union (IoU) threshold for Non-Maximum Suppression (NMS). Lower values result in fewer detections by eliminating overlapping boxes, useful for reducing duplicates. |
638763
| `model_tracker` | string | `botsort` | Sets the tracker to use for multi-object tracking. Options include `botsort`, `bytetrack`, or a custom YAML file. |
639-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
764+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#your-own-custom-yolo-models). |
640765

641766
For example:
642767
```xml

‎label_studio_ml/examples/yolo/README_TIMELINE_LABELS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ This tutorial uses the [YOLO example](https://github.com/HumanSignal/label-studi
8282
| `model_classifier_f1_threshold` | float | 0.95 | F1 score threshold for early stopping during training. Set to prevent overfitting. |
8383
| `model_classifier_accuracy_threshold` | float | 1.00 | Accuracy threshold for early stopping during training. Set to prevent overfitting. |
8484
| `model_score_threshold` | float | 0.5 | Minimum confidence threshold for predictions. Labels with confidence below this threshold will be disregarded. |
85-
| `model_path` | string | None | Path to the custom YOLO model. See more in the section "Custom YOLO Models." |
85+
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](./README.md#your-own-custom-yolo-models). |
8686

8787
**Note:** You can customize the neural network parameters directly in the labeling configuration by adjusting the attributes in the `<TimelineLabels>` tag.
8888

@@ -158,7 +158,7 @@ and it generates predictions for each frame in the video.
158158

159159
#### Custom YOLO models for feature extraction
160160

161-
You can load your own YOLO models using the steps described in the [main README](./README.md#custom-yolo-models).
161+
You can load your own YOLO models using the steps described in the [main README](./README.md#your-own-custom-yolo-models).
162162
However, it should have similar architecture as `yolov8-cls` models. See `utils/neural_nets.py::cached_feature_extraction()` for more details.
163163

164164
#### Cache folder

0 commit comments

Comments
 (0)