Guide how to create and use YOLO-NAS ONNX models with Frigate 0.17 TensorRT Image (GeForce GPU)

[Rick-Hard89]

This is a simple walkthrough for using a custom YOLO-NAS ONNX model with Frigate 0.17.0-beta2-tensorrt.
I'm using a docker-compose install on Debian 12 but might work with other versions (haven't tested)

Example container config (compose.yaml):

services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.17.0-beta2-tensorrt
    runtime: nvidia

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]

    shm_size: "1024mb"

    devices:
      - /dev/nvidia0:/dev/nvidia0
      - /dev/nvidiactl:/dev/nvidiactl
      - /dev/nvidia-uvm:/dev/nvidia-uvm

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/frigate-nvr/config/:/config/

Detector and model configuration (config.yaml):
Adjust width/height/path if you export 480 or 640.

detectors:
  onnx:
    type: onnx

model:
  model_type: yolonas
  width: 320
  height: 320
  input_tensor: nchw
  input_pixel_format: bgr
  path: /config/model_cache/yolo_nas_s_320.onnx
  labelmap_path: /config/labelmap/coco-80.txt

Prepare model directory and fix ownership:

mkdir -p /srv/frigate-nvr/config/model_cache
cd /srv/frigate-nvr/config/model_cache

sudo chown -R $USER:$USER /srv/frigate-nvr/config/model_cache/

Environment setup:

sudo apt update
sudo apt install libgomp1 -y
sudo apt install python3.11-venv -y

python3 -m venv yolo_env
source yolo_env/bin/activate

Install packages in the following order or paste everything at once:
You will see red dependency warnings, ignore them.

pip install torch torchvision onnx onnxruntime
pip install super-gradients==3.5.0 --no-deps
pip install hydra-core omegaconf pycocotools tqdm scipy matplotlib boto3
pip install boto3 Deprecated jsonschema psutil
python -m pip install --force-reinstall --no-deps super-gradients==3.5.0
pip install stringcase
pip install "numpy<2.0.0" "onnx-simplifier<1.0" "einops==0.3.2" "pycocotools==2.0.6" "termcolor==1.1.0" opencv-python tqdm scipy matplotlib
pip install hydra-core omegaconf antlr4-python3-runtime==4.9.3
pip install rapidfuzz
pip install treelib json-tricks wheel
pip install "torchmetrics==0.8.0" "data-gradients~=0.3.1" "coverage~=5.3.1" "pip-tools>=6.12.1" "sphinx~=4.0.2" "sphinx-rtd-theme" "stringcase" "notebook" "pyparsing==2.4.5" "antlr4-python3-runtime==4.9.3" "rapidfuzz" "treelib" "json-tricks"
pip install onnxscript onnxsim
pip install "onnx==1.15.0" "onnxruntime-gpu==1.17.1"
pip install "onnx==1.16.1" "onnxruntime-gpu==1.18.0" "onnxsim" --force-reinstall
pip install "torch<2.3.0" "torchvision<0.18.0" "numpy<2.0" --force-reinstall

Download YOLO-NAS weights

wget https://sg-hub-nv.s3.amazonaws.com/models/yolo_nas_s_coco.pth

Then create the python script that will convert the model:
(Adjust width/height/name if you export 480 or 640.)

nano convert.py

Paste:

import torch
from super_gradients.training import models
from super_gradients.common.object_names import Models
from super_gradients.conversion import DetectionOutputFormatMode

input_model_path = "yolo_nas_s_coco.pth"

# change the two following lines for different size (H, W)
output_model_name = "yolo_nas_s_320.onnx"
input_shape = (320, 320)
#

print(f"Converting")

try:
    model = models.get(Models.YOLO_NAS_S, num_classes=80, checkpoint_num_classes=80)
    model.set_dataset_processing_params(
        class_names=["object"] * 80,
        conf=0.25,
        iou=0.45,
        image_processor={
            "ComposeProcessing": {
                "processings": [
                    {"NormalizeImage": {"mean": [0.0, 0.0, 0.0], "std": [255.0, 255.0, 255.0]}},
                    {"ImagePermute": {"permutation": [2, 0, 1]}}
                ]
            }
        }
    )
    checkpoint = torch.load(input_model_path, map_location="cpu")
    if isinstance(checkpoint, dict):
        if 'net' in checkpoint:
            model.load_state_dict(checkpoint['net'])
        elif 'ema_net' in checkpoint:
            model.load_state_dict(checkpoint['ema_net'])
        else:
            model.load_state_dict(checkpoint)
    else:
        model.load_state_dict(checkpoint.state_dict())
    model.eval()
    model.export(
        output_model_name,
        output_predictions_format=DetectionOutputFormatMode.FLAT_FORMAT,
        input_image_shape=input_shape,
        confidence_threshold=0.25,
        nms_threshold=0.45
    )

    print(f"\nDone, file created: {output_model_name}")

except Exception as e:
    print(f"\nERROR: {e}")

Start the script and wait for it to say Done

python convert.py

Your exported ONNX-file should be in:

/srv/frigate-nvr/config/model_cache/yolo_nas_s_320.onnx

Then create the folder labelmaps and a file called coco-80.txt:

nano /srv/frigate-nvr/config/labelmap/coco-80.txt

with the following list into it:

person
bicycle
car
motorcycle
airplane
bus
train
truck
boat
traffic light
fire hydrant
stop sign
parking meter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
backpack
umbrella
handbag
tie
suitcase
frisbee
skis
snowboard
sports ball
kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
cake
chair
couch
potted plant
bed
dining table
toilet
tv
laptop
mouse
remote
keyboard
cell phone
microwave
oven
toaster
sink
refrigerator
book
clock
vase
scissors
teddy bear
hair drier
toothbrush

Start Frigate

docker compose up -d

Detector Inference Speed (GeForce RTX 3050) from testing with 4x4k and 4x1080p cameras:

320 = ~10ms
480 = ~13ms
640 = ~18ms

[saaapaaa]

Thank you so so much......

[Rick-Hard89]

No problem

[memory-thrasher]

The aws s3 link to the weights is bad (403). Any advice on where to get that file now? I see a ton of yolo weight files on huggingface.co but I do not see anything I could confidently say is the same file. Specifically, I can't find a source of a yolo file that is labeled with both nas and coco. Would any of these work?
https://huggingface.co/Ultralytics/YOLO26/tree/main

[memory-thrasher]

I have a nvidia gpu and yolo v7 with tensorrt but frigate 0.17 refuses to start saying I need an onnx model because my cpu is amd (which makes no sense at all to me).

[NickM-27]

Right, YOLOv9 is ONNX. https://docs.frigate.video/configuration/object_detectors#yolov9-for-other-detectors

[memory-thrasher]

linked instructions are broken. I started a new discussion thread here

[Rick-Hard89]

I also ran into the same problems with downloading models. Which is why i used and shared this method to create my own which is the proper way to do it.
If i remember correctly its not allowed to download the models without paying for them or having some kind of licence. But creating your own is ok. Correct me if i'm wrong

[Rick-Hard89]

i tried looking for new links but it seems Deci have made all of them accessible without auth