[Bug]: RKNN: 0.16.3: [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization #20751
-
Checklist
Describe the problem you are havingUpdate from 0.16.1 to 0.16.2 AND 0.16.3 give me this error: RAXDA Rock 5b 16GB RAM uname -r ls /dev/dri cat /sys/kernel/debug/rknpu/version Error-Message 2025-11-01 17:41:17.178364994 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/transformer/encoder/layers.11/norm1/Constant_output_0for node: /transformer/emb_ln/Mul/SimplifiedLayerNormFusion/ Explore is Unavailable Steps to reproduceimage: ghcr.io/blakeblackshear/frigate:0.16.1-rk)
image: ghcr.io/blakeblackshear/frigate:0.16.2-rk
image: ghcr.io/blakeblackshear/frigate:0.16.3-rk
VersionSystem 0.16.2-4d58206 In which browser(s) are you experiencing the issue with?No response Frigate config fileui:
# Optional: Set a timezone to use in the UI (default: use browser local time)
# timezone: America/Denver
# Optional: Set the time format used.
# Options are browser, 12hour, or 24hour (default: shown below)
time_format: browser
# Optional: Set the date style for a specified length.
# Options are: full, long, medium, short
# Examples:
# short: 2/11/23
# medium: Feb 11, 2023
# full: Saturday, February 11, 2023
# (default: shown below).
date_style: short
# Optional: Set the time style for a specified length.
# Options are: full, long, medium, short
# Examples:
# short: 8:14 PM
# medium: 8:15:22 PM
# full: 8:15:22 PM Mountain Standard Time
# (default: shown below).
time_style: medium
# Optional: Ability to manually override the date / time styling to use strftime format
# https://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html
# possible values are shown above (default: not set)
#strftime_fmt: "%Y/%m/%d %H:%M"
# Optional: Set the unit system to either "imperial" or "metric" (default: metric)
# Used in the UI and in MQTT topics
unit_system: metric
# Include all cameras by default in Birdseye view
birdseye:
enabled: true
mode: motion
ffmpeg:
path: "7.0"
hwaccel_args: preset-rkmpp
detect:
enabled: true
detectors:
rknn_0:
type: rknn
num_cores: 0
rknn_1:
type: rknn
num_cores: 0
rknn_2:
type: rknn
num_cores: 0
# rknn: # required
# type: rknn # required
# # number of NPU cores to use
# # 0 means choose automatically
# # increase for better performance if you have a multicore NPU e.g. set to 3 on rk3588
# num_cores: 3
classification:
bird:
enabled: true
threshold: 0.4
semantic_search:
enabled: True
reindex: True
model: "jinav1"
model_size: small
# Optional: Configuration for face recognition capability
# NOTE: enabled, min_area can be overridden at the camera level
face_recognition:
# Optional: Enable face recognition (default: shown below)
enabled: true
# Optional: Minimum face detection score required to detect a face (default: shown below)
# NOTE: This only applies when not running a Frigate+ model
detection_threshold: 0.5
# Optional: Minimum face distance score required to mark as a potential match (default: shown below)
unknown_score: 0.6
# Optional: Minimum face distance score required to be considered a match (default: shown below)
recognition_threshold: 0.7
# Optional: Min area of detected face box to consider running face recognition (default: shown below)
min_area: 480
# Optional: Min face recognitions for the sub label to be applied to the person object (default: shown below)
min_faces: 1
# Optional: Number of images of recognized faces to save for training (default: shown below)
save_attempts: 100
# Optional: Apply a blur quality filter to adjust confidence based on the blur level of the image (default: shown below)
blur_confidence_filter: true
# Optional: Set the model size used face recognition. (default: shown below)
model_size: small
audio:
enabled: true
listen:
- bark
- fire_alarm
- scream
- speech
- yell
record:
enabled: true
retain:
days: 3
mode: all
alerts:
retain:
days: 15
mode: motion
detections:
retain:
days: 16
mode: motion
# Optional: Review configuration
# NOTE: Can be overridden at the camera level
review:
# Optional: alerts configuration
alerts:
# Optional: enables alerts for the camera (default: shown below)
enabled: true
# Optional: labels that qualify as an alert (default: shown below)
labels:
- car
- cat
- dog
- person
# Optional: detections configuration
detections:
# Optional: enables detections for the camera (default: shown below)
enabled: true
# Optional: labels that qualify as a detection (default: all labels that are tracked / listened to)
#labels:
#- car
#- person
# Optional: required zones for an object to be marked as a detection (default: none)
# NOTE: when settings required zones globally, this zone must exist on all cameras
# or the config will be considered invalid. In that case the required_zones
# should be configured at the camera level.
#required_zones:
#- driveway
cameras:
Hof: # <--- this will be changed to your actual camera later
enabled: true
ffmpeg:
inputs:
- path:
rtsp://mystream
roles:
- record
- path:
rtsp://mystream
roles:
- audio
- detect
zones: {}
motion:
mask: 0.69,0.054,0.689,0.097,0.962,0.097,0.962,0.056
threshold: 30
contour_area: 10
improve_contrast: true
review:
detections: {}
PTZ-3: # <--- this will be changed to your actual camera later
enabled: true
ffmpeg:
inputs:
- path:
rtsp://mystream
roles:
- record
- path:
rtsp://mystream
roles:
- detect
- audio
zones:
Gehweg:
coordinates:
0.996,0.853,0.787,0.84,0.501,0.803,0.207,0.753,0.002,0.711,0,0.336,0.767,0.339,0.768,0.303,0.856,0.304,0.856,0.361,0.996,0.366
loitering_time: 0
objects: person
inertia: 3
motion:
threshold: 30
contour_area: 10
improve_contrast: true
mask: 0.764,0.023,0.765,0.07,0.982,0.07,0.982,0.026
review:
detections: {}
alerts:
required_zones: Gehweg
Carport: # <--- this will be changed to your actual camera later
enabled: true
ffmpeg:
inputs:
- path: rtsp://mystream
roles:
- record
- path: rtsp://mystream
roles:
- audio
- detect
zones: {}
review:
detections: {}
motion:
threshold: 30
contour_area: 10
improve_contrast: true
model: # required
# name of model (will be automatically downloaded) or path to your own .rknn model file
# possible values are:
# - deci-fp16-yolonas_s
# - deci-fp16-yolonas_m
# - deci-fp16-yolonas_l
# your yolonas_model.rknn
path: deci-fp16-yolonas_s
model_type: yolonas
width: 320
height: 320
input_pixel_format: bgr
input_tensor: nhwc
labelmap_path: /labelmap/coco-80.txt
# Optional: logger verbosity settings
logger:
# Optional: Default log verbosity (default: shown below)
default: info
# Optional: Component specific logger overrides
logs:
frigate.event: debug
frigate.data_processing.real_time.face: debug
version: 0.16-0docker-compose file or Docker CLI commandservices:
frigate:
container_name: frigate
privileged: true # this may not be necessary for all setups
restart: unless-stopped
stop_grace_period: 30s # allow enough time to shut down the various services
image: ghcr.io/blakeblackshear/frigate:0.16.2-rk
shm_size: "512mb" # update for your cameras based on calculation above
#security_opt:
# - apparmor=unconfined
# - systempaths=unconfined
devices:
- /dev/dri
- /dev/dma_heap
- /dev/rga
- /dev/mpp_service
#- /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
volumes:
- /etc/localtime:/etc/localtime:ro
- ./config:/config
- ./media:/media/frigate
- /sys/:/sys/:ro
- type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
target: /tmp/cache
tmpfs:
size: 1000000000
ports:
- "8971:8971"
#- "5000:5000" # Internal unauthenticated access. Expose carefully.
- "8554:8554" # RTSP feeds
- "8555:8555/tcp" # WebRTC over tcp
- "8555:8555/udp" # WebRTC over udp
environment:
FRIGATE_RTSP_PASSWORD: "1234"Relevant Frigate log outputinfo | 2025-11-01 17:40:52 | startup | Preparing Frigate...
info | 2025-11-01 17:40:53 | startup | Starting Frigate...
unknown | 2025-11-01 17:40:54 | unknown | 2025-11-01 17:40:54.258743682 [W:onnxruntime:Default, device_discovery.cc:164 DiscoverDevicesForPlatform] GPU device discovery failed: device_discovery.cc:89 ReadFileContents Failed to open file: "/sys/class/drm/card1/device/vendor"
unknown | 2025-11-01 17:40:54 | unknown | /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
| 0 | | 025-11-01 17:40:54.962161082 setattr(self, word, getattr(machar, word).flat[0])
unknown | 2025-11-01 17:40:54 | unknown | /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
unknown | 2025-11-01 17:40:54 | unknown | return self._float_to_str(self.smallest_subnormal)
unknown | 2025-11-01 17:40:54 | unknown | /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
| 0 | | 025-11-01 17:40:54.962725147 setattr(self, word, getattr(machar, word).flat[0])
unknown | 2025-11-01 17:40:54 | unknown | /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
unknown | 2025-11-01 17:40:54 | unknown | return self._float_to_str(self.smallest_subnormal)
info | 2025-11-01 17:40:57 | frigate.util.config | Checking if frigate config needs migration...
info | 2025-11-01 17:40:57 | frigate.util.config | frigate config does not need migration...
info | 2025-11-01 17:41:14 | frigate.app | Starting Frigate (0.16.2-4d58206)
info | 2025-11-01 17:41:14 | peewee_migrate.logs | Starting migrations
info | 2025-11-01 17:41:14 | peewee_migrate.logs | There is nothing to migrate
info | 2025-11-01 17:41:14 | frigate.app | Recording process started: 465
info | 2025-11-01 17:41:14 | frigate.app | Review process started: 478
info | 2025-11-01 17:41:14 | frigate.app | go2rtc process pid: 128
info | 2025-11-01 17:41:14 | detector.rknn_0 | Starting detection process: 490
info | 2025-11-01 17:41:14 | detector.rknn_1 | Starting detection process: 493
info | 2025-11-01 17:41:14 | detector.rknn_2 | Starting detection process: 496
info | 2025-11-01 17:41:14 | frigate.app | Embedding process started: 499
info | 2025-11-01 17:41:14 | frigate.app | Output process started: 520
info | 2025-11-01 17:41:14 | frigate.app | Camera processor started for Hof: 541
info | 2025-11-01 17:41:14 | frigate.app | Camera processor started for PTZ-3: 543
info | 2025-11-01 17:41:14 | frigate.app | Camera processor started for Carport: 550
info | 2025-11-01 17:41:14 | frigate.app | Capture process started for Hof: 574
info | 2025-11-01 17:41:14 | frigate.app | Capture process started for PTZ-3: 588
info | 2025-11-01 17:41:14 | frigate.app | Capture process started for Carport: 600
unknown | 2025-11-01 17:41:14 | unknown | INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
info | 2025-11-01 17:41:14 | frigate.audio_manager | Audio processor started (pid: 612)
info | 2025-11-01 17:41:15 | frigate.api.fastapi_app | Starting FastAPI app
info | 2025-11-01 17:41:15 | frigate.api.fastapi_app | FastAPI started
unknown | 2025-11-01 17:41:17 | unknown | 2025-11-01 17:41:17.178364994 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/transformer/encoder/layers.11/norm1/Constant_output_0for node: /transformer/emb_ln/Mul/SimplifiedLayerNormFusion/
unknown | 2025-11-01 17:41:17 | unknown |
unknown | 2025-11-01 17:41:17 | unknown | Process embeddings_manager:
unknown | 2025-11-01 17:41:17 | unknown | Traceback (most recent call last):
unknown | 2025-11-01 17:41:17 | unknown | File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
unknown | 2025-11-01 17:41:17 | unknown | self.run()
unknown | 2025-11-01 17:41:17 | unknown | File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
unknown | 2025-11-01 17:41:17 | unknown | return run(*args, **kwargs)Relevant go2rtc log outputinfo | 2025-11-01 15:55:50 | startup | Preparing new go2rtc config...
info | 2025-11-01 15:55:50 | startup | Starting go2rtc...
info | 2025-11-01 15:55:50 | startup | go2rtc platform=linux/arm64 revision=fa580c5 version=1.9.9
info | 2025-11-01 15:55:50 | startup | config path=/dev/shm/go2rtc.yaml
info | 2025-11-01 15:55:50 | rtsp | listen addr=:8554
info | 2025-11-01 15:55:50 | api | listen addr=:1984
info | 2025-11-01 15:55:50 | webrtc | listen addr=:8555
info | 2025-11-01 15:55:59 | startup | Starting go2rtc healthcheck service...Operating systemOther Install methodDocker Compose Network connectionWired Camera make and model54X 8MP 4K Sony415 Screenshots of the Frigate UI's System metrics pages
Any other information that may be helpfulExplore is Unavailable |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 10 replies
-
|
The error you're encountering is related to the semantic search embeddings initialization, not the RKNN detector itself(1). The ONNX runtime error occurs when Frigate attempts to load the vision embedding models for semantic search functionality. From your logs, the embeddings process is failing during initialization with an ONNX runtime exception. The error indicates that the vision model and vision model feature extractor are marked as "nok" (not okay), while the text model and tokenizer loaded successfully. To resolve this issue, you have a few options: Option 1: Disable semantic search temporarily Since your RKNN detectors (rknn_0, rknn_1, rknn_2) are starting successfully and the core detection functionality appears to be working, you can disable semantic search in your config: semantic_search:
enabled: falseOption 2: Try a different model size You're currently using the "jinav1" model with "small" size. You could try switching to a different configuration: semantic_search:
enabled: true
reindex: false
model_size: smallRemove the Option 3: Clear the model cache The ONNX model files may be corrupted. You can try clearing the model cache directory (typically Your RKNN detector configuration looks correct with three detectors configured(1)(2), and based on your system metrics screenshots, the detectors appear to be functioning properly. The issue is isolated to the embeddings/semantic search component, not your primary object detection pipeline. 📚 Sources: Have another question? Just tag @inkeep. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, Option 1-3 is not a real solution. |
Beta Was this translation helpful? Give feedback.
-
|
Hint
info | 2025-11-01 20:15:27 | frigate.camera.maintainer | Camera processor started for PTZ-3: 986 |
Beta Was this translation helpful? Give feedback.
-
|
Now i try Docker image: "ghcr.io/blakeblackshear/frigate:stable-rk" onnxruntime crashed...: Attempting to get index by a name which does not exis What is the difference from 0.16.1 (ok) -> 0.16.2 (nok) -> 0.16.3 (nok) -> 0.17.dev (ok) ??? 2025-11-04 18:09:28.356082248 W rknn-toolkit-lite2 version: 2.3.2 ` |
Beta Was this translation helpful? Give feedback.
-
|
0.17 has had this completely rewritten |
Beta Was this translation helpful? Give feedback.
-
|
Update from 0.16.1 to 0.16.2 AND 0.16.3 give me this error.
So should i use the 0.17 dev versions?
|
Beta Was this translation helpful? Give feedback.
-
|
As @NickM-27 mentioned, this has been rewritten for 0.17. We don't intend to back port the changes to 0.16, so yes, running a dev version of 0.17 is your best option. |
Beta Was this translation helpful? Give feedback.
-
|
what is the tag of the docker image for the dev 0.17? |
Beta Was this translation helpful? Give feedback.
-
|
For my Radxa Rock5b the last working 16er docker image was ghcr.io/blakeblackshear/frigate:0.16.1-rk. So i test the *-rk docker from here: https://github.com/blakeblackshear/frigate/pkgs/container/frigate |
Beta Was this translation helpful? Give feedback.
-
|
sorry, I made a mistake of my previous message. I am looking for 0.17. which has solved the error. So, I believe the image will be: |
Beta Was this translation helpful? Give feedback.
-
|
the current image is ghcr.io/blakeblackshear/frigate:0.17.0-beta1 |
Beta Was this translation helpful? Give feedback.
-
|
thanks |
Beta Was this translation helpful? Give feedback.
-
|
Hi, same issue on orange pi 5 too |
Beta Was this translation helpful? Give feedback.
-
|
See above. We don't intend to backport these changes to 0.16, and this has already been improved in 0.17, currently in beta. You can run a beta version or dev build if you need it immediately. |
Beta Was this translation helpful? Give feedback.
-
Hi, yeah ended up doing that, currently on 0.17. |
Beta Was this translation helpful? Give feedback.
As @NickM-27 mentioned, this has been rewritten for 0.17. We don't intend to back port the changes to 0.16, so yes, running a dev version of 0.17 is your best option.