[Camera Support]: Hikvision doorbell in Frigate. Either jumpy video, or lagging and choppy audio #12501
-
Describe the problem you are havingI'm guessing it's the timestamps problem, however I a complete noob and my expertise lies in other fields. Currently I'm having the audio lagging for several seconds behind the video, the lag seems growing over time. Also, the sound makes annoying micropauses every couple of seconds (probably every 50 frames but sometimes not). Previously I've tried different setups including using the substreams for object detection, restreaming the main stream in go2rtc as it was and adding aac stream just before recording, but the result was SO bad as when the 10(?) second video chunks were only recorded partially and the MSE player cursor was skipping many seconds of the timeline every 10 seconds, I was imagining how out of sync the videostream timestamps were. Other experiments gave me green flashes every couple of seconds in the live view. Those errors mostly were appearing after a several hours of uptime. In the doorbell itself I've also tried in desperation to change the source audio codec from alaw to ulaw or vice versa, and tomake the video bitrate constant, to no avail. Constant (and large) bitrate seemed to make the player itself to lag, but I may be wrong. My current setup adds aac audio in ro2rtc and supposedly fixes the timestamps with the /#async preset, copying the video supposedly intact. I've also decided to use the big screen to run the detection against, to capture cats at last, and also to avoid any extra sync problems. It's the best result so far, I need it to work for a couple of days to measure audio the lag, and if it is nearly constant will be hacking some delay for the videostream. I've tried to add /#video-h264 just to (imaginably) fix the timestamps in the video stream too, but that did'nt help and I would like to avoid to reencode with the same codec. However I suspect this is a known problem with the Hikvision doorbells, though the search only gave me the /#async advice. Maybe someone happens to know how to fix the audio lag problem? Version0.13.2-6476f8a Frigate config filemqtt:
enabled: true
host: 192.168.9.10
#logger:
# logs:
# frigate.record.maintainer: debug
ffmpeg:
hwaccel_args: preset-nvidia-h264
global_args: -hide_banner -loglevel warning -threads 6
detect:
enabled: true
fps: 5
# height: 720
# width: 1280
record:
enabled: true
retain:
days: 10
mode: motion
snapshots:
enabled: true
detectors:
coral:
type: edgetpu
device: pci
objects:
track:
- person
- car
- cat
- bicycle
- motorcycle
- dog
- horse
- sheep
- cow
- bear
- bird
birdseye:
enabled: True
mode: continuous
go2rtc:
streams:
frontdoorphone:
- rtsp://admin:aaa@192.168.9.32/Streaming/Channels/101/
- isapi://admin:aaa@192.168.9.32:80/
frontdoor:
- "ffmpeg:frontdoorphone#async#video=copy#audio=aac#hardware=cuda"
backleftout:
- rtsp://user:user2000@192.168.9.33/cam/realmonitor?channel=1&subtype=0
cameras:
frontdoor:
enabled: true
audio:
enabled: true
min_volume: 100
ffmpeg:
output_args:
record: preset-record-generic-audio-copy
inputs:
- path: rtsp://127.0.0.1:8554/frontdoor?video&audio
input_args: preset-rtsp-restream
roles:
- record
- audio
- detect
zones:
cul_de_sac:
coordinates: 69,860,330,1026,655,1014,909,976,1255,839,1408,742,1341,630,1252,586,1094,590,916,591,650,596,524,621,458,652,218,699,94,772
motion:
mask:
- 0,0,1713,0,1696,379,0,382
- 1355,871,1748,880,1743,839,1362,837
backleftout:
enabled: true
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/backleftout
input_args: preset-rtsp-restream
roles:
- record
- detect
motion:
mask:
- 1120,1440,0,1440,0,0,980,0,1098,563,1133,1034Relevant log output2024-07-18 08:19:45.797184240 [INFO] Preparing Frigate...
2024-07-18 08:19:45.806738101 [INFO] Starting Frigate...
2024-07-18 08:19:46.915164720 [2024-07-18 08:19:46] frigate.app INFO : Starting Frigate (0.13.2-6476f8a)
2024-07-18 08:19:52.879960378 [2024-07-18 08:19:52] peewee_migrate.logs INFO : Starting migrations
2024-07-18 08:19:52.946749390 [2024-07-18 08:19:52] peewee_migrate.logs INFO : There is nothing to migrate
2024-07-18 08:19:52.952793089 [2024-07-18 08:19:52] frigate.app INFO : Recording process started: 1425
2024-07-18 08:19:52.955255676 [2024-07-18 08:19:52] frigate.app INFO : go2rtc process pid: 89
2024-07-18 08:19:52.978535589 [2024-07-18 08:19:52] detector.coral INFO : Starting detection process: 1435
2024-07-18 08:19:52.980927583 [2024-07-18 08:19:52] frigate.app INFO : Output process started: 1437
2024-07-18 08:19:52.994634455 [2024-07-18 08:19:52] frigate.app INFO : Camera processor started for frontdoor: 1444
2024-07-18 08:19:52.998110652 [2024-07-18 08:19:52] frigate.detectors.plugins.edgetpu_tfl INFO : Attempting to load TPU as pci
2024-07-18 08:19:52.999213829 [2024-07-18 08:19:52] frigate.app INFO : Camera processor started for backleftout: 1452
2024-07-18 08:19:53.003062223 [2024-07-18 08:19:52] frigate.detectors.plugins.edgetpu_tfl INFO : TPU found
2024-07-18 08:19:53.003250694 [2024-07-18 08:19:53] frigate.app INFO : Capture process started for frontdoor: 1457
2024-07-18 08:19:53.007562484 [2024-07-18 08:19:53] frigate.app INFO : Capture process started for backleftout: 1462
2024-07-18 08:19:53.011864231 [2024-07-18 08:19:53] frigate.app INFO : Audio process started: 1466FFprobe output from your camera[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":1080,"width":1920},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}}]Frigate statsNo response Operating systemDebian Install methodDocker Compose Object DetectorCoral Network connectionWired Camera make and modelHikvision DS-KV6113-WPE1(B) Any other information that may be helpful |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 16 replies
-
|
sounds like you have already tried most things that I would suggest |
Beta Was this translation helpful? Give feedback.
-
|
What does it mean "fps unknown" though? Camera settings tell it's 25 but it doesn't seem to be stated in the stream metadata, does it? I've also tried to add -fpsmax to the go2rtc ffmpeg #video=h264... command, afair it wasn't any good Would update the doorbell firmware though the release notes don't mention anything codec related |
Beta Was this translation helpful? Give feedback.
-
Still issue after about 1hour 2024-07-19 08:16:01.178771989 [INFO] Preparing new go2rtc config... |
Beta Was this translation helpful? Give feedback.
-
I had major issues after it worked for a night: the out of sync went so badly the video is barely watchable, see screencast. The clock jumps, the car jumps, even the playhead jumps. Screencast.from.19.07.2024.09.31.23.webmThen I added just "#raw=-use_wallclock_as_timestamps 1": the video is ok as the audio but the audio lagged for a second in an hour. Currently testing this:
Where "async" means both "use_wallclock_as_timestamps" and ffmpeg's own "async" (which supposedly stands for "audio sync", not " asynchronous"). "opus" supposedly needed for the "webrtc" player to work, "aac" is needed for the frigate recorder to work/ It already has my "micropauses" at the live view, I'll see what it gives in the recordings in a couple of hours upd: with "#async" the snapshots somehow became distorted, but the audio lag didn't go. Will revert to "#raw=-use_wallclock_as_timestamps 1" for a while. |
Beta Was this translation helpful? Give feedback.
-
By the way I can't really comment the EOF error. I just realized I didn't watch my logs! Went straight to the actual recordings in the morning, saw the mess and started modifying the config edit: could be a GPU issue. I have and Nvidia card to do the encoding, thus I have "#hardware=cuda". Can't really say if I had EOFs too edit2: or a network issue! |
Beta Was this translation helpful? Give feedback.
-
|
Well, the udp advice seemed to help make the audio better. The #async switch somewhat does it's job keeping the lag tolerable for security purposes, though not fully eliminating it. It is ahead of the video or behind it at times, and sometimes even in sync. Sticking with this for a while: |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @fedd . I was facing the same issue and your config worked like a charm |
Beta Was this translation helpful? Give feedback.
-
|
If you are still in search for an answer, I have the same doorbell and had the same problems. What worked for me is instead of go2rtc handling the stream, you should frigate handle the stream. Here is my configuration: So as you can see frigate connects to the rtsp go2rtc restream "doorbell_sub" for detection, but records directly from: This takes away any lagg and works super stable. |
Beta Was this translation helpful? Give feedback.
-
|
Will this config provide two-way audio aswell? |
Beta Was this translation helpful? Give feedback.
-
|
Also, some note: no matter the config, having it through ethernet instead of WiFi makes a HUGE difference |
Beta Was this translation helpful? Give feedback.
Well, the udp advice seemed to help make the audio better. The #async switch somewhat does it's job keeping the lag tolerable for security purposes, though not fully eliminating it. It is ahead of the video or behind it at times, and sometimes even in sync. Sticking with this for a while: