Hikvision FFMPEG questions

[maphilli14]

I'm running the following:

Hikvision DS-2CD2543G0-IS (firmware V5.6.6 build 210625)
with these parameters:

# ffprobe -rtsp_transport tcp rtsp://camera:Nsite12345@192.168.45.43:554/Streaming/Channels/1
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, rtsp, from 'rtsp://camera:Nsite12345@192.168.45.43:554/Streaming/Channels/1':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264, yuvj420p(pc, bt709, progressive), 2688x1520 [SAR 1:1 DAR 168:95], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Stream #0:1: Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s

and Frigate on a Win10 in docker on a Lenovo P1G2 laptop. I've tried both nvidia and stable-amd64 images.

Docker compose:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:stable-amd64
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
    volumes:
      - C:/Users/maphi/OneDrive/D-Permanent/Scripts/Docker/Frigate-nvidia/config.yml:/config/config.yml:ro
      - c:/media/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 2000000000 
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "blah"

config:

mqtt:
  # Required: host name
  host: 192.168.45.249
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  user: maphilli14
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  # Optional: interval in seconds for publishing stats (default: shown below)
  password: notgonnatellya
detectors:
  cpu1:
    type: cpu
    num_threads: 3
ffmpeg:
  hwaccel_args:
    - -c:v
    - h264_v4l2m2m
  input_args:
    - -avoid_negative_ts
    - make_zero
    - -fflags
    - nobuffer
    - -flags
    - low_delay
    - -strict
    - experimental
    - -fflags
    - +genpts+discardcorrupt
    - -rw_timeout
    - '200000000'
    - -use_wallclock_as_timestamps
    - '1'
cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: rtsp://camera:secret@192.168.45.43:554/Streaming/Channels/1
          roles:
            - detect
            - rtmp
            - record
      #hwaccel_args: -c:v h264_v4l2m2m
    detect:
      enabled: False
      width: 2688
      height: 1520
      fps: 30

No matter what parameters I've tried I seem to get this error a lot:

frigate  | [2021-10-22 15:48:43] frigate.video                  INFO    : Driveway: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate  | [2021-10-22 15:48:43] frigate.video                  INFO    : Driveway: ffmpeg process is not running. exiting capture thread...
frigate  | [2021-10-22 15:48:58] watchdog.Driveway              ERROR   : FFMPEG process crashed unexpectedly for Driveway.
frigate  | [2021-10-22 15:48:58] watchdog.Driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2021-10-22 15:48:58] watchdog.Driveway              ERROR   : You may have invalid args defined for this camera.
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : [swscaler @ 0x561985e4b900] deprecated pixel format used, make sure you did set range correctly
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : av_interleaved_write_frame(): Connection reset by peer
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : [flv @ 0x561984892dc0] Failed to update header with correct duration.
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : [flv @ 0x561984892dc0] Failed to update header with correct filesize.
frigate  | [2021-10-22 15:48:58] ffmpeg.Driveway.detect         ERROR   : Error writing trailer of rtmp://127.0.0.1/live/Driveway: Connection reset by peer
frigate  | [2021-10-22 15:49:03] frigate.video                  INFO    : Driveway: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate  | [2021-10-22 15:49:03] frigate.video                  INFO    : Driveway: ffmpeg process is not running. exiting capture thread...

I'm just very confused at this point what I should even expect to see on the localhost:5000 pages. Typically it's either green or just frozen. On the same laptop I have no issues in VLC:

and the video is super smooth on the same laptop hardware!

TIA,

Mike

[ozett]

i would make an issue.
and maybe check ffmpeg config for that cam

ffmpeg:
  hwaccel_args:
    - -c:v
    - h264_v4l2m2m

[blakeblackshear]

At that resolution and frame rate, the rtmp functionality is failing. Try adding this to you config:

rtmp:
  enabled: False

Also, I would re-read the docs on camera setup. You probably don't want to run detection at that resolution or frame rate unless you are trying to track F1 cars from the top of a skyscraper.

[F0rged]

So my Hikvision DS-2CD2347G2-LU was running pretty stable for a while now, but then today I started getting this issue.

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-07-27 01:27:51] frigate.app                    INFO    : Starting Frigate (0.10.1-83481af)
[2022-07-27 01:27:51] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-07-27 01:27:51] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-07-27 01:27:51] peewee_migrate                 INFO    : There is nothing to migrate
[2022-07-27 01:27:51] detector.coral                 INFO    : Starting detection process: 216
[2022-07-27 01:27:51] frigate.app                    INFO    : Output process started: 218
[2022-07-27 01:27:51] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2022-07-27 01:27:51] ws4py                          INFO    : Using epoll
[2022-07-27 01:27:51] frigate.app                    INFO    : Camera processor started for driveway_cam2: 221
[2022-07-27 01:27:51] frigate.app                    INFO    : Capture process started for driveway_cam2: 226
[2022-07-27 01:27:51] ws4py                          INFO    : Using epoll
[2022-07-27 01:27:54] frigate.edgetpu                INFO    : TPU found
[2022-07-27 01:28:49] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:60390]
[2022-07-27 01:32:21] watchdog.driveway_cam2         INFO    : No frames received from driveway_cam2 in 20 seconds. Exiting ffmpeg...
[2022-07-27 01:32:21] watchdog.driveway_cam2         INFO    : Waiting for ffmpeg to exit gracefully...
[2022-07-27 01:32:27] frigate.video                  ERROR   : driveway_cam2: Unable to read frames from ffmpeg process.
[2022-07-27 01:32:27] frigate.video                  ERROR   : driveway_cam2: ffmpeg process is not running. exiting capture thread...
[2022-07-27 01:32:37] watchdog.driveway_cam2         ERROR   : Ffmpeg process crashed unexpectedly for driveway_cam2.
[2022-07-27 01:32:37] watchdog.driveway_cam2         ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-27 01:32:37] ffmpeg.driveway_cam2.detect    ERROR   : [swscaler @ 0x5654dda13e80] deprecated pixel format used, make sure you did set range correctly
[2022-07-27 01:32:37] ffmpeg.driveway_cam2.detect    ERROR   : [rtsp @ 0x5654dcaf8c40] CSeq 12 expected, 11 received.

We are just going into a heat wave - we hit 37 degrees Celsius today, and my camera is mounted just under my soffit (likely absorbing both the heat from outside and the heat from the attic as well). Maybe related, maybe not...

I went into the camera and dropped the max bitrate on the stream used by frigate from 6144kbps to 1024kbps, but that didn't seem to solve it...

Have also tried recreating the image, and disabling the rtmp setting in frigate:

rtmp:
  enabled: False

No luck so far...

[F0rged]

Also the RTSP feed appears to run perfectly fine via VLC media player. Could it be my coral overheating?

[NickM-27]

The logs there the coral would have no affect on so that's not it. The logs don't show signs of RTMP issues either (unless these are after disabling that).

Could be heat related, does restarting the camera help? Also how quickly does this happen usually? Seems this time it was 5 minutes, did you watch VLC for 5-10 minutes straight to see if any errors / glitches occurred?

[F0rged]

I just tried switching to CPU detection, but as you suggested, no effect.
Seems like it craps out around 2 minutes in.

From VLC the stream still appears rock solid. Meanwhile when I try watch it from frigate, even when it is running its unstable, with the seconds counter inside the video feed lagging well behind VLC, sometimes pausing completely, and then attempting to catch up, only to slowly fall farther and farther behind.

[NickM-27]

Hmm, if it is the case that nothing has changed with the camera then I don't see why this behavior would occur unless it is system related and hwaccel (GPU) or CPU are struggling to decode

[maphilli14]

Works with an alternative stream now, thanks!!! Great package @blakeblackshear thanks for the help and hard work!