[QUESTION] All my frames are getting dropped I think and data upload rate is very low (rtsp H265 at 1020*1080) #1137

msalmonw · 2024-01-26T09:40:56Z

msalmonw
Jan 26, 2024

Im using a Dahua rtsp camera to stream to kvssink but I do not succeed. If im using h264 on the camera settings than I get "Bus Error". I switched to x265 at 1920x1080 but then again my frames are getting dropped and data upload is very very slow.

** Logging **

gatewise@devomech:~/Desktop/aws-certs $ gst-launch-1.0 rtspsrc location=rtsp://admin:[email protected]:554 ! rtph265depay ! h265parse ! video/x-h265,stream-format=byte-stream,width=1920,height=1080 ! kvssink stream-name="$STREAM_NAME" aws-region="$AWS_REGION" access-key="$AWS_ACCESS_KEY_ID" secret-key="$AWS_SECRET_ACCESS_KEY"
Setting pipeline to PAUSED ...
log4cplus:ERROR could not open file ../kvs_log_configuration
INFO - Logger config being used: ../kvs_log_configuration
2024-01-26 03:34:44 [4157881344] INFO - Setting session token from env for gatewise-camera-01
2024-01-26 03:34:44 [4157881344] INFO - User agent string: AWS-SDK-KVS-CPP-CLIENT/3.4.1
2024-01-26 03:34:44 [4157881344] INFO - createKinesisVideoClient(): Creating Kinesis Video Client
2024-01-26 03:34:44 [4157881344] INFO - heapInitialize(): Initializing native heap with limit size 134217728, spill ratio 0% and flags 0x00000001
2024-01-26 03:34:44 [4157881344] INFO - heapInitialize(): Creating AIV heap.
2024-01-26 03:34:44 [4157881344] INFO - heapInitialize(): Heap is initialized OK
2024-01-26 03:34:44 [4157881344] DEBUG - getSecurityTokenHandler invoked
2024-01-26 03:34:44 [4157881344] DEBUG - Refreshing credentials. Force refreshing: 0 Now time is: 1706261684760696328 Expiration: 0
2024-01-26 03:34:44 [4157881344] INFO - createDeviceResultEvent(): Create device result event.
2024-01-26 03:34:44 [4157881344] DEBUG - clientReadyHandler invoked
2024-01-26 03:34:44 [4157881344] INFO - Try creating stream for gatewise-camera-01
2024-01-26 03:34:44 [4157881344] INFO - Creating Kinesis Video Stream gatewise-camera-01
2024-01-26 03:34:44 [4157881344] INFO - createKinesisVideoStream(): Creating Kinesis Video Stream.
2024-01-26 03:34:44 [4157881344] INFO - logStreamInfo(): SDK version: de4299f942bb89c867d0dc73aeb4bc875d560a8f
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo(): Kinesis Video Stream Info
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Stream name: gatewise-camera-01
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Streaming type: STREAMING_TYPE_REALTIME
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Content type: video/h265
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Max latency (100ns): 600000000
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Fragment duration (100ns): 20000000
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Key frame fragmentation: Yes
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Use frame timecodes: Yes
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Absolute frame timecodes: Yes
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Nal adaptation flags: 0
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Average bandwidth (bps): 4194304
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Framerate: 25
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Buffer duration (100ns): 1200000000
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Replay duration (100ns): 400000000
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Connection Staleness duration (100ns): 600000000
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Store Pressure Policy: 1
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       View Overflow Policy: 1
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Allow stream creation: Yes
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Segment UUID: NULL
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Frame ordering mode: 0
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo(): Track list
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Track id: 1
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Track name: kinesis_video
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Codec id: V_MPEGH/ISO/HEVC
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Track type: TRACK_INFO_TYPE_VIDEO
2024-01-26 03:34:44 [4157881344] DEBUG - logStreamInfo():       Track cpd: NULL
2024-01-26 03:34:44 [4157881344] INFO - writeHeaderCallback(): RequestId: 588e02d0-9e9e-4a71-ab87-df8e6f7a1c2e
2024-01-26 03:34:45 [3941057600] DEBUG - describeStreamCurlHandler(): [] DescribeStream API response: {"StreamInfo":{"CreationTime":1.706103516332E9,"DataRetentionInHours":24,"DeviceName":null,"IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:us-west-2:571292287453:alias/aws/kinesisvideo","MediaType":null,"Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:us-west-2:571292287453:stream/gatewise-camera-01/1706103516332","StreamName":"gatewise-camera-01","Version":"gcae41F4MPYEmwAhLGzl"}}
2024-01-26 03:34:45 [3941057600] INFO - describeStreamResultEvent(): Describe stream result event.
2024-01-26 03:34:45 [3941057600] WARN - describeStreamResult(): Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo
2024-01-26 03:34:45 [3941057600] WARN - describeStreamResult(): Content type returned from the DescribeStream(null) call doesn't match the one specified in the StreamInfo(video/h265)
2024-01-26 03:34:45 [3941057600] WARN - describeStreamResult(): [gatewise-camera-01] Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo
2024-01-26 03:34:45 [3941057600] WARN - describeStreamResult(): [gatewise-camera-01] Content type returned from the DescribeStream call doesn't match the one specified in the StreamInfo
2024-01-26 03:34:45 [3941057600] INFO - writeHeaderCallback(): RequestId: 2dc67329-bf3d-4763-ac48-cbbc1c96cd15
2024-01-26 03:34:45 [3931108416] DEBUG - getStreamingEndpointCurlHandler(): [gatewise-camera-01] GetStreamingEndpoint API response: {"DataEndpoint":"https://s-8e923eff.kinesisvideo.us-west-2.amazonaws.com"}
2024-01-26 03:34:45 [3931108416] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event.
2024-01-26 03:34:45 [3931108416] DEBUG - getStreamingTokenHandler invoked
2024-01-26 03:34:45 [3931108416] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1706261685455260420 Expiration: 18446744073709551615
2024-01-26 03:34:45 [3931108416] INFO - getStreamingTokenResultEvent(): Get streaming token result event.
2024-01-26 03:34:45 [3931108416] DEBUG - streamReadyHandler invoked
2024-01-26 03:34:45 [3931108416] Stream is ready
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://admin:[email protected]:554
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
DEBUG - Dropping frame with flag: 1600 for gatewise-camera-01
2024-01-26 03:34:45 [3826230336] INFO - putStreamResultEvent(): Put stream result event. New upload handle 0
2024-01-26 03:34:45 [3826230336] WARN - notifyDataAvailable(): [gatewise-camera-01] Failed to un-pause curl with error: 43. Curl object 0xe912d058
DEBUG - postReadCallback(): [gatewise-camera-01] Pausing CURL read for upload handle: 0
2024-01-26 03:34:46 [3817837632] INFO - writeHeaderCallback(): RequestId: e31f15ce-c9b1-14c9-bd32-6b3b244ba1ca
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
WARN - curlCompleteSync(): [gatewise-camera-01] curl perform failed for url https://s-8e923eff.kinesisvideo.us-west-2.amazonaws.com/putMedia with result Timeout was reached: Operation too slow. Less than 30 bytes/sec transferred the last 30 seconds
2024-01-26 03:35:16 [3817837632] WARN - curlCompleteSync(): [gatewise-camera-01] HTTP Error 0 : Response: (null)
Request URL: https://s-8e923eff.kinesisvideo.us-west-2.amazonaws.com/putMedia
Request Headers:
    Authorization: AWS4-HMAC-SHA256 Credential=ASIAYKA5MQHO2G5I2YWJ/20240126/us-west-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=834dac426271b73cafa9befdf3
2024-01-26 03:35:16 [3817837632] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: gatewise-camera-01 with upload handle: 0 exited. http status: 0
2024-01-26 03:35:16 [3817837632] WARN - putStreamCurlHandler(): [gatewise-camera-01] Stream with streamHandle 19711040 uploadHandle 0 has exited without triggering end-of-stream. Service call result: 599
2024-01-26 03:35:16 [3817837632] INFO - kinesisVideoStreamTerminated(): Stream 0x12cc440 terminated upload handle 0 with service call result 599.
2024-01-26 03:35:16 [3817837632] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 0
2024-01-26 03:35:16 [3817837632] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:16 [3817837632] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 1
2024-01-26 03:35:16 [3826230336] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:16 [3826230336] DEBUG - streamReadyHandler invoked
2024-01-26 03:35:16 [3826230336] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 2
2024-01-26 03:35:16 [3826230336] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:16 [3826230336] INFO - putStreamResultEvent(): Put stream result event. New upload handle 1
2024-01-26 03:35:16 [3826230336] WARN - notifyDataAvailable(): [gatewise-camera-01] Failed to un-pause curl with error: 43. Curl object 0xe9192800
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - postReadCallback(): [gatewise-camera-01] Pausing CURL read for upload handle: 1
2024-01-26 03:35:17 [3817837632] INFO - writeHeaderCallback(): RequestId: d349597f-5f32-1e33-8d64-266b0573fb02
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
WARN - curlCompleteSync(): [gatewise-camera-01] curl perform failed for url https://s-8e923eff.kinesisvideo.us-west-2.amazonaws.com/putMedia with result Timeout was reached: Operation too slow. Less than 30 bytes/sec transferred the last 30 seconds
2024-01-26 03:35:46 [3817837632] WARN - curlCompleteSync(): [gatewise-camera-01] HTTP Error 0 : Response: (null)
Request URL: https://s-8e923eff.kinesisvideo.us-west-2.amazonaws.com/putMedia
Request Headers:
    Authorization: AWS4-HMAC-SHA256 Credential=ASIAYKA5MQHO2G5I2YWJ/20240126/us-west-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=3a5ea1be51aa1daa96c1da610f
2024-01-26 03:35:46 [3817837632] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: gatewise-camera-01 with upload handle: 1 exited. http status: 0
2024-01-26 03:35:46 [3817837632] WARN - putStreamCurlHandler(): [gatewise-camera-01] Stream with streamHandle 19711040 uploadHandle 1 has exited without triggering end-of-stream. Service call result: 599
2024-01-26 03:35:46 [3817837632] INFO - kinesisVideoStreamTerminated(): Stream 0x12cc440 terminated upload handle 1 with service call result 599.
2024-01-26 03:35:46 [3817837632] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 3
2024-01-26 03:35:46 [3817837632] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:46 [3817837632] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 4
2024-01-26 03:35:46 [3826230336] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:46 [3826230336] DEBUG - streamReadyHandler invoked
2024-01-26 03:35:46 [3826230336] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 5
2024-01-26 03:35:46 [3826230336] DEBUG - defaultStreamStateTransitionHook():
 KinesisVideoStream base result is [599]. Executing KVS retry handler of retry strategy type [1]
2024-01-26 03:35:46 [3826230336] INFO - putStreamResultEvent(): Put stream result event. New upload handle 2
2024-01-26 03:35:46 [3826230336] WARN - notifyDataAvailable(): [gatewise-camera-01] Failed to un-pause curl with error: 43. Curl object 0xe9192800
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - postReadCallback(): [gatewise-camera-01] Pausing CURL read for upload handle: 2
2024-01-26 03:35:52 [3817837632] INFO - writeHeaderCallback(): RequestId: d62e2efd-4549-f05d-8803-51caaa3edc07
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
DEBUG - Dropping frame with flag: 1536 for gatewise-camera-01
^Chandling interrupt.3826230336] 0:01:31.8 / 99:99:99.

Please help!

jdelapla · 2024-01-29T17:31:13Z

jdelapla
Jan 29, 2024

if you replace kvssink with a test sink or a display sink of some kind does the pipeline function?

0 replies

msalmonw · 2024-01-29T18:00:02Z

msalmonw
Jan 29, 2024
Author

Yes it worked, I resolved the issue by compiling the with -DALIGNED_MEMORY_MODEL=TRUE.

Also now I have another problem, I can generate session tokens completely fine with my certificates, but I can not start the stream directly using the certificates. The command I use is:

gst-launch-1.0 -v rtspsrc location=rtsp://admin:[email protected]:554 short-header=TRUE ! rtph264depay ! h264parse ! kvssink stream-name=$"STREAM_NAME" iot-certificate="iot-certificate,endpoint=https://ctdaryo6b4e5l.credentials.iot.us-west-2.amazonaws.com,cert-path=/certificate.pem,key-path=/private.pem.key,ca-path=/cacert.pem,role-aliases=gatewise-camera-role-alias"
And the output:

log4cplus:ERROR could not open file ../kvs_log_configuration INFO - Logger config being used: ../kvs_log_configuration 2024-01-29 11:43:48 [4159011840] INFO - Setting session token from env for gatewise-camera-01 2024-01-29 11:43:48 [4159011840] INFO - Using iot credential provider within KVS sink for gatewise-camera-01 2024-01-29 11:43:48 [4159011840] INFO - User agent string: AWS-SDK-KVS-CPP-CLIENT/3.4.1 2024-01-29 11:43:48 [4159011840] DEBUG - Creating IoT auth callbacks. 2024-01-29 11:43:48 [4159011840] ERROR - Unable to create Iot Credential provider. Error status: 0x4000000e ERROR: from element /GstPipeline:pipeline0/GstKvsSink:kvssink0: Could not initialise supporting library. Additional debug info: /home/gatewise/Desktop/GateWiseCamera/amazon-kinesis-video-streams-producer-sdk-cpp/src/gstreamer/gstkvssink.cpp(1640): gst_kvs_sink_change_state (): /GstPipeline:pipeline0/GstKvsSink:kvssink0: Failed to init kvs producer. Error: Unable to create Iot Credential provider. Error status: 0x4000000e ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. 2024-01-29 11:43:48 [4159011840] Setting pipeline to NULL ... Freeing pipeline ...

0 replies

niyatim23 · 2024-02-07T13:53:12Z

niyatim23
Feb 7, 2024

The error code you're seeing is 0x4000000e: STATUS_DIRECTORY_ENTRY_STAT_ERROR. This could be something related to directory access probably. Can you double check all your paths and permissions?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[QUESTION] All my frames are getting dropped I think and data upload rate is very low (rtsp H265 at 1020*1080) #1137

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

[QUESTION] All my frames are getting dropped I think and data upload rate is very low (rtsp H265 at 1020*1080) #1137

Uh oh!

msalmonw Jan 26, 2024

Replies: 3 comments

Uh oh!

jdelapla Jan 29, 2024

Uh oh!

Uh oh!

msalmonw Jan 29, 2024 Author

Uh oh!

niyatim23 Feb 7, 2024

msalmonw
Jan 26, 2024

jdelapla
Jan 29, 2024

msalmonw
Jan 29, 2024
Author

niyatim23
Feb 7, 2024