decklinksrc pipeline optimization

[jpoet]

When using decklinksrc to capture to a transport stream file, it typically drops a couple of frames per second:

5:08:39.613148080 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:38.003250892 to 5:08:38.003250892
5:08:39.846702910 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:38.236822745 to 5:08:38.236822745
5:08:40.213862509 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:38.603868288 to 5:08:38.603868288
5:08:41.014465627 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:39.404694929 to 5:08:39.404694929
5:08:41.615139285 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:40.005314910 to 5:08:40.005314910
5:08:42.349376088 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:40.739405998 to 5:08:40.739405998
5:08:42.883078722 30644 0x7fc74404c070 WARN        decklinkvideosrc gstdecklinkvideosrc.cpp:868:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Dropped 1 old frames from 5:08:41.273290426 to 5:08:41.273290426

I have played around with queue to try to eliminate those drops, but all I have achieved is to delay them.  Once they start up, they never stop.  My most recent attempt is:

gst-launch-1.0 -v decklinkvideosrc device-number=${DEVICE} mode=12 do-timestamp=true output-cc=true video-format=2 duplex-mode=0 ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! autovideoconvert ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! nvh264enc ! h264parse ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! mpegtsmux name=mux ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! filesink location=/storage1/SDI/video-nv.ts      decklinkaudiosrc device-number=${DEVICE} ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! audioconvert ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! avenc_ac3 bitrate=480000 ! ac3parse ! queue max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! mux.

I feel like I must be handing this wrong.  Can someone give me some advise on how to optimize this pipeline?  Note: my eventual goal, is to include captions.

Thank you,

John

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Mon, 2019-04-01 at 14:28 -0600, John P Poet wrote:

> When using decklinksrc to capture to a transport stream file, it
> typically drops a couple of frames per second:
>
> [...]
>
> I have played around with queue to try to eliminate those drops, but
> all I have achieved is to delay them.  Once they start up, they never
> stop.  My most recent attempt is:
>
> gst-launch-1.0 -v decklinkvideosrc device-number=${DEVICE} mode=12
> do-timestamp=true output-cc=true video-format=2 duplex-mode=0 ! queue
> max-size-buffers=4096 max-size-bytes=0 max-size-time=0 !
> autovideoconvert ! queue max-size-buffers=4096 max-size-bytes=0 max-
> size-time=0 ! nvh264enc ! h264parse ! queue max-size-buffers=4096
> max-size-bytes=0 max-size-time=0 ! mpegtsmux name=mux ! queue max-
> size-buffers=4096 max-size-bytes=0 max-size-time=0 ! filesink
> location=/storage1/SDI/video-nv.ts      decklinkaudiosrc device-
> number=${DEVICE} ! queue max-size-buffers=4096 max-size-bytes=0 max-
> size-time=0 ! audioconvert ! queue max-size-buffers=4096 max-size-
> bytes=0 max-size-time=0 ! avenc_ac3 bitrate=480000 ! ac3parse ! queue
> max-size-buffers=4096 max-size-bytes=0 max-size-time=0 ! mux.
>
> I feel like I must be handing this wrong.  Can someone give me some
> advise on how to optimize this pipeline?  Note: my eventual goal, is
> to include captions.

Considering your pipeline, your problem is most likely that downstream
of the Decklink sources can't handle the data in real-time. For example
the conversion from v210 to NV12 for nvh264enc might be too expensive
to run on a single core on your machine.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Prendergast]

I am facing the same issue with an UltraStudio HD Mini and I'm unsure it is
caused by a lack of system resources.


runs successfully, utilising ~13% CPU monitored by htop, on an 8 core
MacbookPro with no cores maxing out.


is unsuccessful, constantly logging
gstdecklinkvideosrc.cpp:897:gst_decklink_video_src_got_frame:<decklinkvideosrc0>
Dropped 1 old frames from X to X

And the output gets stuck on the first frame.

I have tried changing CPU priority:
 - The first pipeline with minimum priority still runs smoothly.
 - The second pipeline with maximum priority still drops frames.

Is there something else I need to increase to allow it to use more system
resources?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Prendergast]

Apologies, the code didn't attach in the previous post.

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  !
queue2 ! decklinkvideosink mode=1080p5994

Runs successfully, where:

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  !
tee name=t ! decklinkvideosink mode=1080p5994 t. ! fakesink

logs frame drop messages constantly



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Ray Tiley]

On Mon, Oct 7, 2019 at 3:49 PM wunwunforever <[hidden email]> wrote:

Apologies, the code didn't attach in the previous post.

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  !
queue2 ! decklinkvideosink mode=1080p5994

Runs successfully, where:

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  !
tee name=t ! decklinkvideosink mode=1080p5994 t. ! fakesink

logs frame drop messages constantly

How about:

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  !

tee name=t ! queue ! decklinkvideosink mode=1080p5994 t. ! queue ! fakesink sync=true

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Prendergast]

That works - thanks for the help.

Are you able to give insight into why it works? I'm still encountering the "Dropped 1 old frame" message when I try to do more complex things with the pipeline like, for example adding an audiosrc and mixing a testsrc

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  ! tee name=t ! queue ! decklinkvideosink mode=1080p5994 t. ! queue ! fakesink sync=true decklinkaudiosrc ! queue ! mix. audiotestsrc freq=100 ! queue ! mix. audiomixer name=mix ! queue ! decklinkaudiosink 

or adding a fixed delay to the video frames:

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true mode=1080p5994  ! tee name=t ! queue min-threshold-time=2000000000 max-size-bytes=1073741824 max-size-buffers=10000000 ! decklinkvideosink mode=1080p5994 t. ! queue ! fakesink sync=true

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel