Synchronized Pipelines

[Stirling Westrup]

I'm working on a project similar in intent to RidgeRun's gst-interpipe
elements.  http://ridgerun.github.io/gst-interpipe/
and I'm having huge difficulties with buffers being dropped due to
timestamp issues.

We have a number of 'sender' pipelines which are either live or
recorded sources, and which can come and go at random. Then we have a
number of 'receiver' pipelines that can be switched from one 'sender'
to another while running.

Currently we're implementing the sender pipelines by terminating them
with an element derived from GstBaseSink which basically broadcasts
buffers and events. The intent is for the senders to neither know nor
care if anyone is listening.

The source elements for the receiver pipelines have the job of tuning
into one of the senders and sending the received buffers and events
down the line.

We're treating the source elements as live sources and subclassing
GstBaseSrc for them. That's where everything seems to go wrong.

We need all the receivers connected to any one sender to all be
perfectly synchronized, so they need to somehow come up with the exact
same PTS for the buffers, based on a shared clock.

In particular, a receiver that connects to a sender 10-minutes after
another needs to be still generate the same final timestamps (relative
to the system clock).

As far as I can tell live sources each start their own timelines for
sending out buffers, based on their individual running times, and
anything we try to do to deviate from this runs afoul of assumptions
built into the base classes.

Do you have any recommendations of the best/easiest way to do what
we're doing? Should we not be modelling things as sinks and sources
but as pairs of connected transformational elements? At this point I'm
quite confused as to if I'm even approaching this correctly.

--
Stirling Westrup (he/him)
Programmer, Entrepreneur.
http://www.linkedin.com/in/swestrup
([phone]
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Fri, 2020-11-13 at 17:43 -0500, Stirling Westrup wrote:
>
> [...]
>
> Do you have any recommendations of the best/easiest way to do what
> we're doing? Should we not be modelling things as sinks and sources
> but as pairs of connected transformational elements? At this point I'm
> quite confused as to if I'm even approaching this correctly.

What happens sounds very much like you don't enforce the same base time
on all involved pipelines.

You can do so by doing the following *before* setting the pipelines to
PLAYING:

    gst_element_set_base_time(pipeline, X);
    gst_element_set_start_time(pipeline, GST_CLOCK_TIME_NONE);

In addition you also want to force the same clock on all pipelines:

    gst_pipeline_use_clock(pipeline, clock);


You could e.g. use a base time of 0 on all pipelines and the system
clock, or set the base time to the clock time when you start the very
first pipeline.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Gruner]

Adding to Sebastian’s advice: you can play with the “stream-sync” property. It will adjust buffer timestamps according to the individual pipeline base times.

> On 14 Nov 2020, at 04:03, Sebastian Dröge <[hidden email]> wrote:
>
> On Fri, 2020-11-13 at 17:43 -0500, Stirling Westrup wrote:
>>
>> [...]
>>
>> Do you have any recommendations of the best/easiest way to do what
>> we're doing? Should we not be modelling things as sinks and sources
>> but as pairs of connected transformational elements? At this point I'm
>> quite confused as to if I'm even approaching this correctly.
>
> What happens sounds very much like you don't enforce the same base time
> on all involved pipelines.
>
> You can do so by doing the following *before* setting the pipelines to
> PLAYING:
>
>    gst_element_set_base_time(pipeline, X);
>    gst_element_set_start_time(pipeline, GST_CLOCK_TIME_NONE);
>
> In addition you also want to force the same clock on all pipelines:
>
>    gst_pipeline_use_clock(pipeline, clock);
>
>
> You could e.g. use a base time of 0 on all pipelines and the system
> clock, or set the base time to the clock time when you start the very
> first pipeline.
>
> --
> Sebastian Dröge, Centricular Ltd · https://www.centricular.com
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stirling Westrup]

On Sat, Nov 14, 2020 at 5:03 AM Sebastian Dröge
<[hidden email]> wrote:

>
> On Fri, 2020-11-13 at 17:43 -0500, Stirling Westrup wrote:
> >
> What happens sounds very much like you don't enforce the same base time
> on all involved pipelines.
>
> You can do so by doing the following *before* setting the pipelines to
> PLAYING:
>
>     gst_element_set_base_time(pipeline, X);
>     gst_element_set_start_time(pipeline, GST_CLOCK_TIME_NONE);
>
> In addition you also want to force the same clock on all pipelines:
>
>     gst_pipeline_use_clock(pipeline, clock);
>

We can't do that because we need to keep the pipelines independent.
(ie, two of the senders might conceivably have different clocks).
Instead when a 'receiver' connects to a 'sender' its given the clock
of its sender to use, and each incoming mini-object is accompanied by
the base-time of the sending pipeline so adjustments can be made.

But connecting and disconnecting of pipelines has to be possible with
everything running, and already in the Playing state.


--
Stirling Westrup (he/him)
Programmer, Entrepreneur.
http://www.linkedin.com/in/swestrup
([phone]
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel