Handling branches with different latencies

[Baby Octopus]

Hi,

I have a tricky pipeline which has two branches

src ! tee name=splitter \
splitter.! queue ! x264enc tune="zerolatency" ! sink1
splitter.! queue ! x264enc ! sink2

I want the first branch to have minimal latency. But both the branches seem
to take max latency of 2nd branch. Is there a way to handle each brnahc to
operate in its least latency and other branches not affect it?

This may be worked around by setting negative ts-offset in sink1. Is there a
better way of handling this?

~BO



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[HermannSW]

Hi,

so you want different speed branches read from same input stream.
I am not sure whether gstreamer can be configured for that, but if, that
would blow up your memory.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Sun, 2017-11-26 at 23:09 -0700, Baby Octopus wrote:

Hi,

> I have a tricky pipeline which has two branches
>
> src ! tee name=splitter \
> splitter.! queue ! x264enc tune="zerolatency" ! sink1
> splitter.! queue ! x264enc ! sink2
>
> I want the first branch to have minimal latency. But both the
> branches seem to take max latency of 2nd branch. Is there a way to
> handle each branch to operate in its least latency and other branches
> not affect it?
>
> This may be worked around by setting negative ts-offset in sink1. Is
> there a better way of handling this?

Depends a bit on the details. Is this a live pipeline (what is the
source element) or not?

If it's not a live pipeline and the source can produce data faster than
real time, then the solution is to make the queue before the encoder in
the second branch larger, like

  queue max-size-time=5000000000 max-size-buffers=0 max-size-bytes=0

By default the queue max size is ca. 1 second, and x264enc will consume
ca. 2-3 seconds of video before outputting anything, so that may
throttle the other branch (if it prerolls at all) because your queue in
branch 2 will run full and block.

If it's a live pipeline, then all sinks will configure to the MAX of
all the sinks' min latencies. I don't think you can override that in
gst-launch, but you can in code e.g. by sending latency events.

Cheers
 -Tim


--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Baby Octopus]

Hello Tim,

Yes, it is a live pipeline. Src is v4l2src and sink is udpsink and the
gst-launch pipeline was for representation.

I understand that the pipeline sets MAX(all_sink_min_latency) as the global
latency. Is it possible to have a varying latency for each branch and have
different sink render with different latency?

> v4l2src ! tee name=splitter \
> splitter.! queue ! x264enc tune="zerolatency" ! sink1
> splitter.! queue ! x264enc ! sink2

Here, in the absence of 2nd branch(last line), 1st branch(2nd line) would
have had only 100ms latency. But due to branch2, sink1 too has a latency of
1700ms. With both branches being present, is it possible to make sink1 still
render with latency=100ms? What are the steps achieve that?

Thanks in advance

~BO





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 27 novembre 2017 à 06:10 -0700, Baby Octopus a écrit :

> I understand that the pipeline sets MAX(all_sink_min_latency) as the global
> latency. Is it possible to have a varying latency for each branch and have
> different sink render with different latency?
>
> > v4l2src ! tee name=splitter \
> > splitter.! queue ! x264enc tune="zerolatency" ! sink1
> > splitter.! queue ! x264enc ! sink2
>
> Here, in the absence of 2nd branch(last line), 1st branch(2nd line) would
> have had only 100ms latency. But due to branch2, sink1 too has a latency of
> 1700ms. With both branches being present, is it possible to make sink1 still
> render with latency=100ms? What are the steps achieve that?

As Tim said, you need to increase the second queue size, as it might
not be able to hold the amount of buffering x264enc needs with it's
default settings.

On top of that, you probably want to set "async=false" on both sink1
and sink2, this way they won't wait for each other before starting. For
sink2, you'll probably want sync=false too, as you don't want it to
through away late buffers.

If instead you care about having a fixes latency on each sink, you can
implement do-latency callback on the pipeline (GstBin class signal).
This is pretty difficult, but would let you specify that sink1 has Nms
latency while sink2 has Mms latency.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Baby Octopus]

Thanks you Nicolas for your answer!

Alternatively speaking, can we not have a framework where in each
branch/sink of the pipeline will have its own latency based on its upstream
latency rather than max(all_sink_latency). Can we improvise on existing
mechanism where each branch operate without getting affected by parallel
branches or peer branches?


                                                   ______ queue ! enc1 !
sink1 -------BRANCH1
src ! filter1 ! tee name=split ------/
                                                 \______ queue ! enc2 !
sink2  -------BRANCH2


Lets assume queue sizes are set sufficiently to avoid deadlock. Technically
speaking, why should the presence of BRANCH2 affect BRANCH1's latency?

I assume this is more of design limitation. Kindly correct me if I'm wrong

~BO





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2017-11-27 at 22:00 -0700, Baby Octopus wrote:

Hi,

I wonder if you have actually tried my suggestion of using latency
events to distribute different latencies to the different sinks?


> I assume this is more of design limitation.

You can easily achieve what you want by setting sync=false on the
sink(s), then the data will be sent out as soon as possible.

The purpose of the existing mechanism is to maintain correct timing and
synchronisation between branches. Which you can switch off if you're
not interested in that.

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel