render-delay base sink property for lip-sync?

[pl bossart]

By reading the code for an unrelated problem I accidentely came across
the 'render-delay' property in GstBaseSink. Looks to me that this
could be useful for lip-sync issues, as the audio could be delayed to
match video processing (HDMI) and vice-versa (BT A2DP audio out). Two
questions:
- Has anyone has ever used it in a real app? Any pointers would be
appreciated, couldn't find much info with google
- This property looks static/predefined. Somehow it'd need to be
defined by the time delta on the audio and display sides, using
information such as EDID, which can vary depending on the outputs. Has
anyone looked into this?
Thanks
- Pierre

------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

pl bossart wrote:

> By reading the code for an unrelated problem I accidentely came across
> the 'render-delay' property in GstBaseSink. Looks to me that this
> could be useful for lip-sync issues, as the audio could be delayed to
> match video processing (HDMI) and vice-versa (BT A2DP audio out). Two
> questions:
> - Has anyone has ever used it in a real app? Any pointers would be
> appreciated, couldn't find much info with google
> - This property looks static/predefined. Somehow it'd need to be
> defined by the time delta on the audio and display sides, using
> information such as EDID, which can vary depending on the outputs. Has
> anyone looked into this?
> Thanks
>  

A sink subclassing basesink can call
gst_base_sink_set_render_delay() when the latency towards the backend has changed. This posts a latency message to the application (bus) and updates the sink latency.
The new latency is used in gst_base_sink_do_sync() which is used by gst_base_sink_render_object() and that finaly is used from the chin function.

I think it should work fine :)

Stefan

> - Pierre
>
> ------------------------------------------------------------------------------
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for Conference
> attendees to learn about information security's most important issues through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>  

------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[pl bossart]

> A sink subclassing basesink can call
> gst_base_sink_set_render_delay() when the latency towards the backend has changed. This posts a latency message to the application (bus) and updates the sink latency.
> The new latency is used in gst_base_sink_do_sync() which is used by gst_base_sink_render_object() and that finaly is used from the chin function.
>
> I think it should work fine :)

Thanks Stefan. What I was thinking of is indeed to call
gst_base_sink_set_render_delay() when pulseaudio updates the latency
information in gst_pulsering_stream_latency_cb(). However I am not
sure the sink latency (the time needed for a new sample to be played)
is compatible with the gstreamer expectations at the pipeline level.
The latency reported by pulseaudio is initially zero in a prerolled
state, and it'll increase during the first seconds until the reported
latency oscillates around a stable value.

Along the same line, pulseaudio updates the latency every second or
so, not sure what the effect is if the pipeline latency is changed too
often.

Last, do video sinks update the render-delay information as well? If
only the audio sink updates the back-end delay, we might end-up with a
worse experience...
Thanks for your feedback
- Pierre

------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[pl bossart]

> Thanks Stefan. What I was thinking of is indeed to call
> gst_base_sink_set_render_delay() when pulseaudio updates the latency
> information in gst_pulsering_stream_latency_cb().

This is what I had in mind. Seems to update the pipeline latency every
second or so.
-Pierre

------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel