query about audio track switch with decodebin2 (single audio decode, not multiple decode as standard gstreamer)

[Nitin Mahajan]

Dear Edward, Jan and other experts,

I have a specific query. My demux plugin outputs video and audio packets (elementary streams) whereas hlsdemux outputs video and audio but in container. The gstreamer pipeline is running fine.

Now I want to switch audio track but number of pads would remain the same i.e. one video pad and one audio pad. And there is a change in audio codec. So that essentially means to have new audio chain.

As suggested by Edward Hervy on following links:

https://lwn.net/Articles/661403/ 

https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Edward%20Hervey%20-%20decodebin3.pdf

There are some known issues (explained by Edward in above links) that we can not just remove audio pad and add new audio pad. We would have to create new video and audio pad, push eos on old video pad and old audio pad, remove old video pad and remove old audio pad.And then start pushing packets on the new video pad and new audio pad. Currently my gstreamer version is 1.4.5 which is based on decodebin2.

In such scenario just before track change:

- significant video of future timestamps is already buffered in the video chain (old chain)

- significant audio of future timestamps is already buffered in the audio chain  (old chain)

Now when switching to push buffers on new pads, for audio case, we have capability to "rewind" the PTS as per the current presentation time. But for video case, the video packets would have timestamps which is much ahead as compared to audio. I'm not sure if hlsdemux also rewinds the video when it performs the track switch ? 

I think it will create problem at synchronisation stage when video is much ahead. 

How such issues are handled at hlsdemux or inside any other demux plugin  

which perform decode group switch ? Do they also "rewind" video before pushing on video chain ?

Do we need to also FLUSH old video and audio chains also before start pushing buffers on new video and audio pad ?

Now we also have a situation when there is no codec change we are trying to flush audio without removing audio pad and observing synchronization issues:

Please have a look at, http://gstreamer-devel.966125.n4.nabble.com/a-v-freeze-while-flushing-audio-chain-of-pipeline-in-running-state-clock-and-renderer-sync-related-td4683168.html

and suggest how we can address this behaviour.

In both cases we would not intend to flush video. Is there any specific requirement to flush video also to handle a/v freeze..

Kindly share your expert insights to help with decodebin2 based audio track switch with single audio pad that outputs elementary stream.

Thanks

--Nitin

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nitin Mahajan]

May request for pointer to any gstreamer based player implementation that handle audio track change with decodebin2 with single audio pad and single audio decode ?

[Baby Octopus]

1.4 is fairly old, about 3 years now. IS there a way you an use 1.12 or try the new decodebin3?

[Nitin Mahajan]

Thanks but moving to 1.12 is currently not easy due to some priorities. But we can analyze provided we have confirmed status based on behaviour with decodebin2 especially switching audio track use case with hlsdemux with decodebin2..does it download all streams from manifest and expose all audio pads and rely on switching with input selector ? Possibly no. Then in that case, does hlsdemux remove current video and current audio and the create new pads, push eos on old pads and then let it drain before it connects new decodegroup with playsink. In this process if it waits for drain then there would be long track switch latency... Also the timestamp of audio of switched track would be from behind then sync issues can't be avoided for the same video..is it correct understanding and how hlsdemux handles it or if there are known such limitations with hlsdemux and decodebin2. Looking forward to key pointers and insights from experts.

Thanks

On Jun 12, 2017 6:37 PM, "Baby Octopus" <[hidden email]> wrote:

1.4 is fairly old, about 3 years now. IS there a way you an use 1.12 or try
the new decodebin3?



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/query-about-audio-track-switch-with-decodebin2-single-audio-decode-not-multiple-decode-as-standard-g-tp4683271p4683308.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel