GStreamer-devel

GES-Selecting Audio Stream

Classic

List

Threaded

7 messages Options

Chris Wine

GES-Selecting Audio Stream

Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.

--Chris

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thibault Saunier-4

Re: GES-Selecting Audio Stream

Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:

Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thibault Saunier, Igalia - www.igalia.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Chris Wine

Re: GES-Selecting Audio Stream

Hi Thibault,

I am using 1.18.1 on Windows, the x64 mingw version. That code does look like it should be doing the correct thing; however all the audio pads on the uridecodebin are still being created in my case. Here's a piece of the graph that shows the uridecode bin in the GESAudioURISource connecting to the volume adjustment bin: https://imgur.com/a/JpBDBV9

--Chris

On Thu, Dec 3, 2020 at 10:20 AM Thibault Saunier <[hidden email]> wrote:

Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:
Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Chris Wine

Re: GES-Selecting Audio Stream

Here is my select-tracks-for-object handler: https://pastebin.com/jTW1ZCgF

The main idea is that I know the integer index of the audio stream I want (clip_data->target_stream), so I loop through the audio streams to find the stream_id based on the index. Once I know that stream_id, the timeline's track will only be returned if the stream_id of the track element matches.

--Chris

On Thu, Dec 3, 2020 at 11:32 AM Chris Wine <[hidden email]> wrote:

Hi Thibault,

I am using 1.18.1 on Windows, the x64 mingw version. That code does look like it should be doing the correct thing; however all the audio pads on the uridecodebin are still being created in my case. Here's a piece of the graph that shows the uridecode bin in the GESAudioURISource connecting to the volume adjustment bin: https://imgur.com/a/JpBDBV9

--Chris

On Thu, Dec 3, 2020 at 10:20 AM Thibault Saunier <[hidden email]> wrote:
Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:
Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thibault Saunier-4

Re: GES-Selecting Audio Stream

Hi,

I do not see what is wrong. Could you open an issue with a minimal test case I could use to reproduce?

Thanks,

On Thu, Dec 3, 2020 at 4:39 PM Chris Wine <[hidden email]> wrote:

Here is my select-tracks-for-object handler: https://pastebin.com/jTW1ZCgF

The main idea is that I know the integer index of the audio stream I want (clip_data->target_stream), so I loop through the audio streams to find the stream_id based on the index. Once I know that stream_id, the timeline's track will only be returned if the stream_id of the track element matches.

--Chris

On Thu, Dec 3, 2020 at 11:32 AM Chris Wine <[hidden email]> wrote:
Hi Thibault,

I am using 1.18.1 on Windows, the x64 mingw version. That code does look like it should be doing the correct thing; however all the audio pads on the uridecodebin are still being created in my case. Here's a piece of the graph that shows the uridecode bin in the GESAudioURISource connecting to the volume adjustment bin: https://imgur.com/a/JpBDBV9

--Chris

On Thu, Dec 3, 2020 at 10:20 AM Thibault Saunier <[hidden email]> wrote:
Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:
Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thibault Saunier, Igalia - www.igalia.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Chris Wine

Re: GES-Selecting Audio Stream

Sure, I will work on that, thank you. What is the best way to submit the related video file, if that turns out to be important?

On Thu, Dec 3, 2020 at 1:28 PM Thibault Saunier <[hidden email]> wrote:

Hi,

I do not see what is wrong. Could you open an issue with a minimal test case I could use to reproduce?

Thanks,

On Thu, Dec 3, 2020 at 4:39 PM Chris Wine <[hidden email]> wrote:
Here is my select-tracks-for-object handler: https://pastebin.com/jTW1ZCgF

The main idea is that I know the integer index of the audio stream I want (clip_data->target_stream), so I loop through the audio streams to find the stream_id based on the index. Once I know that stream_id, the timeline's track will only be returned if the stream_id of the track element matches.

--Chris

On Thu, Dec 3, 2020 at 11:32 AM Chris Wine <[hidden email]> wrote:
Hi Thibault,

I am using 1.18.1 on Windows, the x64 mingw version. That code does look like it should be doing the correct thing; however all the audio pads on the uridecodebin are still being created in my case. Here's a piece of the graph that shows the uridecode bin in the GESAudioURISource connecting to the volume adjustment bin: https://imgur.com/a/JpBDBV9

--Chris

On Thu, Dec 3, 2020 at 10:20 AM Thibault Saunier <[hidden email]> wrote:
Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:
Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thibault Saunier-4

Re: GES-Selecting Audio Stream

If it is not too big, just attach on gitlab. Otherwise, wherever you want :)

Le jeu. 3 déc. 2020 à 19:14, Chris Wine <[hidden email]> a écrit :

Sure, I will work on that, thank you. What is the best way to submit the related video file, if that turns out to be important?

On Thu, Dec 3, 2020 at 1:28 PM Thibault Saunier <[hidden email]> wrote:
Hi,

I do not see what is wrong. Could you open an issue with a minimal test case I could use to reproduce?

Thanks,

On Thu, Dec 3, 2020 at 4:39 PM Chris Wine <[hidden email]> wrote:
Here is my select-tracks-for-object handler: https://pastebin.com/jTW1ZCgF

The main idea is that I know the integer index of the audio stream I want (clip_data->target_stream), so I loop through the audio streams to find the stream_id based on the index. Once I know that stream_id, the timeline's track will only be returned if the stream_id of the track element matches.

--Chris

On Thu, Dec 3, 2020 at 11:32 AM Chris Wine <[hidden email]> wrote:
Hi Thibault,

I am using 1.18.1 on Windows, the x64 mingw version. That code does look like it should be doing the correct thing; however all the audio pads on the uridecodebin are still being created in my case. Here's a piece of the graph that shows the uridecode bin in the GESAudioURISource connecting to the volume adjustment bin: https://imgur.com/a/JpBDBV9

--Chris

On Thu, Dec 3, 2020 at 10:20 AM Thibault Saunier <[hidden email]> wrote:
Hello,

What you are doing is correct, and it used to be broken but should work just fine in 1.18 thanks to:

https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/commit/09cc8dde0036420932704631f59531a1d84de018

What version of GStreamer are you using?

Regards,

- Thibault

On Wed, Dec 2, 2020 at 9:40 PM Chris Wine <[hidden email]> wrote:
Hi, I'm having a problem with selecting the correct audio stream from an input file that I'm using with GES. The input file has 1 video stream and 4 identical (as far as the CAPS are concerned) audio streams. I am currently using GStreamer/GES mingw x64 1.18.1 on the Windows platform.

As the documentation suggested, I wrote a signal handler for the GESTimeline's "select-tracks-for-object" signal, and I've verified that it is working as expected with returning a NULL for GESTrackElements that are not the audio stream I want, while returning a GPtrArray with the timeline's audio track when it is the audio stream I want.

The problem though is that only the 1st audio stream of the input file is ever chosen for the output. I've been looking through the GES source code, and it appears that when ges_source_create_topbin in GESSource is called for the audio, it will always just look for the first pad created when the "pad-added" signal is on uridecodebin is called; this made me wonder if something in the creation of uridecodebin itself was supposed to be responsible for selecting the correct audio stream. However, the only thing I can find there is the "caps" property which does restrict the streams, but since my 4 audio streams all have identical CAPS, this doesn't end up being a restriction.

My question is first, am I missing something obvious that would make this work as expected? If not, aside from code changes to GESSource/UriSource/AudioUriSource, is there something tricky I can do like finding the uridecodebin, intercepting the pad-added signal before it gets to the GESSource, and only letting it through when getting to the pad that I want to be connected?

Thank you, responses are much appreciated.
--Chris
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Thibault Saunier, Igalia - www.igalia.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel