Switching from playbin to uridecodebin (glupload & audio)

[Gottfried Haider]

Hello all,

I have this pipeline as part of Processing's glvideo library, that
works great - but doesn't play audio:

uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
capsfilter name=filter ! fakesink name=vsink

So I tried this instead

uridecodebin uri=... name=decoder ! queue ! audioconvert !
audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
name=vsink sync=false

which just hang. (The particular mp4 file I was testing this with had
not audio track if that matters.)


Should the second pipeline work in theory? Any other things I could
try to get something like a playbin-like source element that would
also play audio tracks, if the source has one? I uploaded a dot file
of the second pipeline on OS X  for you to look at[1].

(If the pipeline looks right it might just be that the audio decoding
hits this bug [2] on OS X.)


Thanks in advance
Gottfried


[1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
[2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

On 16/09/16 20:10, Gottfried Haider wrote:

> Hello all,
>
> I have this pipeline as part of Processing's glvideo library, that
> works great - but doesn't play audio:
>
> uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
> capsfilter name=filter ! fakesink name=vsink
>
> So I tried this instead
>
> uridecodebin uri=... name=decoder ! queue ! audioconvert !
> audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
> name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
> name=vsink sync=false
>
> which just hang. (The particular mp4 file I was testing this with had
> not audio track if that matters.)
>
>
> Should the second pipeline work in theory? Any other things I could
> try to get something like a playbin-like source element that would
> also play audio tracks, if the source has one? I uploaded a dot file
> of the second pipeline on OS X  for you to look at[1].

If the file didn't have an audio track that means that the audio output
part of the pipeline would not be linked to uridecodebin and thus when
prerolling would never get a buffer and would never complete the state
change to paused.  A backtrace of all threads would probably show that
the audiosink is stuck on the preroll cond/lock.

Your options are to either use playbin with the video-sink property set
to your custom video branch or if you're set on using uridecodebin, you
will have to selectively add the audio part of the pipeline if there is
an audio pad exposed by uridecodebin.  See the following example which
uses decodebin but the same idea is valid for uridecodebin:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-components-decodebin.html.

Cheers
-Matt

> (If the pipeline looks right it might just be that the audio decoding
> hits this bug [2] on OS X.)
>
>
> Thanks in advance
> Gottfried
>
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
> [2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gottfried Haider]

Thanks Matt!

Regarding the playbin-option: Is it possible to create a "custom video branch" (GstBin) as part of a textual pipeline description? (that is being fed to gst_parse_launch)

Or do I need to create a new bin in code, set it up with glupload ! capsfilter ! fakesink (like in testegc [1]), and later use gst_pipeline_new() and gst_parse_bin_from_description() to create a custom pipeline that references the video bin created earlier?

Best

G

[1] https://cgit.freedesktop.org/gstreamer/gst-omx/tree/examples/egl/testegl.c#n1079

On Fri, Sep 16, 2016 at 2:26 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 20:10, Gottfried Haider wrote:
> Hello all,
>
> I have this pipeline as part of Processing's glvideo library, that
> works great - but doesn't play audio:
>
> uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
> capsfilter name=filter ! fakesink name=vsink
>
> So I tried this instead
>
> uridecodebin uri=... name=decoder ! queue ! audioconvert !
> audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
> name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
> name=vsink sync=false
>
> which just hang. (The particular mp4 file I was testing this with had
> not audio track if that matters.)
>
>
> Should the second pipeline work in theory? Any other things I could
> try to get something like a playbin-like source element that would
> also play audio tracks, if the source has one? I uploaded a dot file
> of the second pipeline on OS X  for you to look at[1].

If the file didn't have an audio track that means that the audio output
part of the pipeline would not be linked to uridecodebin and thus when
prerolling would never get a buffer and would never complete the state
change to paused.  A backtrace of all threads would probably show that
the audiosink is stuck on the preroll cond/lock.

Your options are to either use playbin with the video-sink property set
to your custom video branch or if you're set on using uridecodebin, you
will have to selectively add the audio part of the pipeline if there is
an audio pad exposed by uridecodebin.  See the following example which
uses decodebin but the same idea is valid for uridecodebin:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-components-decodebin.html.

Cheers
-Matt

> (If the pipeline looks right it might just be that the audio decoding
> hits this bug [2] on OS X.)
>
>
> Thanks in advance
> Gottfried
>
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
> [2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

On 16/09/16 22:53, Gottfried Haider wrote:

Thanks Matt!

Regarding the playbin-option: Is it possible to create a "custom video branch" (GstBin) as part of a textual pipeline description? (that is being fed to gst_parse_launch)

You can create a bin as the video-sink for gst-launch-1.0.  You just need to enclose the video-sink parameter in double quotes and a bin should be created with your elements inside.

e.g. gst-launch-1.0 playbin uri=file:///path/to/file video-sink="glupload ! glcolorconvert ! glfiltercube ! glimagesinkelement" will cubify the video.

Cheers
-Matt

Or do I need to create a new bin in code, set it up with glupload ! capsfilter ! fakesink (like in testegc [1]), and later use gst_pipeline_new() and gst_parse_bin_from_description() to create a custom pipeline that references the video bin created earlier?

Best

G

[1] https://cgit.freedesktop.org/gstreamer/gst-omx/tree/examples/egl/testegl.c#n1079

On Fri, Sep 16, 2016 at 2:26 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 20:10, Gottfried Haider wrote:
> Hello all,
>
> I have this pipeline as part of Processing's glvideo library, that
> works great - but doesn't play audio:
>
> uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
> capsfilter name=filter ! fakesink name=vsink
>
> So I tried this instead
>
> uridecodebin uri=... name=decoder ! queue ! audioconvert !
> audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
> name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
> name=vsink sync=false
>
> which just hang. (The particular mp4 file I was testing this with had
> not audio track if that matters.)
>
>
> Should the second pipeline work in theory? Any other things I could
> try to get something like a playbin-like source element that would
> also play audio tracks, if the source has one? I uploaded a dot file
> of the second pipeline on OS X  for you to look at[1].

If the file didn't have an audio track that means that the audio output
part of the pipeline would not be linked to uridecodebin and thus when
prerolling would never get a buffer and would never complete the state
change to paused.  A backtrace of all threads would probably show that
the audiosink is stuck on the preroll cond/lock.

Your options are to either use playbin with the video-sink property set
to your custom video branch or if you're set on using uridecodebin, you
will have to selectively add the audio part of the pipeline if there is
an audio pad exposed by uridecodebin.  See the following example which
uses decodebin but the same idea is valid for uridecodebin:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-components-decodebin.html.

Cheers
-Matt

> (If the pipeline looks right it might just be that the audio decoding
> hits this bug [2] on OS X.)
>
>
> Thanks in advance
> Gottfried
>
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
> [2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gottfried Haider]

Thanks, that's great!

One last problem (I hope): how would I be able to programmatically get those elements in the video-sink bin?

gst_bin_get_by_name() on the main pipeline does not work anymore, e.g. when searching for the glupload, even though the function supposedly also looks into child bins.

Also tried iterating over sinks with gst_bin_iterate_sink() and look for my glupload there, but didn't seem to work.

G

On Fri, Sep 16, 2016 at 3:05 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 22:53, Gottfried Haider wrote:

Thanks Matt!

Regarding the playbin-option: Is it possible to create a "custom video branch" (GstBin) as part of a textual pipeline description? (that is being fed to gst_parse_launch)

You can create a bin as the video-sink for gst-launch-1.0.  You just need to enclose the video-sink parameter in double quotes and a bin should be created with your elements inside.

e.g. gst-launch-1.0 playbin uri=file:///path/to/file video-sink="glupload ! glcolorconvert ! glfiltercube ! glimagesinkelement" will cubify the video.

Cheers
-Matt

Or do I need to create a new bin in code, set it up with glupload ! capsfilter ! fakesink (like in testegc [1]), and later use gst_pipeline_new() and gst_parse_bin_from_description() to create a custom pipeline that references the video bin created earlier?

Best

G

[1] https://cgit.freedesktop.org/gstreamer/gst-omx/tree/examples/egl/testegl.c#n1079

On Fri, Sep 16, 2016 at 2:26 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 20:10, Gottfried Haider wrote:
> Hello all,
>
> I have this pipeline as part of Processing's glvideo library, that
> works great - but doesn't play audio:
>
> uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
> capsfilter name=filter ! fakesink name=vsink
>
> So I tried this instead
>
> uridecodebin uri=... name=decoder ! queue ! audioconvert !
> audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
> name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
> name=vsink sync=false
>
> which just hang. (The particular mp4 file I was testing this with had
> not audio track if that matters.)
>
>
> Should the second pipeline work in theory? Any other things I could
> try to get something like a playbin-like source element that would
> also play audio tracks, if the source has one? I uploaded a dot file
> of the second pipeline on OS X  for you to look at[1].

If the file didn't have an audio track that means that the audio output
part of the pipeline would not be linked to uridecodebin and thus when
prerolling would never get a buffer and would never complete the state
change to paused.  A backtrace of all threads would probably show that
the audiosink is stuck on the preroll cond/lock.

Your options are to either use playbin with the video-sink property set
to your custom video branch or if you're set on using uridecodebin, you
will have to selectively add the audio part of the pipeline if there is
an audio pad exposed by uridecodebin.  See the following example which
uses decodebin but the same idea is valid for uridecodebin:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-components-decodebin.html.

Cheers
-Matt

> (If the pipeline looks right it might just be that the audio decoding
> hits this bug [2] on OS X.)
>
>
> Thanks in advance
> Gottfried
>
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
> [2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gottfried Haider]

Nevermind, just got it to work!

On Fri, Sep 16, 2016 at 3:42 PM, Gottfried Haider <[hidden email]> wrote:

Thanks, that's great!

One last problem (I hope): how would I be able to programmatically get those elements in the video-sink bin?

gst_bin_get_by_name() on the main pipeline does not work anymore, e.g. when searching for the glupload, even though the function supposedly also looks into child bins.

Also tried iterating over sinks with gst_bin_iterate_sink() and look for my glupload there, but didn't seem to work.

G

On Fri, Sep 16, 2016 at 3:05 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 22:53, Gottfried Haider wrote:

Thanks Matt!

Regarding the playbin-option: Is it possible to create a "custom video branch" (GstBin) as part of a textual pipeline description? (that is being fed to gst_parse_launch)

You can create a bin as the video-sink for gst-launch-1.0.  You just need to enclose the video-sink parameter in double quotes and a bin should be created with your elements inside.

e.g. gst-launch-1.0 playbin uri=file:///path/to/file video-sink="glupload ! glcolorconvert ! glfiltercube ! glimagesinkelement" will cubify the video.

Cheers
-Matt

Or do I need to create a new bin in code, set it up with glupload ! capsfilter ! fakesink (like in testegc [1]), and later use gst_pipeline_new() and gst_parse_bin_from_description() to create a custom pipeline that references the video bin created earlier?

Best

G

[1] https://cgit.freedesktop.org/gstreamer/gst-omx/tree/examples/egl/testegl.c#n1079

On Fri, Sep 16, 2016 at 2:26 PM, Matthew Waters <[hidden email]> wrote:

On 16/09/16 20:10, Gottfried Haider wrote:
> Hello all,
>
> I have this pipeline as part of Processing's glvideo library, that
> works great - but doesn't play audio:
>
> uridecodebin uri=... ! glupload name=glup ! glcolorconvert !
> capsfilter name=filter ! fakesink name=vsink
>
> So I tried this instead
>
> uridecodebin uri=... name=decoder ! queue ! audioconvert !
> audioresample ! autoaudiosink sync=false decoder. ! queue ! glupload
> name=glup ! glcolorconvert ! capsfilter name=filter ! fakesink
> name=vsink sync=false
>
> which just hang. (The particular mp4 file I was testing this with had
> not audio track if that matters.)
>
>
> Should the second pipeline work in theory? Any other things I could
> try to get something like a playbin-like source element that would
> also play audio tracks, if the source has one? I uploaded a dot file
> of the second pipeline on OS X  for you to look at[1].

If the file didn't have an audio track that means that the audio output
part of the pipeline would not be linked to uridecodebin and thus when
prerolling would never get a buffer and would never complete the state
change to paused.  A backtrace of all threads would probably show that
the audiosink is stuck on the preroll cond/lock.

Your options are to either use playbin with the video-sink property set
to your custom video branch or if you're set on using uridecodebin, you
will have to selectively add the audio part of the pipeline if there is
an audio pad exposed by uridecodebin.  See the following example which
uses decodebin but the same idea is valid for uridecodebin:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-components-decodebin.html.

Cheers
-Matt

> (If the pipeline looks right it might just be that the audio decoding
> hits this bug [2] on OS X.)
>
>
> Thanks in advance
> Gottfried
>
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=768630
> [2] https://sukzessiv.net/~gohai/gstreamer/uridecode-audio.png
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel