question how to add own customized element between source element and decodebin2 element when using playbin2

[bcxa sz]

Hi Developers,

Here we want to do something special between source elemet and
decodebin2 element with playbin2.

For example, when playback one avi file from harddisk, we have one own
element called routine element, And we prefer to let it stay between
the file source element and decodebin2 element to do something
special.

But I don't how to make it inserted by playbin2 when playbin2
construct the pipeline.

best regards

BCXA
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Christophe Oosterlynck]

Hi,

I'm looking to do the same thing. Right after the source, I want to insert another element and let playbin2 do the rest.

From what I see I have 2 possibilities:
* react to "notify::source" signal, add the extra element to the bin, link it to the source and let playbin2 continue
* wait until playbin2 has finished constructing the whole pipeline, unlink source and next element and insert extra element in between.

Can I do one of these safely or shouldn't I fiddle with the playbin2 pipeline behind its back?

What I want to achieve with this: buffer a live udp source.
The buffering mechanism in playbin2 is not usable for live sources as you cannot pause them. I want to insert a queue element right behind the udpsrc element with min-threshold-bytes. This works when I create a custom pipeline with gst-launch. If this is not the best way to handle this, let me know. Nevertheless, I'm still interested in the answer to my question on fiddling with the playbin2 pipeline.

Best regards,

Christophe

[Tim-Philipp Müller-2]

On Tue, 2012-11-13 at 05:40 -0800, Christophe Oosterlynck wrote:

> I'm looking to do the same thing. Right after the source, I want to insert
> another element and let playbin2 do the rest.
>
> From what I see I have 2 possibilities:
> * react to "notify::source" signal, add the extra element to the bin, link
> it to the source and let playbin2 continue
> * wait until playbin2 has finished constructing the whole pipeline, unlink
> source and next element and insert extra element in between.
>
> Can I do one of these safely or shouldn't I fiddle with the playbin2
> pipeline behind its back?
>
> What I want to achieve with this: buffer a live udp source.
> The buffering mechanism in playbin2 is not usable for live sources as you
> cannot pause them. I want to insert a queue element right behind the udpsrc
> element with min-threshold-bytes. This works when I create a custom pipeline
> with gst-launch. If this is not the best way to handle this, let me know.
> Nevertheless, I'm still interested in the answer to my question on fiddling
> with the playbin2 pipeline.

You should not fiddle with playbin's pipeline, and certainly not link
elements behind its back. It won't work. Even if it does work, it might
stop working tomorrow. And we won't even feel bad for a second if your
app breaks doing things like that ;)

The only sane way to do things like this is to write your own element,
which could be a GstBin subclass where you add some source and whatever
additional elements you want and then create a ghost source pad. You can
then register a new (or existing, if you must) URI protocol for it, so
if you do playbin uri=fooudp://a.b.c.d:port it will plug your bin.

Or you could try to come up with API for playbin to add such a thing,
but I think would need to be rather well-justified and deemed
interesting enough to be added.

(What does your "routine" element do, "BCXA" ?)

 Cheers
  -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Christophe Oosterlynck]

Tim-Philipp, thanks for your quick and helpful reply. I'm glad I asked here first before starting to implement something. I will try your suggestion.

I'm also not sure if inserting a queue element right after the udp source is the right way to catch my problems with udp streaming. The actual problem I'm having is corrupted sound on playing a raw udp stream containing a transport stream consisting of H264 video and MPEG 1 layer 2 audio. The image is fine but the sound starts crackling after a while until it sometimes disappears completely until it plays fine again and repeats this in a cycle of 60 seconds.

As there is no buffering for live sources in playbin2, I'm suspecting this to be a problem. Why is there no facility in playbin2 to delay playback of a udp stream a bit to buffer it and handle possible packet delays? 

Best regards,

Christophe

On Tue, Nov 13, 2012 at 3:14 PM, Tim-Philipp Müller-2 [via GStreamer-devel] <[hidden email]> wrote:

On Tue, 2012-11-13 at 05:40 -0800, Christophe Oosterlynck wrote:

> I'm looking to do the same thing. Right after the source, I want to insert
> another element and let playbin2 do the rest.
>
> From what I see I have 2 possibilities:
> * react to "notify::source" signal, add the extra element to the bin, link
> it to the source and let playbin2 continue
> * wait until playbin2 has finished constructing the whole pipeline, unlink
> source and next element and insert extra element in between.
>
> Can I do one of these safely or shouldn't I fiddle with the playbin2
> pipeline behind its back?
>
> What I want to achieve with this: buffer a live udp source.
> The buffering mechanism in playbin2 is not usable for live sources as you
> cannot pause them. I want to insert a queue element right behind the udpsrc
> element with min-threshold-bytes. This works when I create a custom pipeline
> with gst-launch. If this is not the best way to handle this, let me know.
> Nevertheless, I'm still interested in the answer to my question on fiddling
> with the playbin2 pipeline.

You should not fiddle with playbin's pipeline, and certainly not link
elements behind its back. It won't work. Even if it does work, it might
stop working tomorrow. And we won't even feel bad for a second if your
app breaks doing things like that ;)

The only sane way to do things like this is to write your own element,
which could be a GstBin subclass where you add some source and whatever
additional elements you want and then create a ghost source pad. You can
then register a new (or existing, if you must) URI protocol for it, so
if you do playbin uri=fooudp://a.b.c.d:port it will plug your bin.

Or you could try to come up with API for playbin to add such a thing,
but I think would need to be rather well-justified and deemed
interesting enough to be added.

(What does your "routine" element do, "BCXA" ?)

 Cheers
  -Tim

_______________________________________________
gstreamer-devel mailing list

[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

---

If you reply to this email, your message will be added to the discussion below:

http://gstreamer-devel.966125.n4.nabble.com/question-how-to-add-own-customized-element-between-source-element-and-decodebin2-element-when-using-2-tp4353571p4656890.html

To unsubscribe from question how to add own customized element between source element and decodebin2 element when using playbin2, click here.
NAML

[Christophe Oosterlynck]

Tim-Philipp, thanks for your quick and helpful reply. I'm glad I asked here first before starting to implement something. I will try your suggestion.

I'm also not sure if inserting a queue element right after the udp source is the right way to catch my problems with udp streaming. The actual problem I'm having is corrupted sound on playing a raw udp stream containing a transport stream consisting of H264 video and MPEG 1 layer 2 audio. The image is fine but the sound starts crackling after a while until it sometimes disappears completely until it plays fine again and repeats this in a cycle of 60 seconds.

As there is no buffering for live sources in playbin2, I'm suspecting this to be a problem. Why is there no facility in playbin2 to delay playback of a udp stream a bit to buffer it and handle possible packet delays? 

Best regards,

Christophe

On Tue, Nov 13, 2012 at 3:09 PM, Tim-Philipp Müller <[hidden email]> wrote:

On Tue, 2012-11-13 at 05:40 -0800, Christophe Oosterlynck wrote:

> I'm looking to do the same thing. Right after the source, I want to insert
> another element and let playbin2 do the rest.
>
> From what I see I have 2 possibilities:
> * react to "notify::source" signal, add the extra element to the bin, link
> it to the source and let playbin2 continue
> * wait until playbin2 has finished constructing the whole pipeline, unlink
> source and next element and insert extra element in between.
>
> Can I do one of these safely or shouldn't I fiddle with the playbin2
> pipeline behind its back?
>
> What I want to achieve with this: buffer a live udp source.
> The buffering mechanism in playbin2 is not usable for live sources as you
> cannot pause them. I want to insert a queue element right behind the udpsrc
> element with min-threshold-bytes. This works when I create a custom pipeline
> with gst-launch. If this is not the best way to handle this, let me know.
> Nevertheless, I'm still interested in the answer to my question on fiddling
> with the playbin2 pipeline.

You should not fiddle with playbin's pipeline, and certainly not link
elements behind its back. It won't work. Even if it does work, it might
stop working tomorrow. And we won't even feel bad for a second if your
app breaks doing things like that ;)

The only sane way to do things like this is to write your own element,
which could be a GstBin subclass where you add some source and whatever
additional elements you want and then create a ghost source pad. You can
then register a new (or existing, if you must) URI protocol for it, so
if you do playbin uri=fooudp://a.b.c.d:port it will plug your bin.

Or you could try to come up with API for playbin to add such a thing,
but I think would need to be rather well-justified and deemed
interesting enough to be added.

(What does your "routine" element do, "BCXA" ?)

 Cheers
  -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Edward Hervey]

Hi,

  summary: this issue doesn't require a queue to fix the issue.

  What version of GStreamer are you using ?

  The "crackling" is most likely due to clock skew happening, or data
stream getting corrupted.

  Depending on the version of GStreamer you are using and the bitrate
being used, you might want to increase the udpsrc socket buffer-size to
a bigger value (buffer-size property).
  Also note that there is a performance issue with the underlying system
being used by udpsrc (gio) which is currently being fixed. On some
systems it could cause packet drops (because we don't read out the data
from the socket fast enough).

   Edward

On Tue, 2012-11-13 at 21:09 +0100, Christophe Oosterlynck wrote:

> Tim-Philipp, thanks for your quick and helpful reply. I'm glad I asked
> here first before starting to implement something. I will try your
> suggestion.
>
>
> I'm also not sure if inserting a queue element right after the udp
> source is the right way to catch my problems with udp streaming. The
> actual problem I'm having is corrupted sound on playing a raw udp
> stream containing a transport stream consisting of H264 video and MPEG
> 1 layer 2 audio. The image is fine but the sound starts crackling
> after a while until it sometimes disappears completely until it plays
> fine again and repeats this in a cycle of 60 seconds.
>
>
> As there is no buffering for live sources in playbin2, I'm suspecting
> this to be a problem. Why is there no facility in playbin2 to delay
> playback of a udp stream a bit to buffer it and handle possible packet
> delays?
>
>
> Best regards,
>
>
> Christophe
>
>
> On Tue, Nov 13, 2012 at 3:09 PM, Tim-Philipp Müller <[hidden email]>
> wrote:
>         On Tue, 2012-11-13 at 05:40 -0800, Christophe Oosterlynck
>         wrote:
>        
>         > I'm looking to do the same thing. Right after the source, I
>         want to insert
>         > another element and let playbin2 do the rest.
>         >
>         > From what I see I have 2 possibilities:
>         > * react to "notify::source" signal, add the extra element to
>         the bin, link
>         > it to the source and let playbin2 continue
>         > * wait until playbin2 has finished constructing the whole
>         pipeline, unlink
>         > source and next element and insert extra element in between.
>         >
>         > Can I do one of these safely or shouldn't I fiddle with the
>         playbin2
>         > pipeline behind its back?
>         >
>         > What I want to achieve with this: buffer a live udp source.
>         > The buffering mechanism in playbin2 is not usable for live
>         sources as you
>         > cannot pause them. I want to insert a queue element right
>         behind the udpsrc
>         > element with min-threshold-bytes. This works when I create a
>         custom pipeline
>         > with gst-launch. If this is not the best way to handle this,
>         let me know.
>         > Nevertheless, I'm still interested in the answer to my
>         question on fiddling
>         > with the playbin2 pipeline.
>        
>        
>         You should not fiddle with playbin's pipeline, and certainly
>         not link
>         elements behind its back. It won't work. Even if it does work,
>         it might
>         stop working tomorrow. And we won't even feel bad for a second
>         if your
>         app breaks doing things like that ;)
>        
>         The only sane way to do things like this is to write your own
>         element,
>         which could be a GstBin subclass where you add some source and
>         whatever
>         additional elements you want and then create a ghost source
>         pad. You can
>         then register a new (or existing, if you must) URI protocol
>         for it, so
>         if you do playbin uri=fooudp://a.b.c.d:port it will plug your
>         bin.
>        
>         Or you could try to come up with API for playbin to add such a
>         thing,
>         but I think would need to be rather well-justified and deemed
>         interesting enough to be added.
>        
>         (What does your "routine" element do, "BCXA" ?)
>        
>          Cheers
>           -Tim
>        
>         _______________________________________________
>         gstreamer-devel mailing list
>         [hidden email]
>         http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>        
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Christophe Oosterlynck]

Hi,

Only the sound of the stream is getting corrupted. Video plays smooth still. Can this still be explained by clock skew or stream data corruption? I would suspect to see corruption or frame drops in the video too. So there is never a queue (buffering) needed to play a live udp stream?

Do you have more information on the gio performance issue? Could I find out somehow that my system is affected?

GStreamer version:

base: 0.10.36

good: 0.10.31

bad: 0.10.23

ugly: 0.10.19

Best regards,

Christophe

On Wed, Nov 14, 2012 at 8:24 AM, Edward Hervey <[hidden email]> wrote:

Hi,

  summary: this issue doesn't require a queue to fix the issue.

  What version of GStreamer are you using ?

  The "crackling" is most likely due to clock skew happening, or data
stream getting corrupted.

  Depending on the version of GStreamer you are using and the bitrate
being used, you might want to increase the udpsrc socket buffer-size to
a bigger value (buffer-size property).
  Also note that there is a performance issue with the underlying system
being used by udpsrc (gio) which is currently being fixed. On some
systems it could cause packet drops (because we don't read out the data
from the socket fast enough).

   Edward

On Tue, 2012-11-13 at 21:09 +0100, Christophe Oosterlynck wrote:
> Tim-Philipp, thanks for your quick and helpful reply. I'm glad I asked
> here first before starting to implement something. I will try your
> suggestion.
>
>
> I'm also not sure if inserting a queue element right after the udp
> source is the right way to catch my problems with udp streaming. The
> actual problem I'm having is corrupted sound on playing a raw udp
> stream containing a transport stream consisting of H264 video and MPEG
> 1 layer 2 audio. The image is fine but the sound starts crackling
> after a while until it sometimes disappears completely until it plays
> fine again and repeats this in a cycle of 60 seconds.
>
>
> As there is no buffering for live sources in playbin2, I'm suspecting
> this to be a problem. Why is there no facility in playbin2 to delay
> playback of a udp stream a bit to buffer it and handle possible packet
> delays?
>
>
> Best regards,
>
>
> Christophe
>
>
> On Tue, Nov 13, 2012 at 3:09 PM, Tim-Philipp Müller <[hidden email]>
> wrote:
>         On Tue, 2012-11-13 at 05:40 -0800, Christophe Oosterlynck
>         wrote:
>
>         > I'm looking to do the same thing. Right after the source, I
>         want to insert
>         > another element and let playbin2 do the rest.
>         >
>         > From what I see I have 2 possibilities:
>         > * react to "notify::source" signal, add the extra element to
>         the bin, link
>         > it to the source and let playbin2 continue
>         > * wait until playbin2 has finished constructing the whole
>         pipeline, unlink
>         > source and next element and insert extra element in between.
>         >
>         > Can I do one of these safely or shouldn't I fiddle with the
>         playbin2
>         > pipeline behind its back?
>         >
>         > What I want to achieve with this: buffer a live udp source.
>         > The buffering mechanism in playbin2 is not usable for live
>         sources as you
>         > cannot pause them. I want to insert a queue element right
>         behind the udpsrc
>         > element with min-threshold-bytes. This works when I create a
>         custom pipeline
>         > with gst-launch. If this is not the best way to handle this,
>         let me know.
>         > Nevertheless, I'm still interested in the answer to my
>         question on fiddling
>         > with the playbin2 pipeline.
>
>
>         You should not fiddle with playbin's pipeline, and certainly
>         not link
>         elements behind its back. It won't work. Even if it does work,
>         it might
>         stop working tomorrow. And we won't even feel bad for a second
>         if your
>         app breaks doing things like that ;)
>
>         The only sane way to do things like this is to write your own
>         element,
>         which could be a GstBin subclass where you add some source and
>         whatever
>         additional elements you want and then create a ghost source
>         pad. You can
>         then register a new (or existing, if you must) URI protocol
>         for it, so
>         if you do playbin uri=fooudp://a.b.c.d:port it will plug your
>         bin.
>
>         Or you could try to come up with API for playbin to add such a
>         thing,
>         but I think would need to be rather well-justified and deemed
>         interesting enough to be added.
>
>         (What does your "routine" element do, "BCXA" ?)
>
>          Cheers
>           -Tim
>
>         _______________________________________________
>         gstreamer-devel mailing list
>         [hidden email]
>         http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[rland]

> >
> > I'm also not sure if inserting a queue element right after the udp
> > source is the right way to catch my problems with udp streaming. The
> > actual problem I'm having is corrupted sound on playing a raw udp
> > stream containing a transport stream consisting of H264 video and MPEG
> > 1 layer 2 audio. The image is fine but the sound starts crackling
> > after a while until it sometimes disappears completely until it plays
> > fine again and repeats this in a cycle of 60 seconds.
> >
I use gstreamer 1.8.2,and I have the same problem.
Finally, do you have any solution to this problem？

> >
> > As there is no buffering for live sources in playbin2, I'm suspecting
> > this to be a problem. Why is there no facility in playbin2 to delay
> > playback of a udp stream a bit to buffer it and handle possible packet
> > delays?
I think it's not enough to just change the buffer-size of udpsrc,
As we known, buffer-size is only to modify the kernel io's restrictions, in many platforms, this value is limited, generally about 100K is the upper limit.
-----------------
The “buffer-size” property is used to change the default kernel buffersizes used for receiving packets. The buffer size may be increased for high-volume connections, or may be decreased to limit the possible backlog of incoming data. The system places an absolute limit on these values, on Linux, for example, the default buffer size is typically 50K and can be increased to maximally 100K.
----------------

I think 100K should be not enough in case of  high bitrate or  HD stream.