GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

David Ing
I have observed two bugs (possibly related) with GES 1.12, and I wonder if anybody has seen behavior like this.

First let me describe a basic setup.  I have 3 clips.  Each clip is on it's own layer
  • a0.m4a --> audio only for the duration of the entire composition
    • layer priority 2
    • GESEffect:  volume volume=1.0
  • v1.m4v --> video only, the large background screen of a picture-in-picture video composition
    • layer priority 1
    • GESEffect:  volume mute=true
  • v0.m4v --> video only, the small foreground screen of a picture-in-picture video composition 
    • layer priority 0
    • GESEffect:  volume mute=true
The first bug is that when the composition (described above) is rendered, I do not hear any audio.  The only clip which had audio (a0.m4a) is being muted.  Interestingly, I hear the audio when I specify volume volume=1.0 on all the clips.  So this seems to work fine:
  • a0.m4a
    • layer priority 2
    • GESEffect:  volume volume=1.0
  • v1.m4v
    • layer priority 1
    • GESEffect:  volume volume=1.0
  • v0.m4v
    • layer priority 0
    • GESEffect:  volume volume=1.0
The second bug is when I change the order of layers (and their priority) the entire system breaks.  (My code always creates layers in descending order of priority.)  Here is the ordering which breaks the system.
  • v1.m4v
    • layer priority 2
  • v0.m4v
    • layer priority 1
  • a0.m4a
    • layer priority 0
When the system breaks down under the last scenario, I see the following logs.

2018-07-18 21:45:55.299469 DEBUG    CompositionJob.cpp:311 Linked teed encoder eaacenc_audio0 to muxer mux_C:\dev\github\panopto\panopto-core\Debug\x64\test-scratch\2018-07-18T21-45-54.354719Z\CompositionTests\podcast.mp4.
2018-07-18 21:45:55.303481 DEBUG    CompositionJob.cpp:311 Linked teed encoder eavcenc_video0 to muxer mux_C:\dev\github\panopto\panopto-core\Debug\x64\test-scratch\2018-07-18T21-45-54.354719Z\CompositionTests\podcast.mp4.
2018-07-18 21:45:55.307992 DEBUG    CompositionJob.cpp:696 Successfully linked muxer to sink for container C:\dev\github\panopto\panopto-core\Debug\x64\test-scratch\2018-07-18T21-45-54.354719Z\CompositionTests\podcast.mp4.
2018-07-18 21:45:55.317516 WARNING nlecomposition nlecomposition.c:2647 error: The NleComposition structure is not valid
2018-07-18 21:45:55.315511 DEBUG    CompositionJob.cpp:165 Pipeline state changed from NULL to READY
2018-07-18 21:45:55.318018 WARNING basesrc gstbasesrc.c:3480 pad not activated yet
2018-07-18 21:45:55.318018 DEBUG    CompositionJob.cpp:150 Pipeline duration is 777137619 (0:00:17.957006803).
2018-07-18 21:45:55.320024 WARNING nlecomposition nlecomposition.c:2647 error: <GESEffect:nleoperation0> Not enough sinkpads to link all objects to the operation ! 1 / 0, current toplevel seek seek event: 00000181DBF2A150, time 99:99:99.999999999, seq-num 884, GstEventSeek, rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, flags=(GstSeekFlags)GST_SEEK_FLAG_FLUSH+GST_SEEK_FLAG_ACCURATE, cur-type=(GstSeekType)GST_SEEK_TYPE_SET, cur=(gint64)0, stop-type=(GstSeekType)GST_SEEK_TYPE_SET, stop=(gint64)16500000000;
2018-07-18 21:45:55.322028 WARNING basesrc gstbasesrc.c:3480 pad not activated yet
2018-07-18 21:45:55.324035 DEBUG    CompositionJob.cpp:150 Pipeline duration is 777137619 (0:00:17.957006803).
2018-07-18 21:45:55.334061 WARNING nlecomposition nlecomposition.c:2654 error: The NleComposition structure is not valid
2018-07-18 21:45:55.339099 WARNING basesrc gstbasesrc.c:3480 pad not activated yet
2018-07-18 21:45:55.342080 WARNING qtdemux qtdemux_types.c:228 unknown QuickTime node type uuid
2018-07-18 21:45:55.346091 WARNING basesrc gstbasesrc.c:3480 pad not activated yet
2018-07-18 21:45:55.347093 WARNING nlecomposition nlecomposition.c:2654 error: Operation <GESEffect:nleoperation0> has no child objects to be connected to current toplevel seek: seek event: 00000181DBF2A150, time 99:99:99.999999999, seq-num 884, GstEventSeek, rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, flags=(GstSeekFlags)GST_SEEK_FLAG_FLUSH+GST_SEEK_FLAG_ACCURATE, cur-type=(GstSeekType)GST_SEEK_TYPE_SET, cur=(gint64)0, stop-type=(GstSeekType)GST_SEEK_TYPE_SET, stop=(gint64)16500000000;
2018-07-18 21:45:55.342583 ERROR    CompositionJob.cpp:99 Error received from element audio_nlecomposition0: The NleComposition structure is not valid.
nlecomposition.c(2647): _relink_children_recursively (): /GstPipeline:whatever/GESTimeline:gestimeline0/GESAudioTrack:gesaudiotrack0/NleComposition:audio_nlecomposition0:
<GESEffect:nleoperation0> Not enough sinkpads to link all objects to the operation ! 1 / 0, current toplevel seek seek event: 00000181DBF2A150, time 99:99:99.999999999, seq-num 884, GstEventSeek, rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, flags=(GstSeekFlags)GST_SEEK_FLAG_FLUSH+GST_SEEK_FLAG_ACCURATE, cur-type=(GstSeekType)GST_SEEK_TYPE_SET, cur=(gint64)0, stop-type=(GstSeekType)GST_SEEK_TYPE_SET, stop=(gint64)16500000000;
2018-07-18 21:45:55.360129 ERROR    CompositionJob.cpp:120 Job failed after 00:00:00.486775.  It started at 2018-07-18T21:45:55.359627Z and ended at 2018-07-18T21:45:54.872852Z.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

Thibault Saunier-3
Hello,

1. Setting mute=true means that the source will send silent, which is
not what you want. Also you do not need an effect to handle the volume
as there is already a volume element inside sources you can use the
ges_timeline_element_set_child_property(element, "volume"....)[0]
method to set it. You can see what properties audio sources expose
here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html

2. Layer with lower priorities are on top of the stack of layers,
meaning that layer with prio=0 is "the first". Now I am not sure why
you get that error but it seems to be a NLEObject priority issue.
Providing a .xges file would help (just save your timeline and paste
it here if removing the effect doesn't work).

Regards,

Thibault Saunier

[0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

David Ing
Thibault, your suggestion for #1 worked like a charm.  It fixed both of the problems.

I have two followup questions.
  1. This experience has me a little worried.  If GESEffect does not work well for the `volume` plugin, can I expect problems with most of the other plugins?
  2. For future reference, I don't know how to save a *.xges file.  Would I use `ges_timeline_save_to_uri` for that?

On Thu, Jul 19, 2018 at 2:35 PM, Thibault Saunier <[hidden email]> wrote:
Hello,

1. Setting mute=true means that the source will send silent, which is
not what you want. Also you do not need an effect to handle the volume
as there is already a volume element inside sources you can use the
ges_timeline_element_set_child_property(element, "volume"....)[0]
method to set it. You can see what properties audio sources expose
here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html

2. Layer with lower priorities are on top of the stack of layers,
meaning that layer with prio=0 is "the first". Now I am not sure why
you get that error but it seems to be a NLEObject priority issue.
Providing a .xges file would help (just save your timeline and paste
it here if removing the effect doesn't work).

Regards,

Thibault Saunier

[0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

Thibault Saunier-4
> Thibault, your suggestion for #1 worked like a charm.  It fixed both of the problems.
>
> I have two followup questions.
>
> This experience has me a little worried.  If GESEffect does not work well for the `volume` plugin, can I expect problems with most of the other plugins?

Well, we use it very extensively in Pitivi without (known) problem,
how did you use them exactly?

> For future reference, I don't know how to save a *.xges file.  Would I use `ges_timeline_save_to_uri` for that?

Yes.

> On Thu, Jul 19, 2018 at 2:35 PM, Thibault Saunier <[hidden email]> wrote:
>>
>> Hello,
>>
>> 1. Setting mute=true means that the source will send silent, which is
>> not what you want. Also you do not need an effect to handle the volume
>> as there is already a volume element inside sources you can use the
>> ges_timeline_element_set_child_property(element, "volume"....)[0]
>> method to set it. You can see what properties audio sources expose
>> here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html
>>
>> 2. Layer with lower priorities are on top of the stack of layers,
>> meaning that layer with prio=0 is "the first". Now I am not sure why
>> you get that error but it seems to be a NLEObject priority issue.
>> Providing a .xges file would help (just save your timeline and paste
>> it here if removing the effect doesn't work).
>>
>> Regards,
>>
>> Thibault Saunier
>>
>> [0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

David Ing
My timeline consists of 3 layers, where each layer has a GESUriClip.

For each of the 3 clips, ...

I created a GESEffect using either
GESEffect* gesEffect = ges_effect_new("volume volume=1.0")
GESEffect* gesEffect = ges_effect_new("volume mute=true") 

Then I added the effect to the clip
ges_container_add(reinterpret_cast<GESContainer*>(gesUriClip),  reinterpret_cast<GESTimelineElement*>(gesEffect))

Is this generally the correct way to add effects to clips?  (Sometimes I feel like I'm groping around a dark hallway.)

On Thu, Jul 19, 2018 at 3:53 PM, Thibault Saunier <[hidden email]> wrote:
> Thibault, your suggestion for #1 worked like a charm.  It fixed both of the problems.
>
> I have two followup questions.
>
> This experience has me a little worried.  If GESEffect does not work well for the `volume` plugin, can I expect problems with most of the other plugins?

Well, we use it very extensively in Pitivi without (known) problem,
how did you use them exactly?

> For future reference, I don't know how to save a *.xges file.  Would I use `ges_timeline_save_to_uri` for that?

Yes.

> On Thu, Jul 19, 2018 at 2:35 PM, Thibault Saunier <[hidden email]> wrote:
>>
>> Hello,
>>
>> 1. Setting mute=true means that the source will send silent, which is
>> not what you want. Also you do not need an effect to handle the volume
>> as there is already a volume element inside sources you can use the
>> ges_timeline_element_set_child_property(element, "volume"....)[0]
>> method to set it. You can see what properties audio sources expose
>> here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html
>>
>> 2. Layer with lower priorities are on top of the stack of layers,
>> meaning that layer with prio=0 is "the first". Now I am not sure why
>> you get that error but it seems to be a NLEObject priority issue.
>> Providing a .xges file would help (just save your timeline and paste
>> it here if removing the effect doesn't work).
>>
>> Regards,
>>
>> Thibault Saunier
>>
>> [0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

Thibault Saunier-4
This is correct yes. I just figured that your problem is that you add
an *audio* effect on a *video only* clip.... I tested and that fails
in the way you describe (we should probably be smarter here, but it is
a programming error on your side.

On Thu, Jul 19, 2018 at 7:03 PM David Ing <[hidden email]> wrote:

>
> My timeline consists of 3 layers, where each layer has a GESUriClip.
>
> For each of the 3 clips, ...
>
> I created a GESEffect using either
>
> GESEffect* gesEffect = ges_effect_new("volume volume=1.0")
>
> GESEffect* gesEffect = ges_effect_new("volume mute=true")
>
> Then I added the effect to the clip
>
> ges_container_add(reinterpret_cast<GESContainer*>(gesUriClip),  reinterpret_cast<GESTimelineElement*>(gesEffect))
>
> Is this generally the correct way to add effects to clips?  (Sometimes I feel like I'm groping around a dark hallway.)
>
> On Thu, Jul 19, 2018 at 3:53 PM, Thibault Saunier <[hidden email]> wrote:
>>
>> > Thibault, your suggestion for #1 worked like a charm.  It fixed both of the problems.
>> >
>> > I have two followup questions.
>> >
>> > This experience has me a little worried.  If GESEffect does not work well for the `volume` plugin, can I expect problems with most of the other plugins?
>>
>> Well, we use it very extensively in Pitivi without (known) problem,
>> how did you use them exactly?
>>
>> > For future reference, I don't know how to save a *.xges file.  Would I use `ges_timeline_save_to_uri` for that?
>>
>> Yes.
>>
>> > On Thu, Jul 19, 2018 at 2:35 PM, Thibault Saunier <[hidden email]> wrote:
>> >>
>> >> Hello,
>> >>
>> >> 1. Setting mute=true means that the source will send silent, which is
>> >> not what you want. Also you do not need an effect to handle the volume
>> >> as there is already a volume element inside sources you can use the
>> >> ges_timeline_element_set_child_property(element, "volume"....)[0]
>> >> method to set it. You can see what properties audio sources expose
>> >> here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html
>> >>
>> >> 2. Layer with lower priorities are on top of the stack of layers,
>> >> meaning that layer with prio=0 is "the first". Now I am not sure why
>> >> you get that error but it seems to be a NLEObject priority issue.
>> >> Providing a .xges file would help (just save your timeline and paste
>> >> it here if removing the effect doesn't work).
>> >>
>> >> Regards,
>> >>
>> >> Thibault Saunier
>> >>
>> >> [0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
>> >> _______________________________________________
>> >> gstreamer-devel mailing list
>> >> [hidden email]
>> >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> >
>> >
>> > _______________________________________________
>> > gstreamer-devel mailing list
>> > [hidden email]
>> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GES 1.12, Two bugs (possibly related) with (1) GESEffect volume and (2) layer ordering priority

David Ing
Thanks ... Good to know

On Thu, Jul 19, 2018, 6:21 PM Thibault Saunier <[hidden email]> wrote:
This is correct yes. I just figured that your problem is that you add
an *audio* effect on a *video only* clip.... I tested and that fails
in the way you describe (we should probably be smarter here, but it is
a programming error on your side.

On Thu, Jul 19, 2018 at 7:03 PM David Ing <[hidden email]> wrote:
>
> My timeline consists of 3 layers, where each layer has a GESUriClip.
>
> For each of the 3 clips, ...
>
> I created a GESEffect using either
>
> GESEffect* gesEffect = ges_effect_new("volume volume=1.0")
>
> GESEffect* gesEffect = ges_effect_new("volume mute=true")
>
> Then I added the effect to the clip
>
> ges_container_add(reinterpret_cast<GESContainer*>(gesUriClip),  reinterpret_cast<GESTimelineElement*>(gesEffect))
>
> Is this generally the correct way to add effects to clips?  (Sometimes I feel like I'm groping around a dark hallway.)
>
> On Thu, Jul 19, 2018 at 3:53 PM, Thibault Saunier <[hidden email]> wrote:
>>
>> > Thibault, your suggestion for #1 worked like a charm.  It fixed both of the problems.
>> >
>> > I have two followup questions.
>> >
>> > This experience has me a little worried.  If GESEffect does not work well for the `volume` plugin, can I expect problems with most of the other plugins?
>>
>> Well, we use it very extensively in Pitivi without (known) problem,
>> how did you use them exactly?
>>
>> > For future reference, I don't know how to save a *.xges file.  Would I use `ges_timeline_save_to_uri` for that?
>>
>> Yes.
>>
>> > On Thu, Jul 19, 2018 at 2:35 PM, Thibault Saunier <[hidden email]> wrote:
>> >>
>> >> Hello,
>> >>
>> >> 1. Setting mute=true means that the source will send silent, which is
>> >> not what you want. Also you do not need an effect to handle the volume
>> >> as there is already a volume element inside sources you can use the
>> >> ges_timeline_element_set_child_property(element, "volume"....)[0]
>> >> method to set it. You can see what properties audio sources expose
>> >> here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESAudioSource.html
>> >>
>> >> 2. Layer with lower priorities are on top of the stack of layers,
>> >> meaning that layer with prio=0 is "the first". Now I am not sure why
>> >> you get that error but it seems to be a NLEObject priority issue.
>> >> Providing a .xges file would help (just save your timeline and paste
>> >> it here if removing the effect doesn't work).
>> >>
>> >> Regards,
>> >>
>> >> Thibault Saunier
>> >>
>> >> [0] https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-editing-services/html/GESTimelineElement.html#ges-timeline-element-set-child-property
>> >> _______________________________________________
>> >> gstreamer-devel mailing list
>> >> [hidden email]
>> >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> >
>> >
>> > _______________________________________________
>> > gstreamer-devel mailing list
>> > [hidden email]
>> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel