Feeding raw audio data into a GES timeline
Locked
Feeding raw audio data into a GES timeline
 
|
Hi,
I am working on an audio synthesis tool, which allows the user to generate audio to play over the top of a sequence of videos. I'm writing this in python. I am using GES to create a simple video sequence editor. The application needs to be able to play the synthesised audio in sync with the video. I have written a simple proof of concept subclass of GstApp.AppSrc which can do some simple audio synthesis, and feed it into a GStreamer pipeline. This works fine for me if I just connect it up to an autoaudiosink without using any GES stuff. Is there a way to get this gstreamer element to be part of the GES timeline? I tried to do this using the following method: Create a Subclass of GES.SourceClip Implement do_create_track_element on this subclass do_create_track_element instantiates a custom subclass of GES.AudioSource the subclass of AudioSource overrides do_create_source and returns a pipeline that wraps my custom AppSrc. call layer.add_clip on a layer in a GES timeline to add one of these custom clips This method does not work, I see various assertion errors when I try this. Is this the right approach? Am I doing roughly the right thing? Or is there a totally different way to achieve what I want? If this is roughly the right idea then where am I going wrong? Thanks. Dan. _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Feeding raw audio data into a GES timeline
|
|
Hi, Your approach sounds about right, although. I guess the issues are related to assets, could you paste the assertions you get/a piece of code to reproduce? - Thibault Le ven. 19 févr. 2021 à 15:32, Daniel Bailey <[hidden email]> a écrit :
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Feeding raw audio data into a GES timeline
|
|
Hi, thanks for the quick reply. I have stripped down my project code to just the bare minimum to show the approach I am using and the problem I'm having, see the code here: The output from this program is pasted below. Thanks. Dan. Adding asset to layer <__gi__.GESTestClipAsset object at 0x7f16541e4540 (GESTestClipAsset at 0x56462a434b00)> Adding asset to layer <GES.SourceClipAsset object at 0x7f16541e4680 (GESSourceClipAsset at 0x56462a84b050)> Creating audio synth element Created source <__main__.SynthAudioSourceElement object at 0x7f16541e4780 (__main__+SynthAudioSourceElement at 0x56462a4a08f0)> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_priority: assertion 'object->priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_duration: assertion 'priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: ges_track_element_is_active: assertion 'object->priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: ges_track_element_set_track: assertion 'object->priv->nleobject' failed Creating audio synth element Created source <__main__.SynthAudioSourceElement object at 0x7f16541e4680 (__main__+SynthAudioSourceElement at 0x56462a4a08f0)> (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_priority: assertion 'object->priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: _set_duration: assertion 'priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: ges_track_element_is_active: assertion 'object->priv->nleobject' failed (raw_audio_in_ges.py:44181): GES-CRITICAL **: 12:12:52.336: ges_track_element_set_track: assertion 'object->priv->nleobject' failed On Fri, Feb 19, 2021 at 10:52 PM Thibault Saunier <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Feeding raw audio data into a GES timeline
|
|
Hi, I checked a bit your code and there are a few issues to get that working: * the `GESAudioSource::create_source` vmethod is not exposed through GI, and for some reason even properly annotating it makes it not working. * We need https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/merge_requests/231 to fix some annotations so we can subclass GESSource properly * There is a floating reference mishandling in PyGObject leading to a crash, we need https://gitlab.gnome.org/GNOME/pygobject/-/merge_requests/129 to get it working I then updated the code here: https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/snippets/1633 - your source element doesn't seem to work properly yet though. Also not that I have some WIP that would allow doing something like: ``` class YourSource(GstApp.AppSrc): ... Gst.Element.register("myaudiosrc", 0, YourSource) clip = GES.Clip.new(video_source_bin_desc=None, audio_source_bin_desc="myaudiosource") ``` removing the need for any subclass. - Thibault On Sat, Feb 20, 2021 at 9:15 AM Daniel Bailey <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Feeding raw audio data into a GES timeline
|
|
Hi, Thanks for the detailed reply. I'll have a play with your updated code soon :) Is there anywhere your WIP is being tracked so that I follow and know when it is ready? I would love to switch to that approach once it's ready. Thanks, Dan. On Thu, Feb 25, 2021 at 3:01 AM Thibault Saunier <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |