GES: Problem applying certain effects when seeking with a non-1.0 rate

[Steve Rubin]

Hello!

My goal is to play a GES pipeline at a faster or slower rate without affecting the perceived audio pitch.

If I use `gst_element_seek` on the GESPipeline, I can play the pipeline at a different speed, as desired:

  gst_element_seek(GST_ELEMENT(gesPipeline),
                   2,
                   GST_FORMAT_TIME,
                   GST_SEEK_FLAG_FLUSH,
                   GST_SEEK_TYPE_NONE,
                   GST_CLOCK_TIME_NONE,
                   GST_SEEK_TYPE_NONE,
                   GST_CLOCK_TIME_NONE);

properly plays the pipeline at 2x speed (with "chipmunk" audio).

I then tried to add a `pitch` or `scaletempo` effect to preserve the perceived pitch:

  GESEffect *pitch = ges_effect_new("pitch pitch=0.5");
  ges_timeline_element_set_duration(GES_TIMELINE_ELEMENT(pitch),
                                    runningTime * GST_SECOND);
  ges_track_add_element(GES_TRACK(audioTrack),
                        GES_TRACK_ELEMENT(pitch));

But now when I play the pipeline, I get the following errors:

  ERROR        audioaggregator gstaudioaggregator.c:1015:gboolean gst_audio_aggregator_sink_event(GstAggregator *, GstAggregatorPad *, GstEvent *):<smart-adder-adder:sink_0> Got segment event with wrong rate 1.000000, expected 2.000000

  CRITICAL **: 12:40:23.458: gst_audio_buffer_clip: assertion 'segment->format == GST_FORMAT_TIME || segment->format == GST_FORMAT_DEFAULT' failed

  GStreamer-CRITICAL **: 12:40:23.458: gst_segment_to_running_time: assertion 'segment->format == format' failed

  GStreamer-CRITICAL **: 12:40:23.458: gst_segment_to_running_time: assertion 'segment->format == format' failed

The same thing happens if I add a `scaletempo` effect in place of the `pitch` effect. The same thing happens if I try to add these effects to GESClips rather than the GESAudioTrack, e.g.:

  GESEffect *pitch = ges_effect_new("pitch pitch=0.5");

  ges_timeline_element_set_duration(GES_TIMELINE_ELEMENT(pitch), clipDuration);

  ges_container_add(GES_CONTAINER(clip), GES_TIMELINE_ELEMENT(pitch));

I realize that I could control the playback rate/pitch by using only the `pitch` effect and re-writing all the clips in the pipeline every time I want to change the playback rate; but I'd prefer it to be a simple parameter that I can set on the pipeline as a whole (using the seek rate property).

Any help would be greatly appreciated!

Steve Rubin

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Steve Rubin]

In case anyone else is having this issue, here's what worked for me.

I used the playback speed parameter in `gst_element_seek` as in my original
message:


Steve Rubin wrote
>   gst_element_seek(GST_ELEMENT(gesPipeline),
>                    2,
>                    GST_FORMAT_TIME,
>                    GST_SEEK_FLAG_FLUSH,
>                    GST_SEEK_TYPE_NONE,
>                    GST_CLOCK_TIME_NONE,
>                    GST_SEEK_TYPE_NONE,
>                    GST_CLOCK_TIME_NONE);

Then, I augmented the pipeline's output sink to use a `scaletempo` effect:

  GstElement *audioSink = gst_element_factory_make("osxaudiosink",
"mac-audio-sink");
  // Add scaletempo effect to remove "chipmunking"
  GstElement *scaleTempo = gst_element_factory_make("scaletempo", NULL);
  GstPad *sink = gst_element_get_static_pad(scaleTempo, "sink");
  GstBin *sinkBin =
      reinterpret_cast<GstBin *>(gst_bin_new("media-engine-sink"));
  gst_bin_add_many(sinkBin, scaleTempo, audioSink, NULL);
  gst_element_link(scaleTempo, audioSink);
  gst_element_add_pad(reinterpret_cast<GstElement *>(sinkBin),
                      gst_ghost_pad_new(NULL, sink));
  ges_pipeline_preview_set_audio_sink(pipeline,
                                      reinterpret_cast<GstElement
*>(sinkBin));
  gst_object_unref(sink);

This worked - my GESPipeline will play at non-1.0 rates and adjust the audio
pitch appropriately to not "chipmunk."



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel