Appsrc RTSP Audio and Video pipeline sync issue

[Shaf]

I have created the below pipeline for audio and video streaming using RTSP :

Gstreamer Version :
          gst-inspect-1.0 version 1.16.0
           GStreamer 1.16.0

*Audio and Video apsink:*


*RTSP appsrc:*




I am able to view the video and listen to audio (with some lag in audio).
But I could see there are some errors logged in the log file :

0:00:13.968698600  6268   06BB4DF0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink2> warning:
Pipeline construction is invalid, please add queues.
0:00:13.968762300  6268   06BB4DF0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink2> warning: Not
enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.
0:00:13.968936300  6268   06BB4DF0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink0> warning:
Pipeline construction is invalid, please add queues.
0:00:13.968969900  6268   06BB4DF0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink0> warning: Not
enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.
0:00:13.969087400  6268   06BB4DF0 WARN                     bin
gstbin.c:2773:gst_bin_do_latency_func:<media-pipeline> warning: Impossible
to configure latency: max 0:00:00.000000000 < min 0:00:01.800000001. Add
queues or other buffering elements.
0:00:13.971208500  6268   04403C48 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink2> warning:
Pipeline construction is invalid, please add queues.
0:00:13.972308800  6268   04403C48 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink2> warning: Not
enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.
0:00:13.972325700  6268   06B994B0 WARN                   libav
gstavauddec.c:628:gst_ffmpegauddec_drain:<avdec_aac0> send packet failed,
could not drain decoder
0:00:13.973876000  6268   04403C48 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink0> warning:
Pipeline construction is invalid, please add queues.
0:00:13.974672200  6268   04403C48 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<multiudpsink0> warning: Not
enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.
0:00:13.975559800  6268   04403C48 WARN                     bin
gstbin.c:2773:gst_bin_do_latency_func:<media-pipeline> warning: Impossible
to configure latency: max 0:00:00.000000000 < min 0:00:01.800000001. Add
queues or other buffering elements.
0:00:13.976542400  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning Pipeline
construction is invalid, please add queues.
(../libs/gst/base/gstbasesink.c(1218): gst_base_sink_query_latency ():
/GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink2:
Not enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.)
0:00:13.977377100  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning Pipeline
construction is invalid, please add queues.
(../libs/gst/base/gstbasesink.c(1218): gst_base_sink_query_latency ():
/GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink0:
Not enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.)
0:00:13.978032800  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning GStreamer
error: clock problem. (../gst/gstbin.c(2773): gst_bin_do_latency_func ():
/GstPipeline:media-pipeline:
Impossible to configure latency: max 0:00:00.000000000 < min
0:00:01.800000001. Add queues or other buffering elements.)
0:00:13.978168500  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning Pipeline
construction is invalid, please add queues.
(../libs/gst/base/gstbasesink.c(1218): gst_base_sink_query_latency ():
/GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink2:
Not enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.)
0:00:13.978243800  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning Pipeline
construction is invalid, please add queues.
(../libs/gst/base/gstbasesink.c(1218): gst_base_sink_query_latency ():
/GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink0:
Not enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.)
0:00:13.978314500  6268   0447EEA0 WARN               rtspmedia
rtsp-media.c:2973:default_handle_message: 06945418: got warning GStreamer
error: clock problem. (../gst/gstbin.c(2773): gst_bin_do_latency_func ():
/GstPipeline:media-pipeline:
Impossible to configure latency: max 0:00:00.000000000 < min
0:00:01.800000001. Add queues or other buffering elements.)
0:00:16.405496700  6268   0DFDE8C8 WARN            videodecoder
gstvideodecoder.c:2762:gst_video_decoder_prepare_finish_frame:<avdec_h264-1>
decreasing timestamp (0:00:00.427196500 < 0:00:00.497652761)


The audio and video is not in sync.
1. How can I sync the audio and video (appsink and appsrc) ?
2. What is the issue in the above pipeline, where should I add the queue ?

I am using 2 different approach to push the audio and video sink inside the
appsrc "need-data" signal as below:

void needData(GstElement *appsrc, guint unused, Context *ctx) {
        GstFlowReturn ret;
        GstBuffer *buffer;
        GstStructure *capsstruct;
        GstSample *sample = NULL;
        // gst_app_sink_pull_sample(GST_APP_SINK(ctx->glblapp->sink));
        gint rate = 1, frames = 30;
        while (sample == NULL) {
                if (appsrc == ctx->glblapp->videoappsrc) {
                        sample = gst_app_sink_pull_sample(GST_APP_SINK(ctx->glblapp->sink));
                        } else if (appsrc == ctx->glblapp->audioappsrc) {
                                sample = gst_app_sink_pull_sample(GST_APP_SINK(ctx->glblapp->aud_sink));
                        }
                }

                if (appsrc == ctx->glblapp->videoappsrc) {
                GstCaps *capsfromsink = fetch_caps(ctx->glblapp->sink, "sink");
                if (capsfromsink != NULL) {
                        g_object_set(G_OBJECT(appsrc), "caps", capsfromsink, NULL);
                        gst_caps_unref(capsfromsink);
                }
               
                buffer = gst_sample_get_buffer(sample);
                capsstruct = gst_caps_get_structure(capsfromsink, 0);
                if (gst_structure_has_field(capsstruct, FRAMERATE)) {
                        const GValue *framerate_val = gst_structure_get_value(capsstruct,
FRAMERATE);
                        if (G_VALUE_TYPE(framerate_val) == GST_TYPE_FRACTION) {
                                frames = gst_value_get_fraction_numerator(framerate_val);
                                rate = gst_value_get_fraction_denominator(framerate_val);
                        }
                }
                GST_BUFFER_PTS(buffer) = ctx->timestamp;
                GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(rate, GST_SECOND,
frames);
                ctx->timestamp += GST_BUFFER_DURATION(buffer);
                g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);
               
                }
                else if (appsrc == ctx->glblapp->audioappsrc) {
                        GstCaps *audiocapsfromsink = fetch_caps(ctx->glblapp->aud_sink, "sink");
                        if (audiocapsfromsink != NULL) {
                                g_object_set(G_OBJECT(appsrc), "caps", audiocapsfromsink, NULL);
                                gst_caps_unref(audiocapsfromsink);
                        }
                       
                        buffer = gst_sample_get_buffer(sample);
                        GstSegment *seg = gst_sample_get_segment(sample);
                        GstClockTime pts, dts;

                        /* Convert the PTS/DTS to running time so they start from 0 */
                        pts = GST_BUFFER_PTS(buffer);
                        if (GST_CLOCK_TIME_IS_VALID(pts))
                                pts = gst_segment_to_running_time(seg, GST_FORMAT_TIME, pts);

                        dts = GST_BUFFER_DTS(buffer);
                        if (GST_CLOCK_TIME_IS_VALID(dts))
                                dts = gst_segment_to_running_time(seg, GST_FORMAT_TIME, dts);
                       
                        if (buffer) {
                                /* Make writable so we can adjust the timestamps */
                                buffer = gst_buffer_copy(buffer);
                                GST_BUFFER_PTS(buffer) = pts;
                                GST_BUFFER_DTS(buffer) = dts;
                                g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);
                        }
                }

                gst_sample_unref(sample);
        }





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel