Appsrc buffer timestamping problems

[llama]

Hello,

I got problem with pipe like this: ("appsrc name=source ! video/x-raw, width=1920, height=1080, framerate=50/1, format=BGR ! videorate ! videoconvert ! x264enc ! avdec_h264 ! videoconvert ! autovideosink"

I have essentially this code:

  g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

static void cb_need_data(GstElement *appsrc, guint unused_size, gpointer user_data)  {
 // ...
  cv::Mat frame;
  cap >> frame;
  height = frame.rows;
        width = frame.cols;
        auto data = (guchar *)frame.data;
        size = frame.elemSize() * width * height;

        buffer = gst_buffer_new();
        memory = gst_allocator_alloc(NULL, size, NULL);
        gst_buffer_insert_memory(buffer, -1, memory);
        gst_buffer_map(buffer, &map, GST_MAP_WRITE);

        memcpy((guchar *)map.data, data, gst_buffer_get_size(buffer));

       GstClock *pipe_clock = gst_element_get_clock (GST_ELEMENT (pipeline));
                pipets_out = gst_clock_get_time (pipe_clock); // get current pipe clock
                pipets_out -= gst_element_get_base_time (GST_ELEMENT (pipeline)); // subtract pipe base time

      GST_BUFFER_PTS(buffer) = pipets_out;

     gst_buffer_unmap(buffer, &map);
     gst_buffer_unref(buffer);
}

If i add videoerate, it reports Got buffer with GST_CLOCK_TIME_NONE timestamp, discarding it. If i take videorate out, i got first frame and than it freezes.

I also made test on this pipe :
                                "appsrc name=source ! video/x-raw, width=1920, height=1080, framerate=50/1, format=BGR !\
                                 videoconvert ! video/x-raw, format=UYVY, width=1920, height=1080, framerate=50/1 !\
                                 identity name=appsrcid silent=false ! autovideosink")
And i got smooth playback.

Note that i used pipe running time to timestamp buffers because i need good synchronisation with alsasrc that will be added to stream.

What is wrong with my way of timestamping new buffers ? How to make it correct ?

Thanks in advance!

[Thornton, Keith]

Hi,
Have you got any queues in your pipeline? I suspect that x264enc and avdec_h264 need more than one buffer to operate on.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von LC
Gesendet: Freitag, 7. April 2017 09:05
An: [hidden email]
Betreff: Appsrc buffer timestamping problems

Hello,

I got problem with pipe like this: ("appsrc name=source ! video/x-raw, width=1920, height=1080, framerate=50/1, format=BGR ! videorate !
videoconvert ! x264enc ! avdec_h264 ! videoconvert ! autovideosink"

I have essentially this code:

  g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

static void cb_need_data(GstElement *appsrc, guint unused_size, gpointer
user_data)  {
 // ...
  cv::Mat frame;
  cap >> frame;
  height = frame.rows;
        width = frame.cols;
        auto data = (guchar *)frame.data;
        size = frame.elemSize() * width * height;

        buffer = gst_buffer_new();
        memory = gst_allocator_alloc(NULL, size, NULL);
        gst_buffer_insert_memory(buffer, -1, memory);
        gst_buffer_map(buffer, &map, GST_MAP_WRITE);

        memcpy((guchar *)map.data, data, gst_buffer_get_size(buffer));

       GstClock *pipe_clock = gst_element_get_clock (GST_ELEMENT (pipeline));
                pipets_out = gst_clock_get_time (pipe_clock); // get current pipe clock
                pipets_out -= gst_element_get_base_time (GST_ELEMENT (pipeline)); // subtract pipe base time

      GST_BUFFER_PTS(buffer) = pipets_out;

     gst_buffer_unmap(buffer, &map);
     gst_buffer_unref(buffer);
}

If i add videoerate, it reports Got buffer with GST_CLOCK_TIME_NONE timestamp, discarding it. If i take videorate out, i got first frame and than it freezes.

I also made test on this pipe :
                                "appsrc name=source ! video/x-raw, width=1920, height=1080, framerate=50/1, format=BGR !\
                                 videoconvert ! video/x-raw, format=UYVY, width=1920, height=1080, framerate=50/1 !\
                                 identity name=appsrcid silent=false !
autovideosink")
And i got smooth playback.

Note that i used pipe running time to timestamp buffers because i need good synchronisation with alsasrc that will be added to stream.

What is wrong with my way of timestamping new buffers ? How to make it correct ?

Thanks in advance!




--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Appsrc-buffer-timestamping-problems-tp4682567.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[llama]

I have a problem with timestamps because it does not play with x264enc . Autoviedosink shows the first frame and stuck.
I think the problem is a way i am timestamping buffers, but i am not sure how to do it.

[Arjen Veenhuizen]

Did you set the appsrc "format" property to GST_FORMAT_TIME instead of the default GST_FORMAT_BYTES?

[llama]

I did it g_object_set(G_OBJECT(appsrc),"format", GST_FORMAT_TIME, NULL);
Do i have to use some other type instead ?

[Arjen Veenhuizen]

That should suffice.
Also, set your encoder to zerolatency mode and make sure it is not using any b-frames. Furthermore, as suggested previously, insert queues before and after your encoder and decoder