Framerate issue with v4l2sink

[m.lenz]

We are streaming data from appsrc into a v4l2sink. Therefore, we are using v4l2loopback on /dev/videoX. The camera has a fix resolution of 640x480 and a variable framerate. The framerate is set up at the beginning with a library provided by the manufacturer. The data is 16 Bit Gray LE.

The problem is, that Gstreamer always wants to put 33 frame per second into the sink. So, if we choose 60fps, it’s 33fps and if we choose 30fps, it isn’t working. Every value bigger than 33fps is working, but only 33fps in real and every value lower than 33 fps is not working at all.

Before this, we already set the framerate and pushed data into a multifilesink or a udpsink with correct frame rate at the end. So, the initialization should not be the problem. Is there anything special to remind working with v42sink and v4l2loopback?

This is from our application, I removed error checks etc:

cb_need_data (GstElement *appsrc,

          guint       unused_size,

          gpointer    user_data)

{

  static GstClockTime timestamp = 0;

  GstBuffer *buffer;

  GstFlowReturn ret;

   

  GstMapInfo map;

  eDALProxy640USBErr eReturnCode;

 

  buffer = gst_buffer_new_and_alloc(IRIMAGE_NBPIXELS*2); //allocate 2 Bytes for every pixel

  if(gst_buffer_map (buffer, &map, GST_MAP_WRITE)) {

while(true)

         {

             eReturnCode = Proxy640USB_GetImage(m_Handle, (short unsigned int*)map.data, paMeta, GETIMAGE_TIMEOUT);

             if (eReturnCode == eProxy640USBSuccess)

                    break;

         }

  }

 

  GST_BUFFER_PTS (buffer) = timestamp;

  GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 33);

 

  timestamp += GST_BUFFER_DURATION (buffer);

  g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);

  gst_buffer_unmap(buffer, &map);

  gst_buffer_unref(buffer);

}

 

 

gint main (gint argc, gchar *argv[])

{

sprintf(DEVICE, "/dev/video0");

// GStreamer

  GstElement *pipeline, *converter, *videosink, *videosrc, *appsrc;

  gst_init (&argc, &argv);

  loop = g_main_loop_new (NULL, FALSE);

 

// LWIR device

  m_Handle = NULL;

  eAGCProcessingValue agcVal = eAGCEnhanced;

  eDALProxy640USBErr eReturnCode;

  eReturnCode = Proxy640USB_GetModuleCount(&iCount);

 

// Connect to LWIR
eReturnCode = Proxy640USB_ConnectToModule(0, &m_Handle);

 

// Set Processing depending of parameters (NUC/AGC/Shutterless/Framerate)

  Proxy640USB_SetNUCProcessing(m_Handle, bBP, nuc);

  Proxy640USB_SetShutterLessProcessing(m_Handle, bSL);

  Proxy640USB_SetFloatFeature(m_Handle, efFrameRate, FRAMERATE);

  Proxy640USB_SetAGCProcessing(m_Handle, agcVal);

  Proxy640USB_SetUIntFeature(m_Handle, eiVideoUSBOutActivation, 1);

 

// Setup Pipeline

  pipeline = gst_pipeline_new ("LWIR-streamer");

  appsrc = gst_element_factory_make ("appsrc", "source");

  videosrc = gst_element_factory_make("videotestsrc", "source");

  converter = gst_element_factory_make ("videoconvert", "converter");

  videosink = gst_element_factory_make ("v4l2sink", "videosink");

  g_object_set (videosink, "device", DEVICE, NULL);

 

// Setting Capabilities

  g_object_set (G_OBJECT (appsrc), "caps",

        gst_caps_new_simple ("video/x-raw",

                     "format", G_TYPE_STRING, "GRAY16_LE",

                     "width", G_TYPE_INT, IRIMAGE_WIDTH,

                     "height", G_TYPE_INT, IRIMAGE_HEIGHT,

                     "framerate", GST_TYPE_FRACTION, FRAMERATE, 1,

                     NULL), NULL);

 

  gst_bin_add_many (GST_BIN (pipeline), appsrc, converter, videosink,NULL);

  gst_element_link_many (appsrc, converter, videosink, NULL);

 

// Setup appsrc to cd_need_data

  g_object_set (G_OBJECT (appsrc),

        "stream-type", 0,

        "format", GST_FORMAT_TIME, NULL);

  g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data), NULL);

 

  /* play */

  gst_element_set_state (pipeline, GST_STATE_PLAYING);

  g_main_loop_run (loop);

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mardi 24 mars 2020 à 18:10 +0000, Lenz, Mika a écrit :

> We are streaming data from appsrc into a v4l2sink. Therefore, we are
> using v4l2loopback on /dev/videoX. The camera has a fix resolution of
> 640x480 and a variable framerate. The framerate is set up at the
> beginning with a library provided by the manufacturer. The data is 16
> Bit Gray LE.
>
> The problem is, that Gstreamer always wants to put 33 frame per
> second into the sink. So, if we choose 60fps, it’s 33fps and if we
> choose 30fps, it isn’t working. Every value bigger than 33fps is
> working, but only 33fps in real and every value lower than 33 fps is
> not working at all.
>
> Before this, we already set the framerate and pushed data into a
> multifilesink or a udpsink with correct frame rate at the end. So,
> the initialization should not be the problem. Is there anything
> special to remind working with v42sink and v4l2loopback?

Appart that v4l2loopback driver should be rewritten from scratch in
order to make it work properly, no.

On a more serious note, what GStreamer does is list the frame interval
advertise by your driver. So if 33fps is the only thing you manage to
get to negotiate with v4l2sink/v4l2loopback, I'd first analyze the
driver. This driver have had issues from over a decade, including
buffer overflow and all the things you don't want in your product. So I
wouldn't be surprise if more bugs exist in regard to frame interval
handling.

>
> This is from our application, I removed error checks etc:
>
> cb_need_data (GstElement *appsrc,
>           guint       unused_size,
>           gpointer    user_data)
> {
>   static GstClockTime timestamp = 0;
>   GstBuffer *buffer;
>   GstFlowReturn ret;
>    
>   GstMapInfo map;
>   eDALProxy640USBErr eReturnCode;
>  
>   buffer = gst_buffer_new_and_alloc(IRIMAGE_NBPIXELS*2); //allocate 2
> Bytes for every pixel
>   if(gst_buffer_map (buffer, &map, GST_MAP_WRITE)) {
> while(true)
>          {
>              eReturnCode = Proxy640USB_GetImage(m_Handle, (short
> unsigned int*)map.data, paMeta, GETIMAGE_TIMEOUT);
>              if (eReturnCode == eProxy640USBSuccess)
>                     break;
>          }
>   }
>  
>   GST_BUFFER_PTS (buffer) = timestamp;
>   GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1,
> GST_SECOND, 33);
>  
>   timestamp += GST_BUFFER_DURATION (buffer);
>   g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
>   gst_buffer_unmap(buffer, &map);
>   gst_buffer_unref(buffer);
> }
>  
>  
> gint main (gint argc, gchar *argv[])
> {
> sprintf(DEVICE, "/dev/video0");
> // GStreamer
>   GstElement *pipeline, *converter, *videosink, *videosrc, *appsrc;
>   gst_init (&argc, &argv);
>   loop = g_main_loop_new (NULL, FALSE);
>  
> // LWIR device
>   m_Handle = NULL;
>   eAGCProcessingValue agcVal = eAGCEnhanced;
>   eDALProxy640USBErr eReturnCode;
>   eReturnCode = Proxy640USB_GetModuleCount(&iCount);
>  
> // Connect to LWIR
> eReturnCode = Proxy640USB_ConnectToModule(0, &m_Handle);
>  
> // Set Processing depending of parameters
> (NUC/AGC/Shutterless/Framerate)
>   Proxy640USB_SetNUCProcessing(m_Handle, bBP, nuc);
>   Proxy640USB_SetShutterLessProcessing(m_Handle, bSL);
>   Proxy640USB_SetFloatFeature(m_Handle, efFrameRate, FRAMERATE);
>   Proxy640USB_SetAGCProcessing(m_Handle, agcVal);
>   Proxy640USB_SetUIntFeature(m_Handle, eiVideoUSBOutActivation, 1);
>  
> // Setup Pipeline
>   pipeline = gst_pipeline_new ("LWIR-streamer");
>   appsrc = gst_element_factory_make ("appsrc", "source");
>   videosrc = gst_element_factory_make("videotestsrc", "source");
>   converter = gst_element_factory_make ("videoconvert", "converter");
>   videosink = gst_element_factory_make ("v4l2sink", "videosink");
>   g_object_set (videosink, "device", DEVICE, NULL);
>  
> // Setting Capabilities
>   g_object_set (G_OBJECT (appsrc), "caps",
>         gst_caps_new_simple ("video/x-raw",
>                      "format", G_TYPE_STRING, "GRAY16_LE",
>                      "width", G_TYPE_INT, IRIMAGE_WIDTH,
>                      "height", G_TYPE_INT, IRIMAGE_HEIGHT,
>                      "framerate", GST_TYPE_FRACTION, FRAMERATE, 1,
>                      NULL), NULL);
>  
>   gst_bin_add_many (GST_BIN (pipeline), appsrc, converter,
> videosink,NULL);
>   gst_element_link_many (appsrc, converter, videosink, NULL);
>  
> // Setup appsrc to cd_need_data
>   g_object_set (G_OBJECT (appsrc),
>         "stream-type", 0,
>         "format", GST_FORMAT_TIME, NULL);
>   g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data),
> NULL);
>  
>   /* play */
>   gst_element_set_state (pipeline, GST_STATE_PLAYING);
>   g_main_loop_run (loop);
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[m.lenz]

I recognized that I forgot to change back one line in my code. I did this for
testing reason.

Following

  GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND,
33);

of course has to be

  GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND,
FRAMERATE);


I also forced fps by v4l2loopback, because this parameters wasn't handed
over, this was every time 30fps. But with correct fps in v4l2 device it
still shows the same problems.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mercredi 25 mars 2020 à 04:57 -0500, m.lenz a écrit :

> I recognized that I forgot to change back one line in my code. I did this for
> testing reason.
>
> Following
>
>   GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND,
> 33);
>
> of course has to be
>
>   GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND,
> FRAMERATE);
>
>
> I also forced fps by v4l2loopback, because this parameters wasn't handed
> over, this was every time 30fps. But with correct fps in v4l2 device it
> still shows the same problems.

It's a pandora box, but if you need more help, share a log with
GST_DEBUG="v4l2*:7" for when it fails. That being said, I don't really
like to support this driver since it has basically nothing to do with
V4L2. If you want to get somewhere, I'd first running and fixing
v4l2-compliance test result in that driver.

>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel