vaapijpegdec fails to decode Logitech C930e jpeg image

[GStreamer-devel mailing list]

Dear Engineers,

    We use “vaapijpegdec” with “vaapipostproc” to decode video in stream of jpeg images at frame size of 1280 x 720 from a number of webcams.  It performs well for Logitech C500, C525, C615, C920, C922, as well as Microsoft LifeCam HD1080.  However, it fails to decompress video of Logitech C930e.  It may produce one or two frames, but then the pipeline stalls.  The decoded first two frames are not right either, as it misses the color and contains ghost images.  For your information, the “guvcview” software decoder can decompress the jpeg images of C930e, as well as those from all other webcams, without any problem.  It appears C930e contains extra tag that is rejected by “vaapijpegdec” and it causes the pipeline to halt.   We also test Gstreamer software decoder “jpegdec” with “videoconvert”.  It fails on C930e also, although it passes all other webcams.  For C930e, “jpegdec”  produces wrong images similar to that of “vaapijpegdec” and Gstreamer prints out many error messages.

   I did create this issue 6 days ago on gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134.  I got a comment from Olivier Crête with message of “Can you attach a couple pictures from the C930e that fails?”.  I generated couple of jpeg files from C930e.  But my email sent back to him was rejected.  I am not sure he has received my response and whether he will work on it or not.  I then attached these two jpeg files to the open issue 134.  Here is my response to Olivier, and I hope some of you can look into this issue.

    I recorded two consecutive jpeg frames of C930e, which is not good for vaapijpegdec (as well as jpegdec).   For your comparison, I also recorded two consecutive jpeg frames of C525, which is good for vaapijpegdec and jpegdec.   Both of them can be open by VLC player.  However, I was surprised to find the C525 jpeg cannot be open by certain photo viewers.  I hope these data are useful for you to find the problem.  If you need anything else, just let me know.

    By the way, I use them in C++ program of Gstreamer Pipeline under Linux OS.  I only tested Logitech models up to C930e.  I suspect most new models might have the same problem.  I also use vaapih264enc and vaapipostproc functions, and they are good for all the webcams I have tested.

    Best regards,

Ching Lai.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Fri, 15 Feb 2019 at 11:15, chingmlai--- via gstreamer-devel wrote:
>

[..]

>
>    I did create this issue 6 days ago on
>    gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134.  I got a
>    comment from Olivier Crête with message of “Can you attach a couple
>    pictures from the C930e that fails?”.  I generated couple of jpeg files
>    from C930e.  But my email sent back to him was rejected.  I am not sure he
>    has received my response and whether he will work on it or not.  I then
>    attached these two jpeg files to the open issue 134.  Here is my response
>    to Olivier, and I hope some of you can look into this issue.

Hi,

Would you please upload the images in the created issue?
gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134

And also attach, in gitlab's issue, the a debug log to know why the pipeline
stalls. Also describe how is your decoding pipeline.

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[chingmlai]

Hi,

    In our system, the video data is captured in jpeg/image format by using v4l2 library functions.  The pipeline uses elements of appsrc, vaapijpegdec, vaapipostproc, and appsink.

    The appsrc is set with src_caps of
    src_caps = gst_caps_new_simple ("image/jpeg",   "width", G_TYPE_INT, 1280 "height", G_TYPE_INT, 720,
                                "framerate", GST_TYPE_FRACTION, 30, 1, NULL);
    The appsink is set with sink_caps of
    sink_caps = gst_caps_new_simple ("video/x-raw",  "width", G_TYPE_INT, 1280,  "height", G_TYPE_INT, 720,
                        "format", G_TYPE_STRING, "NV12",   "framerate", GST_TYPE_FRACTION, 30, 1, NULL);

    They are added and linked to the pipeline by
    gst_bin_add_many (GST_BIN(pipeline), appsrc, vaapijpegdec, vaapipostproc, appsink, NULL);
    gst_element_link_many(appsrc, vaapijpegdec, vaapipostproc, appsink, NULL);

    For each captured frame,

  Step 1. The jpeg data is pushed into appsrc by calling
  g_signal_emit_by_name (appsrc, "push-buffer", gst_buffer, &ret);
  where the gst_buffer contains data copied from the captured jpeg data, as well as the time stamp and duration.

  Step 2. The decoded data is pulled from appsink by calling
  g_signal_emit_by_name (appsink, "pull-sample", &gst_sample, &ret);
  where the returning gst_sample contains the decoded NV12 data.

    In our test, the program is running under Linux OS with appsrc and appsink as global Gstreamer elements.  We experienced that in Step 1, the function g_signal_emit_by_name (appsrc, "push-buffer", gst_buffer, &ret) always returns immediately.  However, in Step 2, g_signal_emit_by_name (appsink, "pull-sample", &gst_sample, &ret) does not return immediately.  For C930e webcam, it may never return after one or two frames.  We described this indefinite waiting status as stall in the pipeline.

    We do not use formal debug logging.  But we tested the process by comparing it with a software decoding function, which produces correct results for all webcams.  When the software decoder option is selected, the captured jpeg data is decoded into YUYV format by calling

    m_pJpegDecoder->jpeg_decode((uint8_t *)yuyv_data, (uint8_t *)jpeg_data, jpeg_size);
    where m_pJpegDecode is created and initialized by
    m_pJpegDecoder = new CJpegDecoder();
    m_pJpegDecoder->jpeg_init_decoder(1280, 720);

   The yuyv_data is then converted to NV12 format.  The source code of this  CjpegDecoder class is attached here and uploaded to website of gstreamer-vaapi open issue 134.


    For your information, I received message from 'gstreamer-devel' following my last email -

    “Your email to 'gstreamer-devel' with the subject vaapijpegdec fails to decode Logitech C930e jpeg image
Is being held until the list moderator can review it for approval.  The reason it is being held:
    Post by non-member to a members-only list.”

    Best regards,

Ching Lai

----- Original Message -----
From: "Víctor Jáquez" <[hidden email]>
To: "Discussion of the development of and with GStreamer" <[hidden email]>
Cc: [hidden email], [hidden email]
Sent: Tuesday, February 19, 2019 5:49:52 AM
Subject: Re: vaapijpegdec fails to decode Logitech C930e jpeg image

On Fri, 15 Feb 2019 at 11:15, chingmlai--- via gstreamer-devel wrote:
>

[..]

>
>    I did create this issue 6 days ago on
>    gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134.  I got a
>    comment from Olivier Crête with message of “Can you attach a couple
>    pictures from the C930e that fails?”.  I generated couple of jpeg files
>    from C930e.  But my email sent back to him was rejected.  I am not sure he
>    has received my response and whether he will work on it or not.  I then
>    attached these two jpeg files to the open issue 134.  Here is my response
>    to Olivier, and I hope some of you can look into this issue.

Hi,

Would you please upload the images in the created issue?
gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134

And also attach, in gitlab's issue, the a debug log to know why the pipeline
stalls. Also describe how is your decoding pipeline.

vmjl

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel