GStreamer-devel

I need help verifying there's a bug in the Intel Media SDK Gstreamer element

classic Classic list List threaded Threaded
♦
♦
 2 messages Options
Ben Rush
Reply | Threaded
Open this post in threaded view
♦
♦
|

I need help verifying there's a bug in the Intel Media SDK Gstreamer element

Ben Rush
The whole point of this email is to verify I've encountered a bug, and not that I'm doing some stupid. 

I've been successfully using the Intel Media SDK GStreamer element from their repo here: https://github.com/intel/gstreamer-media-SDK/tree/topic_linux_and_window for a while now. My successful setup is I've got a camera connected to a machine (a Windows machine), and then I'm using the GStreamer's RTSP server to take the camera feed, encode it on the GPU (with the help of the Intel GStreamer h265/h264 plugins), and then broadcast it. There were a few bumps along the way (mostly bugs and issues in their code), but I've since got it resolved and have things working. GStreamer is awesome. My point thus far is I've been able to get these Intel plugins to work with the rest of GStreamer. 

I've been given another use case, and I'm encountering what appears to be another issue in Intel's code. I would normally think I'm doing something dumb, but given the # of issues I ran into when getting their elements to work in the aforementioned setup, I'm considering I've run into another bump. 

Essentially I'm NOW writing a video player, and need to decode traffic. So I'm on the player side of the equation now. Again, I'm using the Intel Media SDK to decode. Now, according to gst-inspect, the output pins for either the h264 or h265 intel decoders should produce BGRA. This is important because the user interface control I'm using doesn't support NV12, and requires a BGRA buffer. I don't want to do the conversion from NV12 to RGB* on the CPU 'cause that defeats the whole reason I'm using the Intel Media SDK in the first place. Example output from gst-inspect on their h265 element: 

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
              alignment: au
                profile: { (string)main, (string)main-10 }
          stream-format: byte-stream

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:MFXSurface)
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2, (string)ENCODED }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1

However, NOTHING I do successfully decodes into BGRA. I can write the following gst-launch script and see the video stream just fine when specifying NV12: 

gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink 

If I change the format to BGRA, however, the player doesn't pop up:

 gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink   

What's weird is I see both the CPU and GPU chewing away doing the above script, but nothing displays. By dumping out the dot files, I see the pipeline fails to successfully enter the PLAY state. Digging further...

If I attach and debug the Intel MFX element and run the above script that fails, I see the following line fails: https://github.com/intel/gstreamer-media-SDK/blob/94ab6055b60e9d67a3ccb6b13fddb6c3076e204d/gst/mfx/gstmfxdec.c#L424. It looks like this: 

return gst_video_decoder_finish_frame (vdec, frame); 

The function returns GST_FLOW_NOT_NEGOTIATED.  Setting GST_DEBUG=4, I see the following warning which further indicates that some kind of negotiation failed: 

WARN                GST_PADS gstpad.c:4230:gst_pad_peer_query:<mfxdec_h264-0:src> could not send sticky events 

What I need is confirmation that the above gst-launch commands OUGHT to give me BGRA at the autovideosink if things were working correctly in the Intel decoders. Everything thus far seems to indicate I'm running into some bug. To be doubly sure, I see the same behavior when I replace autovideosink with appsink and write some C++ around it. 

Thanks for your expertise and extra pair of eyes. 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Ben Rush
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: I need help verifying there's a bug in the Intel Media SDK Gstreamer element

Ben Rush
Doh! I noticed a typo in my email. The second gst-launch script should be: 

 gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=BGRA' ! autovideosink   

My bad. 

On Tue, Jun 18, 2019 at 12:37 PM Ben Rush <[hidden email]> wrote:
The whole point of this email is to verify I've encountered a bug, and not that I'm doing some stupid. 

I've been successfully using the Intel Media SDK GStreamer element from their repo here: https://github.com/intel/gstreamer-media-SDK/tree/topic_linux_and_window for a while now. My successful setup is I've got a camera connected to a machine (a Windows machine), and then I'm using the GStreamer's RTSP server to take the camera feed, encode it on the GPU (with the help of the Intel GStreamer h265/h264 plugins), and then broadcast it. There were a few bumps along the way (mostly bugs and issues in their code), but I've since got it resolved and have things working. GStreamer is awesome. My point thus far is I've been able to get these Intel plugins to work with the rest of GStreamer. 

I've been given another use case, and I'm encountering what appears to be another issue in Intel's code. I would normally think I'm doing something dumb, but given the # of issues I ran into when getting their elements to work in the aforementioned setup, I'm considering I've run into another bump. 

Essentially I'm NOW writing a video player, and need to decode traffic. So I'm on the player side of the equation now. Again, I'm using the Intel Media SDK to decode. Now, according to gst-inspect, the output pins for either the h264 or h265 intel decoders should produce BGRA. This is important because the user interface control I'm using doesn't support NV12, and requires a BGRA buffer. I don't want to do the conversion from NV12 to RGB* on the CPU 'cause that defeats the whole reason I'm using the Intel Media SDK in the first place. Example output from gst-inspect on their h265 element: 

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
              alignment: au
                profile: { (string)main, (string)main-10 }
          stream-format: byte-stream

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:MFXSurface)
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2, (string)ENCODED }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE, (string)YUY2 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1

However, NOTHING I do successfully decodes into BGRA. I can write the following gst-launch script and see the video stream just fine when specifying NV12: 

gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink 

If I change the format to BGRA, however, the player doesn't pop up:

 gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin ! 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink   

What's weird is I see both the CPU and GPU chewing away doing the above script, but nothing displays. By dumping out the dot files, I see the pipeline fails to successfully enter the PLAY state. Digging further...

If I attach and debug the Intel MFX element and run the above script that fails, I see the following line fails: https://github.com/intel/gstreamer-media-SDK/blob/94ab6055b60e9d67a3ccb6b13fddb6c3076e204d/gst/mfx/gstmfxdec.c#L424. It looks like this: 

return gst_video_decoder_finish_frame (vdec, frame); 

The function returns GST_FLOW_NOT_NEGOTIATED.  Setting GST_DEBUG=4, I see the following warning which further indicates that some kind of negotiation failed: 

WARN                GST_PADS gstpad.c:4230:gst_pad_peer_query:<mfxdec_h264-0:src> could not send sticky events 

What I need is confirmation that the above gst-launch commands OUGHT to give me BGRA at the autovideosink if things were working correctly in the Intel decoders. Everything thus far seems to indicate I'm running into some bug. To be doubly sure, I see the same behavior when I replace autovideosink with appsink and write some C++ around it. 

Thanks for your expertise and extra pair of eyes. 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page