Decode h264 buffer and get the cv MAT

[jetsonkiller]

Hi,

I have an HW that sending me h264 buffer of video in 30 fps and im trying to get the frame in cv mat format.
im getting the h264 frames in CB function like that:

void
H264Cb(uint8_t* buf, int bufLen)
{
  // doing something with the buffer
}

Do you have an idea to solve it?

[jagdspyder]

To implement this via Gstreamer, Rather than having a callback, it might be easier to have your data flow as GstBuffer via an element and use h264parse, avdec_h264 and videoconvert elements to parse, decode and convert color space to BGR then via a simple custom element you can write a chain function to convert the Gstbuffer to cv::Mat.
This should help you out https://stackoverflow.com/questions/65308163/converting-gstreamer-yuv-420-i420-raw-frame-to-opencv-cvmat

If you wanna do it in your callback function, then I would suggest using ffmpeg/libav to do so. Though you will obtain your frames in AvFrame, you can still convert them back to cv::Mat via sws_scale converter