GStreamer-devel

How to capture a still image while previewing live video?

Classic

List

Threaded

7 messages Options

Paul Stuart

How to capture a still image while previewing live video?

Hi,

We need to simultaneously capture/preview video on a TI DM365 while asynchronously taking JPEG stills from the same video stream. Playing around with gst-launch, it only seems to work correctly when v4l2src num-buffers=1. Sample working pipeline:

gst-launch -v v4l2src num-buffers=1 always-copy=FALSE input-src=composite \
! TIVidResize contiguousInputFrame=TRUE \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! queue \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! queue \
! filesink location=output.jpg

If I modify the above to remove num-buffers=1, or even just set num-buffers=2, it hangs and never produces a valid JPEG. I would have expected it to just overwrite output.jpg

If, I change this to something that is, for instance, previewing the video at the same time, things break as well. Sample Pipeline:

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! queue \
! tee name= t \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Is it possible to create a gstreamer pipeline using “standard” parts to accomplish what I need, or will I need to modify my JPEG encoder element to make this work?

Thanks!

Paul

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Wes Miller

Re: How to capture a still image while previewing live video?

Administrator

Hi,

We need to simultaneously capture/preview video on a TI DM365 while asynchronously taking JPEG stills from the same video stream. Playing around with gst-launch, it only seems to work correctly when v4l2src num-buffers=1. Sample working pipeline:

gst-launch -v v4l2src num-buffers=1 always-copy=FALSE input-src=composite \
! TIVidResize contiguousInputFrame=TRUE \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! queue \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! queue \
! filesink location=output.jpg

Have you considered multifilesink? Makes a single output file for each frame that arrives in the sink. The downside of this is, of course, it makes a file for EVERY frame. Also, because you are not keeping the entire stream you may loose (some?) timestamps and tags. I don't know for sure. If that information is really important to you, consider turning your stream in mjpeg and saving it into a container, Matroska is a good choice. You will still have your individual frames that you can later extract and only one frame.

Wes

If I modify the above to remove num-buffers=1, or even just set num-buffers=2, it hangs and never produces a valid JPEG. I would have expected it to just overwrite output.jpg

If, I change this to something that is, for instance, previewing the video at thesame time, things break as well. Sample Pipeline:

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! queue \
! tee name= t \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Is it possible to create a gstreamer pipeline using “standard” parts to accomplish what I need, or will I need to modify my JPEG encoder element tomake this work?

Thanks!

Paul

CONFIDENTIALITY NOTE:

This e-mail and any attachments are confidential. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received this e-mail in error, please notify us immediately by returning it to the sender and delete this copy from your system. Thank you for your cooperation.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Kapil Agrawal

Re: How to capture a still image while previewing live video?

Hello Paul,

I think your pipeline should look like, added a queue after tee

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! tee name= t \
! queue
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Regards
Kapil

--
www.mediamagictechnologies.com (Gstreamer, ffmpeg, Red5, Streaming)
twitter handle: @gst_kaps
http://www.linkedin.com/in/kapilagrawal

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Paul Stuart

Re: How to capture a still image while previewing live video?

Hello Kapil,
Right you are!

But...still doesn't work. I'm strongly suspecting TI's TIImgenc1 implementation at this point. Looking at the source code for it, it looks like it *should* work, as they've implemented a circular buffer system to buffer incoming and outgoing buffers. When I turn logging on, I see the following snippet:

GST_ERROR_SYSTEM gstelement.c:1761:gst_element_message_full:<tiimgenc10> posting message: Failed to queue input buffer into circular buffer

0:06:58.383303446 2220 0xfabc0 INFO GST_ERROR_SYSTEM gstelement.c:1784:gst_element_message_full:<tiimgenc10> posted error message: Failed to queue input buffer into
circular buffer

So...digging deeper into TIImgenc1

Thanks!
Paul

Kapil Agrawal wrote:

Hello Paul,

I think your pipeline should look like, added a queue after tee

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! tee name= t \
! queue
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Regards
Kapil

--
www.mediamagictechnologies.com (Gstreamer, ffmpeg, Red5, Streaming)
twitter handle: @gst_kaps
http://www.linkedin.com/in/kapilagrawal

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Kapil Agrawal

Re: How to capture a still image while previewing live video?

ah I guess I know the issue,
most probably v4l2src is not copying the captured video in dm365 dmai buffer, and that could be an issue. I dont recall it immediately, but just a clue ?

Best Luck
Kapil

On Wed, Sep 14, 2011 at 11:00 PM, Paul Stuart <[hidden email]> wrote:

Hello Kapil,
Right you are!

But...still doesn't work. I'm strongly suspecting TI's TIImgenc1 implementation at this point. Looking at the source code for it, it looks like it *should* work, as they've implemented a circular buffer system to buffer incoming and outgoing buffers. When I turn logging on, I see the following snippet:

GST_ERROR_SYSTEM gstelement.c:1761:gst_element_message_full:<tiimgenc10> posting message: Failed to queue input buffer into circular buffer

0:06:58.383303446 2220 0xfabc0 INFO GST_ERROR_SYSTEM gstelement.c:1784:gst_element_message_full:<tiimgenc10> posted error message: Failed to queue input buffer into
circular buffer

So...digging deeper into TIImgenc1

Thanks!
Paul

Kapil Agrawal wrote:
Hello Paul,

I think your pipeline should look like, added a queue after tee

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! tee name= t \
! queue
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Regards
Kapil

--
www.mediamagictechnologies.com (Gstreamer, ffmpeg, Red5, Streaming)
twitter handle: @gst_kaps
http://www.linkedin.com/in/kapilagrawal

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
http://www.linkedin.com/in/kapilagrawal
www.mediamagictechnologies.com
(Consultancy - Multimedia, Gstreamer, ffmpeg, Streaming for platforms- linux, windows, TI, android)

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Paul Stuart

Re: How to capture a still image while previewing live video?

Fixed it - Turns out it was a couple issues in the implementation of TI's TIImgenc1 element.

They spin up a separate JPEG encoding thread that takes frames from a circular buffer populated by the chain routine, and then pushes encoded frames onto the source pad. The encoded frames are stored in buffers pulled from a buffTab. At the end of the loop, the encoded buffer's "useMask" was not being set to zero (unused), rather it was being &= with ~0x2. So, maybe something got out of sync with the flags, not sure, but setting the useMask to zero made everything peachy.

Also had issues with their setting of source caps each time a new frame was encoded. Not sure why that didn't work, but removing the code proved to move things along from a deadlock. Will revisit this one.

Thanks for everyone's help!

paul

Kapil Agrawal wrote:

ah I guess I know the issue,
most probably v4l2src is not copying the captured video in dm365 dmai buffer, and that could be an issue. I dont recall it immediately, but just a clue ?

Best Luck
Kapil

On Wed, Sep 14, 2011 at 11:00 PM, Paul Stuart <[hidden email]> wrote:

Hello Kapil,
Right you are!

But...still doesn't work. I'm strongly suspecting TI's TIImgenc1 implementation at this point. Looking at the source code for it, it looks like it *should* work, as they've implemented a circular buffer system to buffer incoming and outgoing buffers. When I turn logging on, I see the following snippet:

GST_ERROR_SYSTEM gstelement.c:1761:gst_element_message_full:<tiimgenc10> posting message: Failed to queue input buffer into circular buffer

0:06:58.383303446 2220 0xfabc0 INFO GST_ERROR_SYSTEM gstelement.c:1784:gst_element_message_full:<tiimgenc10> posted error message: Failed to queue input buffer into
circular buffer

So...digging deeper into TIImgenc1

Thanks!
Paul

Kapil Agrawal wrote:
Hello Paul,

I think your pipeline should look like, added a queue after tee

gst-launch -v v4l2src always-copy=FALSE input-src=composite \
! tee name= t \
! queue
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! tidisplaysink2 video-standard=vga display-output=lcd mmap-buffer=FALSE dma-copy=TRUE \
t. \
! queue \
! 'video/x-raw-yuv,width=(int)736,height=(int)480' \
! videorate \
! TIImgenc1 iColorSpace=UYVY oColorSpace=YUV420P qValue=75 engineName=codecServer \
! filesink location=output.jpg

Regards
Kapil

--
www.mediamagictechnologies.com (Gstreamer, ffmpeg, Red5, Streaming)
twitter handle: @gst_kaps
http://www.linkedin.com/in/kapilagrawal

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
http://www.linkedin.com/in/kapilagrawal
www.mediamagictechnologies.com
(Consultancy - Multimedia, Gstreamer, ffmpeg, Streaming for platforms- linux, windows, TI, android)

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Wes Miller

Re: [SPAM (Keyword)] - Re: How to capture a still image while previewing live video?

Administrator

Also had issues with their setting of source caps each time a new frame was encoded. Not sure why that didn't work, but removing the code proved to move things along from a deadlock. Will revisit this one.

When I was using a dm365 I discovered that having caps in things like udpsrc where the caps can be a property of the element, did n't work well. Moving the caps into their own pipeline stage fixed the problem. Might this help you?

Thanks for everyone's help!

paul