Take snapshot from running pipeline in gstreamer with iOS

[Macjon]

I'm trying to create a snapshot from a running pipeline in iOS. I use a button to take the snapshot.

I have the following pipeline:
udpsrc auto-multicast=true address=224.1.1.1 port=5004"
+ " ! application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAH6aAQAYZAA\\,aM4wpIAA\", payload=(int)96, ssrc=(uint)19088743, timestamp-offset=(uint)0, seqnum-offset=(uint)0"
+ " ! rtpjitterbuffer latency=400"
+ " ! rtph264depay ! avdec_h264 ! videoconvert"
+ " ! tee name=snapshot snapshot. ! queue ! valve name=snap drop=true ! jpegenc ! filesink name=filepath location=screenshot.jpg async=false snapshot. ! queue"
+ " ! autovideosink

So I use the following code in my button to handle the valve:
GstElement *element = gst_bin_get_by_name (GST_BIN (pipeline), "snap");
if (strcmp("drop", "drop") == 0)
{
    gboolean prop_val = FALSE;

    // if the property value is true, then send an EOS.
    if ( strcmp("false", "true") == 0 )
    {
        gst_element_send_event(element, gst_event_new_eos());
        prop_val = TRUE;
    } else {
        prop_val = FALSE;
    }

    g_object_set (element, "drop", prop_val, NULL);
}

But with this code I can only take one screenshot. And I can't set the filename of the image.

How can I save the screenshot without blocking the stream and save the image in the documents folder with a custom name every time the button is clicked?

[Sebastian Dröge-3]

On Mo, 2016-07-04 at 09:12 -0700, Macjon wrote:
>
> How can I save the screenshot without blocking the stream and save the image
> in the documents folder with a custom name every time the button is clicked?

Check the last-sample property on the video-sink. With that you can get
access to the last displayed video frame, and e.g.
with gst_video_convert_sample() you can then convert this to a JPG.

--

Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Macjon]

Is there any example code on how to use the last-sample property and the gst_video_convert_sample()?

[Sebastian Dröge-3]

On Tue, 2016-08-16 at 06:19 -0700, Macjon wrote:
> Is there any example code on how to use the last-sample property and
> the gst_video_convert_sample()?

Probably but I can't remember any right now. It's very simple though.

You just get the value of the "last-sample" property at any time you
want to take a snapshot, and then pass the sample you receive to
gst_video_convert_sample(). As a parameter this also takes the caps you
want to convert it to, so you could pass in caps for "image/png" for
example.

The resulting GstSample this returns will then contain a buffer of the
format you specified via caps.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Macjon]

I've got a peace of code that works. I removed the tee from my pipeline. And now I have the following objective-c code:

-(UIImage*) takeSnapshot
{
    GstSample *videobuffer = NULL;
    GstCaps *caps;
    gint width, height;
    GstMapInfo map;

    g_object_get(G_OBJECT(video_sink), "last-sample", &videobuffer, NULL);

    if (videobuffer)
    {
        caps = gst_sample_get_caps(videobuffer);

        if (!caps) {
            return NULL;
        }

        GstStructure *s = gst_caps_get_structure(caps, 0);
        /* we need to get the final caps on the buffer to get the size */
        gboolean res;
        res = gst_structure_get_int (s, "width", &width);
        res |= gst_structure_get_int (s, "height", &height);
        if (!res) {
            return NULL;
        }

        GstBuffer *snapbuffer = gst_sample_get_buffer(videobuffer);
        if (snapbuffer && gst_buffer_map (snapbuffer, &map, GST_MAP_READ))
        {
            CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,
                                                                  map.data,
                                                                  height * width * 4,
                                                                  NULL);

            CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
            CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
            CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;

            CGImageRef imageRef = CGImageCreate(width,
                                            height,
                                            8,
                                            4 * 8,
                                            width * 4,
                                            colorSpaceRef,
                                            bitmapInfo,
                                            provider,
                                            NULL,
                                            NO,
                                            renderingIntent);

            UIImage *uiImage = [UIImage imageWithCGImage:imageRef];
            CGColorSpaceRelease(colorSpaceRef);
            CGImageRelease(imageRef);

            return uiImage;
        }

        return NULL;
    }

    return NULL;
}

[Ruben Gonzalez]

Hi all

I have written a simple snippet in C to take a snapshot from running pipeline and test gst_video_convert_sample [1]. IMHO take snapshot should be a must inside GStreamer and a function like gst_video_write_sample_to_file would be a good idea.

Looking inside gst_video_convert_sample and gst_play_sink_convert_sample, coding gst_video_write_sample_to_file should be easy using a filesink. If these feature is useful for the community, let me know it and I could try to do it.

[1] https://github.com/rubenrua/GstreamerCodeSnippets/blob/master/C/takepic.c

2016-08-17 12:47 GMT+02:00 Macjon <[hidden email]>:

I've got a peace of code that works. I removed the tee from my pipeline. And
now I have the following objective-c code:

-(UIImage*) takeSnapshot
{
    GstSample *videobuffer = NULL;
    GstCaps *caps;
    gint width, height;
    GstMapInfo map;

    g_object_get(G_OBJECT(video_sink), "last-sample", &videobuffer, NULL);

    if (videobuffer)
    {
        caps = gst_sample_get_caps(videobuffer);

        if (!caps) {
            return NULL;
        }

        GstStructure *s = gst_caps_get_structure(caps, 0);
        /* we need to get the final caps on the buffer to get the size */
        gboolean res;
        res = gst_structure_get_int (s, "width", &width);
        res |= gst_structure_get_int (s, "height", &height);
        if (!res) {
            return NULL;
        }

        GstBuffer *snapbuffer = gst_sample_get_buffer(videobuffer);
        if (snapbuffer && gst_buffer_map (snapbuffer, &map, GST_MAP_READ))
        {
            CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,
                                                                  map.data,
                                                                  height *
width * 4,
                                                                  NULL);

            CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
            CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
            CGColorRenderingIntent renderingIntent =
kCGRenderingIntentDefault;

            CGImageRef imageRef = CGImageCreate(width,
                                            height,
                                            8,
                                            4 * 8,
                                            width * 4,
                                            colorSpaceRef,
                                            bitmapInfo,
                                            provider,
                                            NULL,
                                            NO,
                                            renderingIntent);

            UIImage *uiImage = [UIImage imageWithCGImage:imageRef];
            CGColorSpaceRelease(colorSpaceRef);
            CGImageRelease(imageRef);

            return uiImage;
        }

        return NULL;
    }

    return NULL;
}



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Take-snapshot-from-running-pipeline-in-gstreamer-with-iOS-tp4678410p4679126.html

Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Ruben Gonzalez]

Hi all again,

Looking inside GES I have found a reference implementation ges_pipeline_save_thumbnail [1]. I will use it to fix some memory leaks in my snippet.

[1] https://github.com/GStreamer/gst-editing-services/blob/e1f6b9fefba6ae735b7045e982015b13660bf812/ges/ges-pipeline.c#L1234

2016-08-18 23:22 GMT+02:00 Ruben Gonzalez <[hidden email]>:

Hi all

I have written a simple snippet in C to take a snapshot from running pipeline and test gst_video_convert_sample [1]. IMHO take snapshot should be a must inside GStreamer and a function like gst_video_write_sample_to_file would be a good idea.

Looking inside gst_video_convert_sample and gst_play_sink_convert_sample, coding gst_video_write_sample_to_file should be easy using a filesink. If these feature is useful for the community, let me know it and I could try to do it.

[1] https://github.com/rubenrua/GstreamerCodeSnippets/blob/master/C/takepic.c

2016-08-17 12:47 GMT+02:00 Macjon <[hidden email]>:

I've got a peace of code that works. I removed the tee from my pipeline. And
now I have the following objective-c code:

-(UIImage*) takeSnapshot
{
    GstSample *videobuffer = NULL;
    GstCaps *caps;
    gint width, height;
    GstMapInfo map;

    g_object_get(G_OBJECT(video_sink), "last-sample", &videobuffer, NULL);

    if (videobuffer)
    {
        caps = gst_sample_get_caps(videobuffer);

        if (!caps) {
            return NULL;
        }

        GstStructure *s = gst_caps_get_structure(caps, 0);
        /* we need to get the final caps on the buffer to get the size */
        gboolean res;
        res = gst_structure_get_int (s, "width", &width);
        res |= gst_structure_get_int (s, "height", &height);
        if (!res) {
            return NULL;
        }

        GstBuffer *snapbuffer = gst_sample_get_buffer(videobuffer);
        if (snapbuffer && gst_buffer_map (snapbuffer, &map, GST_MAP_READ))
        {
            CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,
                                                                  map.data,
                                                                  height *
width * 4,
                                                                  NULL);

            CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
            CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
            CGColorRenderingIntent renderingIntent =
kCGRenderingIntentDefault;

            CGImageRef imageRef = CGImageCreate(width,
                                            height,
                                            8,
                                            4 * 8,
                                            width * 4,
                                            colorSpaceRef,
                                            bitmapInfo,
                                            provider,
                                            NULL,
                                            NO,
                                            renderingIntent);

            UIImage *uiImage = [UIImage imageWithCGImage:imageRef];
            CGColorSpaceRelease(colorSpaceRef);
            CGImageRelease(imageRef);

            return uiImage;
        }

        return NULL;
    }

    return NULL;
}



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Take-snapshot-from-running-pipeline-in-gstreamer-with-iOS-tp4678410p4679126.html

Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel