Custom GstMeta data dissapearing when received in a new pipeline

[debruyn]

Good day,

I am developing a system that receives a UDP stream from an ARM board. I need to set my own custom metadata on each buffer that is being sent to the PC (SIte ID and Camera ID). I already wrote a metadata API (implemented as a shared library) for this based on this example : https://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/section-allocation-meta.html . The problem is when I set the metadata on the buffer it sets fine, but when i receive the buffers on the other side the metadata is not there. Can anybody please help with this problem. Is it even possible to implement custom metadata like this?

Here is my code snippets for firstly setting the data and secondly reading it.

Setting the metadata (On ARM):

/* This sets up a probe that will give any traffic information on any incoming buffers on a specific pad */
gulong CamPipeline::setupBufferProbe (GstElement * _gstProbeElement)
{
    GstPad * gstPad;
    gulong gstProbeId;

    gstPad = gst_element_get_static_pad (_gstProbeElement, "src");
    gstProbeId = gst_pad_add_probe (gstPad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback) bufferProbe, this, NULL);

    gst_object_unref (gstPad);

    return gstProbeId;
}

/* The actual stream buffer probe */
GstPadProbeReturn CamPipeline::bufferProbe(GstPad * _gstProbePad, GstPadProbeInfo * _gstProbeInfo, gpointer _gstData)
{
    CamPipeline* oPipeObject = (CamPipeline*) _gstData;
    GstBuffer * gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);

    GstRcMeta* gstRcMeta;
    gchar deviceId[10] = "Site";
    gchar camId[10] = "Cam";
    gstRcMeta = gst_buffer_add_rc_meta(gstBuffer, deviceId, camId);

    return GST_PAD_PROBE_OK;
}

Reading the Metadata( On PC)

/* The actual stream buffer probe */
GstPadProbeReturn CamPipeline::bufferProbe(GstPad * _gstProbePad, GstPadProbeInfo * _gstProbeInfo, gpointer _gstData)
{
        GstBuffer * gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);

    CamPipeline* oPipeObject = (CamPipeline*) _gstData;
    //oPipeObject->debug("Meta received");

    GstRcMeta* gstRcMeta;
    gstRcMeta = gst_buffer_get_rc_meta(gstBuffer);
    if(gstRcMeta != NULL)
    {
        cout << gstRcMeta->deviceID << "   " << gstRcMeta->camID <<endl;
    }
    else
    {
        cout <<" NOPE  " <<endl;
    }
}

[Nicolas Dufresne-4]

Le lundi 27 juin 2016 à 04:56 -0700, debruyn a écrit :

> I am developing a system that receives a UDP stream from an ARM
> board. I
> need to set my own custom metadata on each buffer that is being sent
> to the
> PC (SIte ID and Camera ID). I already wrote a metadata API
> (implemented as a
> shared library) for this based on this example :
> https://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/se
> ction-allocation-meta.html
> . The problem is when I set the metadata on the buffer it sets fine,
> but
> when i receive the buffers on the other side the metadata is not
> there. Can
> anybody please help with this problem. Is it even possible to
> implement
> custom metadata like this?
>
> Here is my code snippets for firstly setting the data and secondly
> reading
> it. (attached is the source file of my Custom metadata)

GstMeta are meant for metadata that are transported inside the
pipeline, if you need metadata over the wire, you need to use features
of your chosen streaming protocol.

As an example, when using RTP, you can add a pad probe after you
payloader, and instead custom data, see
gst_rtp_buffer_set_extension_data() (or the one byte header and two
bytes header variants).

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[debruyn]

Tnx for the reply, i appreciate it will look into it and post my findings

[debruyn]

So I have been looking into things, so what i am attempting to do is to catch the buffers as they are being passed down. This being done with a pad probe. The pipe on the board is simple : rtspsrc -> Queue2 -> udpsink. Have a few questions on some logistics.

1. Am i correct in stating that the rtspsrc contains a rtpbin element that will make the RTPBuffers for me?

2. I am having problems to convert a gstBuffer to a gstRTPBuffer, or just in general to obtain the RTPBufferbeing sent to the udpsink. I just want to test if there are already any extension data set(thus meaning I have access to the buffer and can write to it).Here is my code for that part:

/* In main thread */
setupBufferProbe(gstQueue);

/* This sets up a probe that will give any traffic information on any incoming buffers on a specific pad */
gulong CamPipeline::setupBufferProbe (GstElement * _gstProbeElement)
{
    GstPad * gstPad;
    gulong gstProbeId;

    gstPad = gst_element_get_static_pad (_gstProbeElement, "src");
    gstProbeId = gst_pad_add_probe (gstPad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback) bufferProbe, this, NULL);

    gst_object_unref (gstPad);

    return gstProbeId;
}

/* The actual stream buffer probe */
GstPadProbeReturn CamPipeline::bufferProbe(GstPad * _gstProbePad, GstPadProbeInfo * _gstProbeInfo, gpointer _gstData)
{
    CamPipeline *oPipeObject = (CamPipeline*) _gstData;
    GstBuffer *gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);

    GstRTPBuffer * gstRtpBuffer = ((GstRTPBuffer*) gstBuffer);

    if(gst_rtp_buffer_get_extension(gstRtpBuffer))
    {
        oPipeObject->debug("Extension set");
    }
    else
    {
        oPipeObject->debug("Extension not set");
    }
    return GST_PAD_PROBE_OK;
}

Any assitance would be appreciated

[debruyn]

Nvm saw how to do it in a post of you in 2014 : http://gstreamer-devel.966125.n4.nabble.com/streaming-video-metadata-tt4669761.html#a4670063

[Tim Müller]

On Wed, 2016-06-29 at 01:29 -0700, debruyn wrote:

Hi,

> So I have been looking into things, so what i am attempting to do is
> to catch the buffers as they are being passed down. This being done
> with a pad probe.
> The pipe on the board is simple : rtspsrc -> Queue2 -> udpsink. Have
> a few questions on some logistics. 
>
> 1. Am i correct in stating that the rtspsrc contains a rtpbin element
> that will make the RTPBuffers for me?

I think there may be a misunderstanding about the GstRTPBuffer API.
There are no RTPBuffers flowing through the pipeline. There will be
normal GstBuffers flowing through the pipeline containing RTP packet
data.

GstRTPBuffer is just a convenience API to parse a GstBuffer that
contains an RTP packet into the various constituent bits and bobs
(header, flags, header extension, payload, etc.); or to create an RTP
packet.


> 2. I am having problems to convert a gstBuffer to a gstRTPBuffer, or
> just in general to obtain the RTPBufferbeing sent to the udpsink. I
> just want to test if there are already any extension data set(thus
> meaning I have access to the buffer and can write to it).Here is my
> code for that part:

See above.

> /* The actual stream buffer probe */
> GstPadProbeReturn CamPipeline::bufferProbe(GstPad * _gstProbePad,
> GstPadProbeInfo * _gstProbeInfo, gpointer _gstData)
> {
>     CamPipeline *oPipeObject = (CamPipeline*) _gstData;
>     GstBuffer *gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);
>
>     GstRTPBuffer * gstRtpBuffer = ((GstRTPBuffer*) gstBuffer);

You can't just cast, that's not how it works. Try something like this:

  GstRTPBuffer rtp = GST_RTP_BUFFER_INIT;

  if (gst_rtp_buffer_map (gstBuffer, GST_MAP_READ, &rtp)) {
    ...
    g_print ("extension: %d\n", gst_rtp_buffer_get_extension (&rtp));

    gst_rtp_buffer_unmap (&rtp);
  }

Cheers
 -Tim

--

Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[debruyn]

Thanks alot Tim! I seem to be progressing slowly but surely. Understand what you mean by the rtp buffer being mapped from the normal buffer. Question though I am getting an error when trying to use the following enum:  GST_MAP_READWRITE. GST_MAP_READ or GST_MAP_WRITE seems to working fine. The error message states ->

/usr/include/gstreamer-1.0/gst/gstmemory.h:185: error: invalid conversion from 'int' to 'GstMapFlags' [-fpermissive]
 #define GST_MAP_READWRITE      (GST_MAP_READ | GST_MAP_WRITE)
                                ~~~~~~~~~~~~~~^~~~~~~~~~~~~

Any idea why this is happening?

[debruyn]

This is also spammed at the output : gst_rtp_buffer_map: assertion 'rtp->buffer == NULL' failed.  When running with GST_MAP_READ.

Also GST_RTP_BUFFER_INIT gives the same enum error as GST_MAP_READWRITE..

[Mailing List SVR]

Il 29/06/2016 12:54, debruyn ha scritto:

> Thanks alot Tim! I seem to be progressing slowly but surely. Understand what
> you mean by the rtp buffer being mapped from the normal buffer. Question
> though I am getting an error when trying to use the following enum:
> GST_MAP_READWRITE. GST_MAP_READ or GST_MAP_WRITE seems to working fine. The
> error message states ->
>
> /usr/include/gstreamer-1.0/gst/gstmemory.h:185: error: invalid conversion
> from 'int' to 'GstMapFlags' [-fpermissive]
>   #define GST_MAP_READWRITE      (GST_MAP_READ | GST_MAP_WRITE)
>                                  ~~~~~~~~~~~~~~^~~~~~~~~~~~~
>
> Any idea why this is happening?

You need a cast to GstMapFlags something like this

(GstMapFlags)(GST_MAP_READWRITE)

I think this happen in c++ only,

Nicola

>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Custom-GstMeta-data-dissapearing-when-received-in-a-new-pipeline-tp4678272p4678298.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[debruyn]

Casting the GstMapFlag worked on GST_MAP_READWRITE but the assertion error stil presists. Code snippet for the bufferprobe

/* The actual stream buffer probe */
GstPadProbeReturn CamPipeline::bufferProbe(GstPad * _gstProbePad, GstPadProbeInfo * _gstProbeInfo, gpointer _gstData)
{
    CamPipeline *oPipeObject = (CamPipeline*) _gstData;
    GstBuffer *gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);

    GstRTPBuffer rtpBuffer;
   // cout<<"asdsad"<<endl;
    if(gstBuffer != NULL)
    {
        if(gst_rtp_buffer_map(gstBuffer,(GstMapFlags)GST_MAP_READWRITE,&rtpBuffer))
        {
            //g_print("extension %d\n",gst_rtp_buffer_get_extension(&rtpBuffer));
            gst_rtp_buffer_unmap(&rtpBuffer);
        }
    }
    return GST_PAD_PROBE_OK;
}

[Tim Müller]

On Wed, 2016-06-29 at 04:42 -0700, debruyn wrote:

>     GstRTPBuffer rtpBuffer;

Compare with my code snippet, which had:

  GstRTPBuffer rtp = GST_RTP_BUFFER_INIT;

Cheers
 -Tim

--

Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[debruyn]

No see Tim the problem is I am coding in C++, which is a bit more typesafe than C. Hence the complaint. Got the error when using the GstRTPBuffer rtpBuffer = GST_RTP_BUFFER_INIT. But when "manually initializing it i got rid of the problem.
The solution for the GST_RTP_BUFFER_INIT problem =

GstRTPBuffer rtpBuffer = { NULL,0 ,{ NULL, NULL, NULL, NULL },{ 0, 0, 0, 0 }, {
                                   { NULL, (GstMapFlags)0, NULL, 0, 0,{ 0, },{ 0, } },
                                   { NULL, (GstMapFlags)0, NULL, 0, 0,{ 0, },{ 0, } },
                                   { NULL, (GstMapFlags)0, NULL, 0, 0,{ 0, },{ 0, } },
                                   { NULL, (GstMapFlags)0, NULL, 0, 0,{ 0, },{ 0, } } } };

Tnx for the help on this though, sometimes development can make you pull a few hairs out
regards
debruyn

[Tim Müller]

On Wed, 2016-06-29 at 05:18 -0700, debruyn wrote:

> No see Tim the problem is I am coding in C++, which is a bit more
> typesafe than C. Hence the complaint.

I wasn't talking about the compiler warning but the

  gst_rtp_buffer_map: assertion 'rtp->buffer == NULL' failed

Sorry if that wasn't clear :)

  memset (&rtpBuffer, 0, sizeof(GstRTPBuffer));

should also work for what it's worth.

Cheers
 -Tim


--

Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[debruyn]

Morning Guys, so i got the buffer to be mapped to a RTP buffer successfully and all in C++, Tim you memset also worked so added that. So now I am setting my one_byte_header and it seems to set, but its is spewing out Gstreamer-Critical messages again.

how the buffer probe code looks:

    CamPipeline *oPipeObject = (CamPipeline*) _gstData;
    GstBuffer *gstBuffer = GST_PAD_PROBE_INFO_BUFFER(_gstProbeInfo);

    GstRTPBuffer rtpBuffer;
    memset (&rtpBuffer, 0, sizeof(GstRTPBuffer));

    if(gstBuffer != NULL)
    {
        if(gst_rtp_buffer_map(gstBuffer,(GstMapFlags)GST_MAP_READWRITE,&rtpBuffer))
        {
            guint8 uiID = 1;
            int iSiteID = 5;
            gpointer pData = &iSiteID;

            gst_rtp_buffer_set_extension(&rtpBuffer,TRUE);
            if(gst_rtp_buffer_add_extension_onebyte_header(&rtpBuffer,uiID,pData,sizeof(int)))
            {
                oPipeObject->debug("Extension header set");
            }
            else
            {
                oPipeObject->debug("Extension not set");
            }
        }
    }

    gst_rtp_buffer_unmap(&rtpBuffer);
    return GST_PAD_PROBE_OK;

My error messages while nothing crashes:
** (cdd:12195): CRITICAL **: gst_rtp_buffer_unmap: assertion 'rtp->buffer != NULL' failed

(cdd:12195): GStreamer-CRITICAL **: gst_buffer_remove_memory_range: assertion 'GST_IS_BUFFER (buffer)' failed

** (cdd:12195): CRITICAL **: gst_rtp_buffer_map: assertion 'GST_IS_BUFFER (buffer)' failed

(cdd:12195): GStreamer-CRITICAL **: gst_buffer_insert_memory: assertion 'GST_IS_BUFFER (buffer)' failed
1467271500 DEBUG CamPipeline : Extension header set

** (cdd:12195): CRITICAL **: gst_rtp_buffer_unmap: assertion 'rtp->buffer != NULL' failed

(cdd:12195): GStreamer-CRITICAL **: gst_buffer_remove_memory_range: assertion 'GST_IS_BUFFER (buffer)' failed

** (cdd:12195): CRITICAL **: gst_rtp_buffer_map: assertion 'GST_IS_BUFFER (buffer)' failed

(cdd:12195): GStreamer-CRITICAL **: gst_buffer_insert_memory: assertion 'GST_IS_BUFFER (buffer)' failed
1467271500 DEBUG CamPipeline : Extension header set

Any reason why this is happening?