Avoid data copying for appsink and tee

[Stefan Persson]

Hi,

I would like to analyse the video data that is being to sent to
autovideosink for display in the Android app.

My plan is to use an appsink element and tee the output to that one. However
I'd like to avoid any copying the data as the appsink will only read the
content and output statistics.

How can I minimize the performance loss? Are there any caps for appsink or
tee I can set to force pass by reference?

I'd like to avoid having to write a complete plugin for this as the code
will only be used in one small part of the app.

Best Regards




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mar. 30 avr. 2019 08 h 10, Stefan Persson <[hidden email]> a écrit :

Hi,

I would like to analyse the video data that is being to sent to
autovideosink for display in the Android app.

My plan is to use an appsink element and tee the output to that one. However
I'd like to avoid any copying the data as the appsink will only read the
content and output statistics.

How can I minimize the performance loss? Are there any caps for appsink or
tee I can set to force pass by reference?

Appsink passes references by default. So you should try. You need not to hold for too long on the buffers to avoid reducing perf.

If you still see copies (high CPU load), that could be because of the lack of video meta support. You can then implement allocation query to advertise video meta, or replace the tee/appsink with a pad probe. Just ping again if you need more details about this alternative.

I'd like to avoid having to write a complete plugin for this as the code
will only be used in one small part of the app.

Best Regards




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Persson]

Thanks for the help!

>You can then implement allocation query to advertise video meta, or replace
the tee/appsink with a pad probe. Just ping again if you need more details
about this alternative.

I'm investigating the pad probe but got stuck trying to read the raw pixel
data from the video buffer.

My pipeline is the following:
tcpserversrc ! tsdemux ! h264parse ! avdec_h264 ! videoconvert !
autovideosink name=sink

I attach the pad probe with:
GstPad *pad = gst_element_get_static_pad(ctx->sink, "sink");
gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)
gst_probe_callback, ctx, NULL);

The callback is:
static GstPadProbeReturn gst_probe_callback(GstPad *pad, GstPadProbeInfo
*info, GstContext *ctx) {
  GstBuffer *buffer = GST_PAD_PROBE_INFO_BUFFER(info);
  GstMapInfo map;
  gst_buffer_map(buffer, &map, GST_MAP_READ);
  int offset = 0;
  while (offset < map.size && offset < 20000) {
    /* Data is in map.data? */
    /* *(map.data + offset) /*
    /* This is random (and often just 0). */
  }
}

Since the platform is Android the autovideosink surface is an Android
Surface object received from the Java side via JNI/NDK and configured with
gst_video_overlay_set_window_handle. Video playback works fine. It's just
the pixel data that is strange (or undefined?)

What step am I missing or doing wrong?

Best Regards
Stefan



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mer. 8 mai 2019 06 h 56, Stefan Persson <[hidden email]> a écrit :

Thanks for the help!

>You can then implement allocation query to advertise video meta, or replace
the tee/appsink with a pad probe. Just ping again if you need more details
about this alternative.

I'm investigating the pad probe but got stuck trying to read the raw pixel
data from the video buffer.

My pipeline is the following:
tcpserversrc ! tsdemux ! h264parse ! avdec_h264 ! videoconvert !
autovideosink name=sink

I attach the pad probe with:
GstPad *pad = gst_element_get_static_pad(ctx->sink, "sink");
gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)
gst_probe_callback, ctx, NULL);

The callback is:
static GstPadProbeReturn gst_probe_callback(GstPad *pad, GstPadProbeInfo
*info, GstContext *ctx) {
  GstBuffer *buffer = GST_PAD_PROBE_INFO_BUFFER(info);
  GstMapInfo map;
  gst_buffer_map(buffer, &map, GST_MAP_READ);

Check your return value here if you are fan of memory safe code.

For video frames, it's recommended to use GstVideoFrame wrapper to map/unmap. 

  int offset = 0;
  while (offset < map.size && offset < 20000) {
    /* Data is in map.data? */
    /* *(map.data + offset) /*
    /* This is random (and often just 0).

0 could be valid, and a video image can look random.

*/
  }
}

Since the platform is Android the autovideosink surface is an Android
Surface object received from the Java side via JNI/NDK and configured with
gst_video_overlay_set_window_handle. Video playback works fine. It's just
the pixel data that is strange (or undefined?)

What step am I missing or doing wrong?

Best Regards
Stefan



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel