displaying format=(string)GRAY8 on ximagesink

[Erik Rull]

Hi all,

I was struggling with some shm src / sink issues and found out that the
ximagesink sink does not seem to support 8 bit gray images.

What do I have to do to get the video displayed?
I see that the frames are there due to a identity that I check for arriving items...

If I convert everything to BGRx I get the frames. But I don't want to waste that
much bandwidth. Also several videoconverts before it do not help.

The section of the pipeline looks like this:

shmsrc socket-path=/tmp/pad_shm is-live=1 !
video/x-raw,format=(string)GRAY8,framerate=(fraction)8/1,pixel-aspect-ratio=(fraction)1/1,interlace-mode=(string)progressive,width=401,height=404
! videoconvert  ! identity name=identity ! ximagesink max-lateness=500000000

the part before comes from an appsrc which generates proper 8 bit frames:
appsrc name=src stream-type=0 is-live=1 max-bytes=10000000
caps=video/x-raw,format=(string)GRAY8,framerate=8/1,width=401,height=404 !
shmsink render-delay=0 socket-path=/tmp/pad_shm shm-size=10000000
wait-for-connection=0 max-lateness=500000000 buffer-time=0

Any ideas?

Best regards,

Erik
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Josh Doe]

On Wed, Apr 22, 2020 at 1:02 PM Erik Rull <[hidden email]> wrote:

I was struggling with some shm src / sink issues and found out that the
ximagesink sink does not seem to support 8 bit gray images.

What do I have to do to get the video displayed?
I see that the frames are there due to a identity that I check for arriving items...

If I convert everything to BGRx I get the frames. But I don't want to waste that
much bandwidth. Also several videoconverts before it do not help.

The section of the pipeline looks like this:

shmsrc socket-path=/tmp/pad_shm is-live=1 !
video/x-raw,format=(string)GRAY8,framerate=(fraction)8/1,pixel-aspect-ratio=(fraction)1/1,interlace-mode=(string)progressive,width=401,height=404
! videoconvert  ! identity name=identity ! ximagesink max-lateness=500000000

the part before comes from an appsrc which generates proper 8 bit frames:
appsrc name=src stream-type=0 is-live=1 max-bytes=10000000
caps=video/x-raw,format=(string)GRAY8,framerate=8/1,width=401,height=404 !
shmsink render-delay=0 socket-path=/tmp/pad_shm shm-size=10000000
wait-for-connection=0 max-lateness=500000000 buffer-time=0

You don't exactly say what the result is with your current pipeline. Have you tried substituting "avimux ! filesink location=test.mp4" or similar for ximagesink to see what you're getting?  

 

I ran your pipelines but with videotestsrc instead of appsink, and it works fine. Only thing I can see going on is that your appsrc isn't actually producing data as GRAY8, 401x404. Also, are you padding the rows to 404x404? GStreamer assumes widths are multiples of 4-bytes, so you'll need to pad rows with three pixels.

-Josh

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Erik Rull]

Dear Josh,

Josh Doe wrote:
>
> You don't exactly say what the result is with your current pipeline. Have you
> tried substituting "avimux ! filesink location=test.mp4" or similar for
> ximagesink to see what you're getting? 

I haven't tried.

> I ran your pipelines but with videotestsrc instead of appsink, and it works
> fine. Only thing I can see going on is that your appsrc isn't actually producing
> data as GRAY8, 401x404. Also, are you padding the rows to 404x404? GStreamer
> assumes widths are multiples of 4-bytes, so you'll need to pad rows with three
> pixels.

Oh - I was not aware of that. This would explain a lot - the BGRx is
automatically 4 byte-aligned, the 8 bit data not.
Do you have more information on these constraints? How do I have to handle the
padding? Must I pad to 4 byte and also round up the width to be a multiple of 4?
Or is the pad to 4 sufficient and gstreamer leaves out what is not necessary?

> -Josh

Best regards,

Erik

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Josh Doe]

On Thu, Apr 23, 2020 at 5:49 AM Erik Rull <[hidden email]> wrote:

> I ran your pipelines but with videotestsrc instead of appsink, and it works
> fine. Only thing I can see going on is that your appsrc isn't actually producing
> data as GRAY8, 401x404. Also, are you padding the rows to 404x404? GStreamer
> assumes widths are multiples of 4-bytes, so you'll need to pad rows with three
> pixels.

Oh - I was not aware of that. This would explain a lot - the BGRx is
automatically 4 byte-aligned, the 8 bit data not.
Do you have more information on these constraints? How do I have to handle the
padding? Must I pad to 4 byte and also round up the width to be a multiple of 4?
Or is the pad to 4 sufficient and gstreamer leaves out what is not necessary?

Allocate enough memory for the 404x404 padded image, then memcpy each 401 pixel row to the start of each 404 pixel row, leaving the caps as saying 401. In other words you pad each row with 3 bytes, uninitialized data is fine. GStreamer elements and the video library assume there is padding and will account for it. I have to do this all the time in my plugins, see [0] for an example.

[0]: https://github.com/joshdoe/gst-plugins-vision/blob/master/sys/imperxflex/gstframelinksrc.c#L603

-Josh 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Erik Rull]

Josh Doe wrote:

> On Thu, Apr 23, 2020 at 5:49 AM Erik Rull <[hidden email]
> <mailto:[hidden email]>> wrote:
> Allocate enough memory for the 404x404 padded image, then memcpy each 401 pixel
> row to the start of each 404 pixel row, leaving the caps as saying 401. In other
> words you pad each row with 3 bytes, uninitialized data is fine. GStreamer
> elements and the video library assume there is padding and will account for it.
> I have to do this all the time in my plugins, see [0] for an example.
>
> [0]: https://github.com/joshdoe/gst-plugins-vision/blob/master/sys/imperxflex/gstframelinksrc.c#L603
>
> -Josh 
>

That was it, it works!

Thanks a lot!

Best regards,

Erik
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel