Set partial pixel to be transparent in video
Locked
 
|
This post was updated on .
Hi,
I wrote to customized element (myelement) to set partial pixels of one video frame to be transparent. My machine is little endian. This is myelement implementation, I want to make the upper half of 640x480 (i.e. the upper 640x240 pixels ) invisible by setting the Alpha value to be 0x0 GstMapInfo info; if (!gst_buffer_map(buffer, &info, GST_MAP_READ)) { return GST_FLOW_ERROR; } for (int i = 0; i < 240; i++) { for (int j = 0; j < 640; j++) info.data[(i*640+j)*4] = 0x0; } GST_FLOW_OK; This is my test command: gst_launch -v videotestsrc ! video/x-raw,format=RGBA,width=640, height=480 ! videoconvert ! myelement ! x264enc ! mpegtsmux ! filesink location=test.ts I expect the upper half of the test.ts should be transparent. However, the result is the pixel at (row,col)=(0,4), (0,8) (0,12)...... become a GREEN pixel. Would anyone help to comment which part is wrong? Thanks a lot. |
Re: Set partial pixel to be transparent in video
|
|
Hi Sean. GStreamer uses RGBA in byteorder and not in word-order. See https://en.wikipedia.org/wiki/RGBA_color_space for byte-order/word-order.Something like below ifr (format != RGBA && format != BGRA && format != ARGB && format != ABGRA) goto fail; uint8_t *p = frame_address_start ((start_row*width)+start_col)*4 + (format == RGBA || format == BGRA) ? 3 : 0; stride = (start_col + width - end_col) << 2; for (j=start_row; j <= end_row; j++) { for (i=start_col; i <= end_col; i++) { *p = 0; p += 4; }} On Mon, Sep 26, 2016 at 7:52 AM, Sean <[hidden email]> wrote: Hi, _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Set partial pixel to be transparent in video
|
|
Smaller correction if (format != RGBA && format != BGRA && format != ARGB && format != ABGRA) goto fail; uint8_t *p = frame_address_start + ((start_row*width)+start_col)* stride = (start_col + width - end_col) << 2; for (j=start_row; j <= end_row; j++) { for (i=start_col; i <= end_col; i++) { *p = 0; p += 4; }p += stride; } On Mon, Sep 26, 2016 at 11:10 AM, Peter Maersk-Moller <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Set partial pixel to be transparent in video
|
|
Another note, x264enc does not support any format with an alpha channel. If "myelement" is expecting RGBA, then your pipeline should fail with "no-negotiated". You should review your caps negotiation. Nicolas Le lundi 26 septembre 2016 à 11:12 +0200, Peter Maersk-Moller a écrit :
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
In reply to this post by Peter Maersk-Moller-2
Hi, Peter,
Thanks for pointing out the "byte-ordering" of RGBA in Gstreamer. However, if we don't care the performance and only care the correctness. I think the following implement should work in myelement (i. e. writing the 4th byte of each pixel): for (int i = 0; i < 240; i++) { for (int j = 0; j < 640; j++) info.data[(i*640+j)*4-1] = 0x0; } However, the result is the same (i.e. the pixel at (row,col)=(0,4), (0,8) (0,12)...... become a GREEN pixel). Is it related to the "stride" value? If yes, how would I get stride information? I checked "GstMapInfo" which doesn't have stride information? Thanks. |
|
|
In reply to this post by Nicolas Dufresne-4
Hi, Nicolas,
What Caps should I specify if I want to modify Alpha value in Raw video (AYUV or ARGB or RGBA) before x264enc? Thanks a lot. Sean |
Re: Set partial pixel to be transparent in video
|
|
In reply to this post by Sean
Hi Sean.
See commets inline. On Mon, Sep 26, 2016 at 6:50 PM, Sean <[hidden email]> wrote: Hi, Peter, Not quite. When i and j are 0, you get info.data[-1]. Furthermore I don't know the type of info.data. If it not a byte array, you will get it terribly wrong. So something like this: u_int8_t* p = ((u_int8_t*)&info.data[0])+3; for (int i = 0; i < 240 ; i++) { for (int j = 0 ; j < 640 ; j++) { *p = 0x0; p += 4; } } If that doesn't work, then maybe the format in info.data isn't RGBA byte-order. Then you could try add 0, 1 or 2 until it worked. That would tell you here the alpha is placed.
Setting something to 0 should not produce green. However you have not explained how you get to see the pixel is green. So how do you get to see that. Maybe you should show your working pipeline for testing this. Are you determining that you see green after encoding it and then displaying it using a player? So what is your pipeline and how do you see it is green? You can not reliably test a pixel value by sending it through an encoder. If I slightly modify your pipeline to this moving the videoconvert and specifying I420 for input to x264enc: gst_launch -v videotestsrc ! video/x-raw,format=RGBA,width= Then what would you expect to see? What would you expect a RGB pixel with alpha set to zero to be converted and encoded as? Again you can not test a pixel value reliably by sending it through an encoder. By the way your original example pipeline would convert video from RGBA to YUV 4:4:4 BEFORE entering you element. And YUV 4:4:4 is definitely not RGBA. So your green might very well be a result and indicator of an invalid YUV value. Not all 2^24 possible YUV values are valid. The stride value is the number of bytes between the first pixel in row 0 and the first pixel in row 1. If the format is RGBA, then the stride value is 4 times the width. If is 24 bit RGB, then it is 3 times the width unless it is RGB in 32 bit. However the stride I used in my example was the number of bytes between the last pixel in first row you wanted to apply a zero alpha value to and the first pixel in the next row you wanted to apply the zero alpha setting to. If you wanted to apply zero alpha to all pixels in first and second row up to some number, then stride would be zero. I just provided code for the general example and if you for a specific subset set startcol=1 and endcol=640, then it will also work.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Hi, Peter,
Thanks for the detailed comment. I saw the GREEN pixel from from the encoded test.ts file via a player. I also tried 4 possible offset (0, 1, 2, 3) of the Alpha byte-position. The result is the same (GREEN pixel at at (row,col)=(0,4), (0,8)...). It seems that the Alpha information is dropped during x264enc and in your modified pipeline, the input format of x264enc is I420, the Alpha information is lost as well. So, is there any way keep the Alpha information after encoding? I checked this link (https://fossies.org/linux/gst-plugins-good/tests/check/elements/alpha.c). The RGBA format is the same as you mentioned. But I tried 4 possible Alpha offset, no one works. Thanks a lot. |
Re: Set partial pixel to be transparent in video
|
|
Hi Sean.
See comments inline. On Tue, Sep 27, 2016 at 6:46 AM, Sean <[hidden email]> wrote: Hi, Peter, Well, if you were using the pipeline you original posted, you are not cahnging the alpha value of pixels but rather creating invalid values of YUV before encoding as H.264 which correctly replaces invalid YUV values with a colour of green. It seems that the Alpha information is dropped Yes all alpha information is dropped before video encoding. That is according to specs. Nearly no video codec support preserving the alpha information. The ones that do, you quite likely will not want to use. See https://www.digitalrebellion.com/blog/posts/list_of_video_formats_supporting_alpha_channels and in your modified pipeline, the input format of x264enc is As according to specs. So, is there any way keep the No. Plain and simple. I checked this link As I said, with your original pipeline you just create invalid YUV values, with my modified pipeline you correctly change the alpha value, however when converted to be encoded, alpha values are thrown away. You can pseudo stream using png still images preserving alpha information costing bandwidth and CPU. The questions are a) What are you trying to do? b) Why ?
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
This post was updated on .
Hi, Peter,
Thanks for the clarification. And you're right. The problem comes from that x264enc doesn't recognized the Alpha channel (or my pipeline sent an invalid YUV to x264enc). I solved this problem by adding "compositor" into my pipeline such that "compositor" will take ARGB/AYUV as input, do alpha blending and output YUV (with Alpha information included). The, the output YUV will become a valid input to x264enc and the output of x264enc (test.ts) will have partial transparent pixels. 1. I use offset 0 to modify the Alpha value of ARGB in myelement. 2. Here is the pipeline that works: gst_launch -v compositor name=comp ! videoconvert ! x264enc ! mpegtsmux ! filesink location=test.ts videotestsrc pattern=red ! video/x-raw,format=AYUV,width=640,height=480 ! myelement ! comp.sink_0 |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |