Video orientation change

[bojan74]

My pipeline is something like:
appsrc->rtpjitterbbuffer->rtpvp8depay->vp8dec->timeoverlay->videoconvert->vp8enc->webmmux->filesink

So I am creating webm video file with time on it from previously saved rtp
packets of vp8 video.
Mostly everything works ok.

But now I noticed a problem in case if user on mobile phone change phone
orientation (rotate phone) during video call - for example from portrait to
landscape. In this case new caps are generated in vp8 (width and height are
switched) and webmmux report error:


From this moment further creation of webm stopped.

Can you give me any advice what it is possible to do in such situation?

I was thinking if it is possible for example at least to detect such
situation and then to rotate video with videoflip to have same caps as on
start. Ok video will maybe look strange but better this than to have
nothing.

Or maybe to stop video in such situation and started new track in webm with
different caps if this is possible?

Thanks in advance!





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[bojan74]

Since it looks like raw text is not visible I am adding again errors:

0:00:21.546105300 20592 0000000003A74B00 INFO               GST_EVENT
gstevent.c:847:gst_event_new_caps: creating caps event video/x-raw,
format=(string)I420, width=(int)1280, height=(int)720,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1,
chroma-site=(string)mpeg2, colorimetry=(string)bt709,
framerate=(fraction)25/1
0:00:21.559990500 20592 0000000003A74B00 INFO               GST_EVENT
gstevent.c:847:gst_event_new_caps: creating caps event video/x-vp8,
profile=(string)0, streamheader=(buffer)<
4f56503830010100050002d00000010000010000001900000001,
4f56503830022024000000456e636f6465642077697468204753747265616d657220767038656e6320312e31372e320000000001
>, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction)25/1, interlace-mode=(string)progressive,
colorimetry=(string)bt709, chroma-site=(string)mpeg2
0:00:21.560080700 20592 0000000003A74B00 WARN             matroskamux
matroska-mux.c:975:gst_matroska_mux_video_pad_setcaps:<webmmux0> error: Caps
changes are not supported by Matroska
0:00:21.560126900 20592 0000000003A74B00 INFO        GST_ERROR_SYSTEM
gstelement.c:2224:gst_element_message_full_with_details:<webmmux0> posting
message: Could not multiplex stream.
0:00:21.560171000 20592 0000000003A74B00 INFO        GST_ERROR_SYSTEM
gstelement.c:2251:gst_element_message_full_with_details:<webmmux0> posted
error message: Could not multiplex stream.



Now I found some solution to create bigger caps from start 1280x1280 so
final video is all time in the same format and first portrait video (on left
side of the bigger canvas) is then correctly transform to landscape view (on
top side of the bigger canvas).

Anyway does anyone know better solution if caps change (from portrait to
landscape) to solve this problem in final webm video?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le jeudi 23 juillet 2020 à 07:22 -0500, bojan74 a écrit :

> Since it looks like raw text is not visible I am adding again errors:
>
> 0:00:21.546105300 20592 0000000003A74B00 INFO               GST_EVENT
> gstevent.c:847:gst_event_new_caps: creating caps event video/x-raw,
> format=(string)I420, width=(int)1280, height=(int)720,
> interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1,
> chroma-site=(string)mpeg2, colorimetry=(string)bt709,
> framerate=(fraction)25/1
> 0:00:21.559990500 20592 0000000003A74B00 INFO               GST_EVENT
> gstevent.c:847:gst_event_new_caps: creating caps event video/x-vp8,
> profile=(string)0, streamheader=(buffer)<
> 4f56503830010100050002d00000010000010000001900000001,
> 4f56503830022024000000456e636f6465642077697468204753747265616d657220767038656e6320312e31372e320000000001
> > , width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
> framerate=(fraction)25/1, interlace-mode=(string)progressive,
> colorimetry=(string)bt709, chroma-site=(string)mpeg2
> 0:00:21.560080700 20592 0000000003A74B00 WARN             matroskamux
> matroska-mux.c:975:gst_matroska_mux_video_pad_setcaps:<webmmux0> error: Caps
> changes are not supported by Matroska
> 0:00:21.560126900 20592 0000000003A74B00 INFO        GST_ERROR_SYSTEM
> gstelement.c:2224:gst_element_message_full_with_details:<webmmux0> posting
> message: Could not multiplex stream.
> 0:00:21.560171000 20592 0000000003A74B00 INFO        GST_ERROR_SYSTEM
> gstelement.c:2251:gst_element_message_full_with_details:<webmmux0> posted
> error message: Could not multiplex stream.
>
>
>
> Now I found some solution to create bigger caps from start 1280x1280 so
> final video is all time in the same format and first portrait video (on left
> side of the bigger canvas) is then correctly transform to landscape view (on
> top side of the bigger canvas).
>
> Anyway does anyone know better solution if caps change (from portrait to
> landscape) to solve this problem in final webm video?

Someone would have to give webm/matroska format a look to verify if
resolution changes are supported mid-stream. If they are, it seems that
the GStreamer muxer would be missing this feature. So far, I have only
used resolution change with MPEG TS container.

>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[bojan74]

As I wrote I solved the webm problem with rotating video (from portrait to
landscape) last time with setting bigger dimensions using capsfilter (width,
height).

But now I get a case where it looks there is some other change in caps than
just dimensions.
If I compare two caps looks like that:
video/x-raw, format=(string)I420, width=(int)360, height=(int)640,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2,
colorimetry=(string)bt709, framerate=(fraction)0/1

video/x-raw, format=(string)I420, width=(int)270, height=(int)480,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,
colorimetry=(string)bt601, framerate=(fraction)0/1

So beside height and width, there is also change in colorimetry and
chroma-site.

That's why I tried to fix also these two parameters inside capsfilter:
video/x-raw, width=1000, height=1000, chroma-site=(string)jpeg,
colorimetry=(string)bt601

But now I get this error:
0:00:10.114854700 12236 000000000383C0C0 WARN                GST_PADS
gstpad.c:4303:gst_pad_peer_query:<vp8dec0:src> *could not send sticky
events*
0:00:10.114991500 12236 000000000383C0C0 INFO              videomixer
videomixer2.c:485:gst_videomixer2_pad_sink_setcaps:<videomixer2-0:sink_0>
Setting caps video/x-raw, format=(string)I420, width=(int)360,
height=(int)640, interlace-mode=(string)progressive,
multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2,
colorimetry=(string)bt709, framerate=(fraction)0/1
0:00:10.117084500 12236 000000000383C0C0 INFO              videomixer
videomixer2.c:485:gst_videomixer2_pad_sink_setcaps:<videomixer2-0:sink_0>
Setting caps video/x-raw, format=(string)I420, width=(int)360,
height=(int)640, interlace-mode=(string)progressive,
multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2,
colorimetry=(string)bt709, framerate=(fraction)0/1
0:00:10.117370400 12236 000000000383C0C0 INFO            videodecoder
gstvideodecoder.c:3336:gst_video_decoder_clip_and_push_buf:<vp8dec0> First
buffer since flush took 0:00:10.005458500 to produce
0:00:10.117435200 12236 000000000383C0C0 INFO              videomixer
videomixer2.c:485:gst_videomixer2_pad_sink_setcaps:<videomixer2-0:sink_0>
Setting caps video/x-raw, format=(string)I420, width=(int)360,
height=(int)640, interlace-mode=(string)progressive,
multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2,
colorimetry=(string)bt709, framerate=(fraction)0/1
0:00:10.117698800 12236 000000000383C0C0 INFO                    task
gsttask.c:312:gst_task_func:<rtpjitterbuffer_video0:src> Task going to
paused
0:00:10.123762900 12236 000000000383C100 WARN                 basesrc
gstbasesrc.c:3070:gst_base_src_loop:<appsrc_video0> error: Internal data
stream error.
0:00:10.123804600 12236 000000000383C100 WARN                 basesrc
gstbasesrc.c:3070:gst_base_src_loop:<appsrc_video0> *error: streaming
stopped, reason not-negotiated (-4)*
0:00:10.123847500 12236 000000000383C100 INFO        GST_ERROR_SYSTEM
gstelement.c:2224:gst_element_message_full_with_details:<appsrc_video0>
posting message: Internal data stream error.
0:00:10.123889400 12236 000000000383C100 INFO        GST_ERROR_SYSTEM
gstelement.c:2251:gst_element_message_full_with_details:<appsrc_video0>
posted error message: Internal data stream error.


My pipeline in this test was:
appsrc->rtpjitterbuffer->rtpvp8depay->vp8dec0->videomixer->capsfilter->videoconverter->vp8enc->webmux



Do I need to set some other caps too to be able to negotiate?

But if I don't add these two parameters chroma-site=(string)jpeg,
colorimetry=(string)bt601 there is no problem with negotiations but I got
error:
matroska-mux.c:975:gst_matroska_mux_video_pad_setcaps:<webmmux0> error: Caps
changes are not supported by Matroska


So how to fix caps for webm but anyway to be negotiated?






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[bojan74]

If it helps to anyone, I solved the negotiation problem with inserting
another videoconvert in front of capsfilter:
appsrc->rtpjitterbuffer->rtpvp8depay->vp8dec0->videomixer->*videoconvert->*capsfilter->videoconvert->vp8enc->webmux




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel