libav error while decoding MB

[Shishigami]

I'm on Windows 10 using Gstreamer 1.16.1.

I'm trying to run the following pipelines:

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264,
stream-format=byte-stream ! rtph264pay ! udpsink host=localhost port=5200

gst-launch-1.0 udpsrc address=localhost port=5200 ! application/x-rtp,
encoding-name=H264, payload=96 ! rtph264depay ! h264parse ! avdec_h264 !
autovideosink

But it seems like the packets are in the wrong order? I get the following
output:

Setting pipeline to PAUSED ...
0:00:00.140465500 20676 00000000051F0360 WARN            d3dvideosink
d3dhelpers.c:1146:d3d_set_window_handle:<autovideosink0-actual-sink-d3dvideo>
Window HWND already set to: 0
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:08.562361200 20676 0000000005182500 WARN                GST_PADS
gstpad.c:4231:gst_pad_peer_query:<avdec_h264-0:src> could not send sticky
events
Redistribute latency...
0:00:08.594871600 20676 0000000005182500 ERROR                  libav :0::
Reference 2 >= 2
0:00:08.597654600 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 3002
0:00:08.628660600 20676 0000000005182500 WARN            videodecoder
gstvideodecoder.c:2762:gst_video_decoder_prepare_finish_frame:<avdec_h264-0>
decreasing timestamp (0:00:08.350789300 < 0:00:08.383611400)
0:00:08.694997400 20676 0000000005182500 ERROR                  libav :0::
Reference 3 >= 3
0:00:08.698399100 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 19 14, bytestream 1444
0:00:08.728272400 20676 0000000005182500 ERROR                  libav :0::
Reference 2 >= 2
0:00:08.731733800 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 19 12, bytestream 3688
0:00:09.028552500 20676 0000000005182500 ERROR                  libav :0::
Reference 3 >= 2
0:00:09.031700600 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 3236
0:00:09.095675900 20676 0000000005182500 ERROR                  libav :0::
gray chroma
0:00:09.099261300 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 2792

How do I fix this? Any help is greatly appreciated!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lun. 4 nov. 2019 10 h 40, Shishigami <[hidden email]> a écrit :

I'm on Windows 10 using Gstreamer 1.16.1.

I'm trying to run the following pipelines:

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264,
stream-format=byte-stream ! rtph264pay ! udpsink host=localhost port=5200

gst-launch-1.0 udpsrc address=localhost port=5200 ! application/x-rtp,
encoding-name=H264, payload=96 ! rtph264depay ! h264parse ! avdec_h264 !
autovideosink

But it seems like the packets are in the wrong order? I get the following
output:

Setting pipeline to PAUSED ...
0:00:00.140465500 20676 00000000051F0360 WARN            d3dvideosink
d3dhelpers.c:1146:d3d_set_window_handle:<autovideosink0-actual-sink-d3dvideo>
Window HWND already set to: 0
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:08.562361200 20676 0000000005182500 WARN                GST_PADS
gstpad.c:4231:gst_pad_peer_query:<avdec_h264-0:src> could not send sticky
events
Redistribute latency...
0:00:08.594871600 20676 0000000005182500 ERROR                  libav :0::
Reference 2 >= 2
0:00:08.597654600 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 3002
0:00:08.628660600 20676 0000000005182500 WARN            videodecoder
gstvideodecoder.c:2762:gst_video_decoder_prepare_finish_frame:<avdec_h264-0>
decreasing timestamp (0:00:08.350789300 < 0:00:08.383611400)
0:00:08.694997400 20676 0000000005182500 ERROR                  libav :0::
Reference 3 >= 3
0:00:08.698399100 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 19 14, bytestream 1444
0:00:08.728272400 20676 0000000005182500 ERROR                  libav :0::
Reference 2 >= 2
0:00:08.731733800 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 19 12, bytestream 3688
0:00:09.028552500 20676 0000000005182500 ERROR                  libav :0::
Reference 3 >= 2
0:00:09.031700600 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 3236
0:00:09.095675900 20676 0000000005182500 ERROR                  libav :0::
gray chroma
0:00:09.099261300 20676 0000000005182500 ERROR                  libav :0::
error while decoding MB 17 13, bytestream 2792

How do I fix this? Any help is greatly appreciated!

Few things to notice, you didn't configure key frame distance on your encoder, is not using live source (see is-live property on test sources) and not using a jitterbuffer buffer. This is not yet RTP streaming, so don't expect very good results.

--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Shishigami]

Thank you for your response!


Nicolas Dufresne-5 wrote
> you didn't configure key frame distance on your encoder

Are you referring to the property key-int-max? What is a good value to
choose for this? I've followed your suggestions and edited the pipelines,
but I still get the same errors.

gst-launch-1.0 videotestsrc is-live=true ! x264enc key-int-max=15 !
video/x-h264, stream-format=byte-stream ! rtph264pay ! udpsink
host=localhost port=5200

gst-launch-1.0 udpsrc address=localhost port=5200 ! application/x-rtp,
encoding-name=H264, payload=96 ! rtpjitterbuffer ! rtph264depay ! h264parse
! avdec_h264 ! autovideosink


Nicolas Dufresne-5 wrote
> This is not yet RTP streaming, so don't expect very good results.

Could you elaborate what you mean? I actually want to use RTSP in the end:

I'm working on an application that manipulates frames from a webcam with
OpenCV and then writes it to UDP sink. Then I read it from UDP and make it
available as RTSP stream using gst-rtsp-server.
I get the exact same errors there, which is why I wanted to simplify it and
get it working on the command line first.

Sender:

cv::VideoCapture capture("ksvideosrc ! videoconvert ! appsink");
cv::VideoWriter writer("appsrc ! videoconvert ! x264enc ! video/x-h264,
stream-format=byte-stream ! rtph264pay ! udpsink host=localhost port=5000",
0, (double)30, cv::Size(640, 480), true);
// ...
writer << frame;

Receiver:

gst_rtsp_media_factory_set_launch(factory, "udpsrc address=localhost
port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay !
avdec_h264 ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96");

Is this the wrong approach?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 04 novembre 2019 à 09:48 -0600, Shishigami a écrit :

> Thank you for your response!
>
>
> Nicolas Dufresne-5 wrote
> > you didn't configure key frame distance on your encoder
>
> Are you referring to the property key-int-max? What is a good value to
> choose for this? I've followed your suggestions and edited the pipelines,
> but I still get the same errors.
>
> gst-launch-1.0 videotestsrc is-live=true ! x264enc key-int-max=15 !
> video/x-h264, stream-format=byte-stream ! rtph264pay ! udpsink
> host=localhost port=5200
>
> gst-launch-1.0 udpsrc address=localhost port=5200 ! application/x-rtp,
> encoding-name=H264, payload=96 ! rtpjitterbuffer ! rtph264depay ! h264parse
> ! avdec_h264 ! autovideosink
>
>
> Nicolas Dufresne-5 wrote
> > This is not yet RTP streaming, so don't expect very good results.
>
> Could you elaborate what you mean? I actually want to use RTSP in the end:
>
> I'm working on an application that manipulates frames from a webcam with
> OpenCV and then writes it to UDP sink. Then I read it from UDP and make it
> available as RTSP stream using gst-rtsp-server.
> I get the exact same errors there, which is why I wanted to simplify it and
> get it working on the command line first.

Sounds like you want a totally different configuration for your
encoder. Let's go simple, and set tune=zerolatency on x264enc, that you
already yield better results. B-Frames don't work so great with RTP.

>
> Sender:
>
> cv::VideoCapture capture("ksvideosrc ! videoconvert ! appsink");
> cv::VideoWriter writer("appsrc ! videoconvert ! x264enc ! video/x-h264,
> stream-format=byte-stream ! rtph264pay ! udpsink host=localhost port=5000",
> 0, (double)30, cv::Size(640, 480), true);
> // ...
> writer << frame;
>
> Receiver:
>
> gst_rtsp_media_factory_set_launch(factory, "udpsrc address=localhost
> port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay !
> avdec_h264 ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96");
>
> Is this the wrong approach?
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Shishigami]

I have added tune=zerolatency to the x264enc and I do get a picture in VLC
now: https://i.imgur.com/pCOBZHr.png

I still get a lot of the same errors though and its not really usable. Is
this really related to RTP though? I have modified the pipeline of the RTSP
server such that it simply uses the webcam right away without reading from
UDP:

"ksvideosrc ! videoconvert ! x264enc tune=zerolatency ! rtph264pay name=pay0
pt=96"

That way I do get a perfect picture and no errors (albeit the latency is
very high). If UDP is an issue, is there any alternative I can use? Thank
you!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Shishigami]

I'm really at my wits end, using the exact same configuration as before I get
different errors now:

Sender

warning: Error opening file
(/build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp:901)
warning: ksvideosrc ! videoconvert ! appsink
(/build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp:902)
0:00:00.146751500  3232 00000240C22F3580 WARN              ksvideosrc
gstksvideosrc.c:460:gst_ks_video_src_open_device:<ksvideosrc0> failed to
create/open KsClock
0:00:00.151659300  3232 00000240C22F35C0 FIXME                default
gstutils.c:3981:gst_pad_create_stream_id_internal:<ksvideosrc0:src> Creating
random stream-id, consider implementing a deterministic way of creating a
stream-id
0:00:00.757294900  3232 00000240C22F39C0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<appsink0> warning: Pipeline
construction is invalid, please add queues.
0:00:00.758404300  3232 00000240C22F39C0 WARN                basesink
gstbasesink.c:1218:gst_base_sink_query_latency:<appsink0> warning: Not
enough buffering available for  the processing deadline of
0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional
data. Shortening processing latency to 0:00:00.000000000.

Receiver

0:00:57.926290900 12564 000002231759C600 ERROR                  libav :0::
left block unavailable for requested intra4x4 mode -1
0:00:57.927470400 12564 000002231759C600 ERROR                  libav :0::
error while decoding MB 0 2, bytestream 62481
0:00:57.948366700 12564 000002231759C600 ERROR                  libav :0::
gray chroma
0:00:57.949353400 12564 000002231759C600 ERROR                  libav :0::
error while decoding MB 15 8, bytestream 16185
x264 [error]: baseline profile doesn't support 4:4:4
0:00:57.957040200 12564 000002231759C600 WARN                 x264enc
gstx264enc.c:1803:gst_x264_enc_init_encoder:<x264enc0> Bad downstream
profile name: baseline

A quick test from the command line shows that it can open my webcam without
issue though:

gst-launch-1.0 -v ksvideosrc ! videoconvert ! fpsdisplaysink

I'm sorry if I'm just being stupid, but I'm really lost



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Shishigami]

I've got it almost working now. The image shows up as intended (also from an
OpenCV Mat instead of the ksvideosrc, which is what I wanted to do
originally) but for some reason it only shows the first frame? It never
changes.

I'm using the following pipelines now, any help is appreciated!

Sender:

appsrc is-live=true format=time ! videoconvert ! video/x-raw, format=I420 !
x264enc tune=zerolatency key-int-max=15 ! video/x-h264,
stream-format=byte-stream ! rtph264pay pt=96 ! udpsink host=localhost
port=5000

Receiver:

udpsrc address=localhost port=5000 ! application/x-rtp, payload=96,
clock-rate=90000 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! x264enc
tune=zerolatency key-int-max=15 ! rtph264pay name=pay0 pt=96



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel