RSPI 4 and Cam streaming
Locked
 
|
HI all,
I have 3 devices. RSPI 4 with Gstreamer 1.16.2. Laptop Windows 7 and Gstremer 1.16.2. Laptop Ubuntu 18.04 Gstremer 1.14.5. from RSPI 4 I stream camera with: gst-launch-1.0 v4l2src device=/dev/video0 ! v4l2convert ! queue ! v4l2h264enc ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay config-interval=-1 name=payloader pt=96 ! udpsink host=IP_ADDRESS port=9000 from Ubuntu do this: gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false And I can see my camera stream just fine but only if I add config-interval=-1 without that its not working. However if I run same line as Ubuntu on Windows: gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false I can not see camera stream (with or without config-interval=-1 set). What could be a problem here? Do I need some additional parameters to configure on sender/receiver side when streaming to Windows? Dejan -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: RSPI 4 and Cam streaming
|
|
Le jeudi 27 août 2020 à 03:28 -0500, DejanCotra a écrit :
> HI all, > > I have 3 devices. > RSPI 4 with Gstreamer 1.16.2. > Laptop Windows 7 and Gstremer 1.16.2. > Laptop Ubuntu 18.04 Gstremer 1.14.5. > > from RSPI 4 I stream camera with: > gst-launch-1.0 v4l2src device=/dev/video0 ! v4l2convert ! queue ! > v4l2h264enc ! video/x-h264,profile=constrained-baseline ! h264parse ! > rtph264pay config-interval=-1 name=payloader pt=96 ! udpsink host=IP_ADDRESS > port=9000 You might want to control few things on the encoder. What the encoder support is HW specific, you can get a list with: v4l2-ctl -d /dev/videoX -l Then setting controls happens like this: v4l2h264enc extra-controls=s,bitrate=1000000 ! ... For documentation of the possible controls: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-codec.html Notably check your HW default for V4L2_CID_MPEG_VIDEO_GOP_SIZE (gop- size). > > from Ubuntu do this: > gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, > media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! > rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false It's recommended to add an rtpjitterbuffer with a configured latency. That will allow smoother playback (if you revert to sync=true of course). > > And I can see my camera stream just fine but only if I add > config-interval=-1 without that its not working. If the Pi4 HW supports it, see V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR control. This is basically the same thing. It's not uncommon to use this mode, it's not the default for backward compatibility reason. > > However if I run same line as Ubuntu on Windows: > gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, > media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! > rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false > > I can not see camera stream (with or without config-interval=-1 set). > > What could be a problem here? Do I need some additional parameters to > configure on sender/receiver side when streaming to Windows? It could be a regression from 1.14 to 1.16, but also could also be the difference between OS (like socket buffer-size, see udpsrc property). It could also be a timing issue if your HW produced open gop (never insert IDR). Try and enable some debugging, it may help. GST_DEBUG=3 gst-launch-1.0 ... > > Dejan > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
hi Nicolas,
Thanks for response I will check out things that you mentioned. I have one more question if you maybe have some hint it would be great. Im using RSPi 4 and Gstremer 1.16.2 to stream mic and camera over webrtc. Other peer is Windows 10 java application with Gstremer 1.16.2 bindings. So when I use pipeline with x264enc : v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480 ! videoconvert ! queue ! x264enc bitrate=600 speed-preset=ultrafast tune=zerolatency key-int-max=15 ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 appsrc name=audiosrc format=3 is-live=1 do-timestamp=1 caps=audio/x-raw,channels=1,rate=16000,format=S16LE,layout=interleaved ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue ! " RTP_CAPS_OPUS "97 In my Windows app I can see video and hear sound just fine. however when I change video part of pipeline to use HW encoder: v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480 ! queue ! v4l2convert ! queue ! v4l2h264enc ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 So basically I change videoconvert to v4l2convert and x264enc to v4l2h264enc. In my Windows app I never receive incoming stream for video, audio works just fine. I even try to play around with extra-controls for v4l2h264enc (extra-controls=s,video_bitrate=600,h264_i_frame_period=15,h264_profile=1,h264_level=10) but nothing seems to help. Dejan -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: RSPI 4 and Cam streaming
|
|
Le vendredi 28 août 2020 à 05:39 -0500, DejanCotra a écrit :
> hi Nicolas, > > Thanks for response I will check out things that you mentioned. > > I have one more question if you maybe have some hint it would be great. > > Im using RSPi 4 and Gstremer 1.16.2 to stream mic and camera over webrtc. > > Other peer is Windows 10 java application with Gstremer 1.16.2 bindings. > > So when I use pipeline with x264enc : > v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, > width=(int)640, height=(int)480 ! videoconvert ! queue ! x264enc bitrate=600 > speed-preset=ultrafast tune=zerolatency key-int-max=15 ! > video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay > config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 > > appsrc name=audiosrc format=3 is-live=1 do-timestamp=1 > caps=audio/x-raw,channels=1,rate=16000,format=S16LE,layout=interleaved ! > audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue ! " > RTP_CAPS_OPUS "97 > > In my Windows app I can see video and hear sound just fine. > > however when I change video part of pipeline to use HW encoder: > v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, > width=(int)640, height=(int)480 ! queue ! v4l2convert ! queue ! v4l2h264enc > ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay > config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 > > So basically I change videoconvert to v4l2convert and x264enc to > v4l2h264enc. In my Windows app I never receive incoming stream for video, > audio works just fine. > I even try to play around with extra-controls for v4l2h264enc > (extra-controls=s,video_bitrate=600,h264_i_frame_period=15,h264_profile=1,h264_level=10) > but nothing seems to help. And no error or warning show up ? Try setting env GST_DEBUG=3, just as a start. Note that this specific combination is a bit early firmware and kernel wise (remember none of these drivers have been submitted upstream yet, and likely for a reason). There is a bit of performance hit in there, since by default there is a copy between v4l2convert and v4l2h264enc (this is using contiguous memory with CPU cache disable, so the copy is very very slow). Assuming the driver supports one or another, you may want: v4l2convert capture-io-mode=dmabuf-import ! queue ! v4l2h264enc ! ... v4l2convert ! queue ! v4l2h264enc output-io-mode=dmabuf-import ! ... One day this will become automatic, but this isn't as simple as it looks like, specially due to various kernel API limitations and driver bugs. p.s. profile and level will be set automatically from the caps negotiation, no need to use extra-controls for that, video_bitrate is in bits per second, so should be 600000. Also, you don't want to set the h264_i_frame_period but the video_gop_size. Quite subtle difference, but documented: https://www.kernel.org/doc/html/v5.6/media/uapi/v4l/ext-ctrls-codec.html That may not solve all your problems of course, but it should in theory avoid combining multiple bugs together. I got a Pi4 here now, perhaps you could point me to the system image you are running, I'll be able to give it a try. > > Dejan > > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Hi Nicolas,
no there is no error as far as I can see with GST_DEBUG=3. Regarding v4l2convert I remove it from pipeline as it turns out its not needed after all so pipeline looks now like this: v4l2src ! video/x-raw,width=(int)640,height=(int)480 ! queue ! v4l2h264enc extra-controls=s,h264_i_frame_period=15,h264_profile=1,h264_level=10 ! video/x-h264 ! h264parse ! rtph264pay config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 I remove also video_bitrate, but yes nice catch I just copied it from x264enc and for it its kbit/sec :) ok I guees I can try to remove profile and level as well. How can I set video_gop_size, then I run v4l2-ctl -d /dev/video11 -L there is no video_gop_size control? What is more interesting when I use pipeline like above I was able to stream my camera to https://webrtc.nirbheek.in/ (I use this website for testing) but my windows Gstremer application still has same problem :( Unfortunately I can not share my application its under NDA. both RPi4 app which is QT application nor Windows application. Plus Im running on RPi4 AGL (Automotive Grade Linux) and I had to apply some tweaks to official AGL image and build additional libraries in order to run my QT application. Thank you very much for support. Br, Dejan -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: RSPI 4 and Cam streaming
|
|
Le vendredi 28 août 2020 à 13:31 -0500, DejanCotra a écrit :
> Hi Nicolas, > > no there is no error as far as I can see with GST_DEBUG=3. > Regarding v4l2convert I remove it from pipeline as it turns out its not > needed after all so pipeline looks now like this: > v4l2src ! video/x-raw,width=(int)640,height=(int)480 ! queue ! v4l2h264enc > extra-controls=s,h264_i_frame_period=15,h264_profile=1,h264_level=10 ! > video/x-h264 ! h264parse ! rtph264pay config-interval=-1 name=payloader ! > queue ! " RTP_CAPS_H264 "96 > > I remove also video_bitrate, but yes nice catch I just copied it from > x264enc and for it its kbit/sec :) > ok I guees I can try to remove profile and level as well. > How can I set video_gop_size, then I run v4l2-ctl -d /dev/video11 -L there > is no video_gop_size control? It's possible the driver author has miss-interpreted the spec, something we would have catched if he had submitted his driver for upstream review. You would have to get onto RPi forum to verify the this is in fact controlling the gop_size (idr distance) rather then the I frame distance, which is a niche but different concept. > > What is more interesting when I use pipeline like above I was able to stream > my camera to https://webrtc.nirbheek.in/ (I use this website for testing) > but my windows Gstremer application still has same problem :( > > Unfortunately I can not share my application its under NDA. both RPi4 app > which is QT application nor Windows application. > Plus Im running on RPi4 AGL (Automotive Grade Linux) and I had to apply some > tweaks to official AGL image and build additional libraries in order to run > my QT application. > > Thank you very much for support. If you have any new information worth sharing (and that you are allowed to of course), just let us know we'll do are best to help. > > Br, > Dejan > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Hi Nicolas,
I finally manage to receive my HW encoded stream (problem was in how I started pipeline nothing to do with RSPI4 or gstreamer). But I run into another problem Im fighting with this one for some time now with no luck. From RSPI4 Im streaming camera via webrtc (QT application with Gstremer 1.16.2) Im using pipeline: *v4l2src ! video/x-raw,width=(int)640,height=(int)480 ! queue ! v4l2h264enc ! queue ! video/x-h264 ! h264parse ! rtph264pay ! queue ! " RTP_CAPS_H264 "96* on receiving end its windows 10 application with Java bindings for Gsrteamer 1.16.2, it handles incoming video stream with: *queue ! videoconvert ! autovideosink*. Stream starts ok. But after 1 to 35 minutes it just freezes. If I use SW encoder and send camera stream like this: *v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480 ! videoconvert ! queue ! x264enc bitrate=600 speed-preset=ultrafast tune=zerolatency key-int-max=15 ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 !* It runs smoothly for 2h no problem (then I switch it off). So only difference is encoder. Do you have any hint? Further more I try to with error logs * ( qputenv("GST_DEBUG", "4") )*. No errors. Aslo I try with * ( qputenv("GST_DEBUG", "v4l2videoenc*:7") )* and while streaming is running there are print outs: *4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> Allocate output buffer 4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> Process output buffer 4l2videoenc gstv4l2videoenc.c:718:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> Handling frame 1938 v4l2videoenc gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> Oldest frame is 1937 0:01:10.613883808 and 1 frames left v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> Allocate output buffer v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> Process output buffer v4l2videoenc gstv4l2videoenc.c:718:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> Handling frame 1939 v4l2videoenc gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> Oldest frame is 1938 0:01:10.649288698 and 1 frames left v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> Allocate output buffer v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> Process output buffer v4l2videoenc gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> Oldest frame is 1939 0:01:10.681470957 and 0 frames left v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> Allocate output buffer v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> Process output buffe* once stream freezes printouts just stop with no error. -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: RSPI 4 and Cam streaming
|
|
Le mardi 22 septembre 2020 à 07:37 -0500, DejanCotra a écrit :
> Hi Nicolas, > > I finally manage to receive my HW encoded stream (problem was in how I > started pipeline nothing to do with RSPI4 or gstreamer). > > But I run into another problem Im fighting with this one for some time now > with no luck. > > From RSPI4 Im streaming camera via webrtc (QT application with Gstremer > 1.16.2) Im using pipeline: > *v4l2src ! video/x-raw,width=(int)640,height=(int)480 ! queue ! v4l2h264enc > ! queue ! video/x-h264 ! h264parse ! rtph264pay ! queue ! " RTP_CAPS_H264 > "96* > > on receiving end its windows 10 application with Java bindings for Gsrteamer > 1.16.2, it handles incoming video stream with: > *queue ! videoconvert ! autovideosink*. > > Stream starts ok. But after 1 to 35 minutes it just freezes. > > If I use SW encoder and send camera stream like this: > *v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2, > width=(int)640, height=(int)480 ! videoconvert ! queue ! x264enc bitrate=600 > speed-preset=ultrafast tune=zerolatency key-int-max=15 ! > video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay > config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96 !* > > It runs smoothly for 2h no problem (then I switch it off). > > So only difference is encoder. Do you have any hint? > > Further more I try to with error logs * ( qputenv("GST_DEBUG", "4") )*. No > errors. > Aslo I try with * ( qputenv("GST_DEBUG", "v4l2videoenc*:7") )* > and while streaming is running there are print outs: > *4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Allocate output buffer > 4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Process output buffer > 4l2videoenc > gstv4l2videoenc.c:718:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> > Handling frame 1938 > v4l2videoenc > gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> > Oldest frame is 1937 0:01:10.613883808 and 1 frames left > v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Allocate output buffer > v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Process output buffer > v4l2videoenc > gstv4l2videoenc.c:718:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> > Handling frame 1939 > v4l2videoenc > gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> > Oldest frame is 1938 0:01:10.649288698 and 1 frames left > v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Allocate output buffer > v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Process output buffer > v4l2videoenc > gstv4l2videoenc.c:620:gst_v4l2_video_enc_get_oldest_frame:<v4l2h264enc0> > Oldest frame is 1939 0:01:10.681470957 and 0 frames left > v4l2videoenc gstv4l2videoenc.c:640:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Allocate output buffer > v4l2videoenc gstv4l2videoenc.c:653:gst_v4l2_video_enc_loop:<v4l2h264enc0> > Process output buffe* > once stream freezes printouts just stop with no error. Perhaps follow up with the RPi dev ? You will likely be asked to run this in a debugger, and create a backtrace at the point it get stuck. Perhaps keep us CCed, we can try and help root cause this. I have been running some pipeline on other SoC for few days, so I suspect there is a driver specific issue here. > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |