webrtc example: feeding video data from a pipeline

[Guennadi Liakhovetski]

Hi,

I'm trying to run webrtc examples on a Raspberry Pi by feeding video input
to gstreamer from a pipeline, written to by a raspivid process. I have
first tried to run a simple RTP streaming by first starting a listener on
a PC, then creating a named pipe at /tmp/cam and starting gstreamer per

gst-launch-1.0 -v filesrc location=/tmp/cam ! h264parse ! rtph264pay ! udpsink host=192.168.1.16 port=9001

and then in a different window

raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000 -o /tmp/cam

This workd. Gstreamer launches, waiting for data on the pipe and then once
the camera process starts, streaming kicks off.

Next I tried to achieve the same with sendonly and sendrecv gstreamer
examples. First I tried to only modify the pipeline command in the source
by replacing videotestsrc with filesrc like above, this didn't work. I
also tried integrating starting of the raspivid process from the examples
by adding a fork() and exec() calls to them. I see that both succeed, but
streaming doesn't start. What could be the problem and how can I fix it?

Thanks
Guennadi
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Toshick]

Hello Guennadi,

Did you manage to run sendonly or sendrecv examples with videotestsrc on the RPI target?

First of all, you should ensure that examples work. This confirms that you have all the required plugins at the RPI platform.

After, you should proceed with changing the pipeline.

Best regards,

Anton.

On Sun, Oct 4, 2020 at 10:21 PM Guennadi Liakhovetski <[hidden email]> wrote:

Hi,

I'm trying to run webrtc examples on a Raspberry Pi by feeding video input
to gstreamer from a pipeline, written to by a raspivid process. I have
first tried to run a simple RTP streaming by first starting a listener on
a PC, then creating a named pipe at /tmp/cam and starting gstreamer per

gst-launch-1.0 -v filesrc location=/tmp/cam ! h264parse ! rtph264pay ! udpsink host=192.168.1.16 port=9001

and then in a different window

raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000 -o /tmp/cam

This workd. Gstreamer launches, waiting for data on the pipe and then once
the camera process starts, streaming kicks off.

Next I tried to achieve the same with sendonly and sendrecv gstreamer
examples. First I tried to only modify the pipeline command in the source
by replacing videotestsrc with filesrc like above, this didn't work. I
also tried integrating starting of the raspivid process from the examples
by adding a fork() and exec() calls to them. I see that both succeed, but
streaming doesn't start. What could be the problem and how can I fix it?

Thanks
Guennadi
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Guennadi Liakhovetski]

Hi Anton,

Sorry about a delayed response - was a rather busy week.

On Mon, 5 Oct 2020, Anton Pryima wrote:

> Hello Guennadi,
>
> Did you manage to run sendonly or sendrecv examples with videotestsrc on
> the RPI target?

The sendonly / unidirectional app works with a firefox version, that comes
with ubuntu 20.04 (80.0) but self-installed firefox on a Debian machine
(80.0.1) shows a static image. Similar with the sendrecv. Although there
are error messages in both cases.

I should also note, that the R-Pi isn't happy with that kind of a load -
it reboots after a few seconds of running, say, sendrecv, even if I modify
the example to ignore the incoming video stream.

I attached two files with error messages from respective test programs and
the browser web console.

Thanks
Guennadi

> First of all, you should ensure that examples work. This confirms that you
> have all the required plugins at the RPI platform.
>
> After, you should proceed with changing the pipeline.
>
> Best regards,
> Anton.
>
> On Sun, Oct 4, 2020 at 10:21 PM Guennadi Liakhovetski <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I'm trying to run webrtc examples on a Raspberry Pi by feeding video input
> > to gstreamer from a pipeline, written to by a raspivid process. I have
> > first tried to run a simple RTP streaming by first starting a listener on
> > a PC, then creating a named pipe at /tmp/cam and starting gstreamer per
> >
> > gst-launch-1.0 -v filesrc location=/tmp/cam ! h264parse ! rtph264pay !
> > udpsink host=192.168.1.16 port=9001
> >
> > and then in a different window
> >
> > raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000 -o /tmp/cam
> >
> > This workd. Gstreamer launches, waiting for data on the pipe and then once
> > the camera process starts, streaming kicks off.
> >
> > Next I tried to achieve the same with sendonly and sendrecv gstreamer
> > examples. First I tried to only modify the pipeline command in the source
> > by replacing videotestsrc with filesrc like above, this didn't work. I
> > also tried integrating starting of the raspivid process from the examples
> > by adding a fork() and exec() calls to them. I see that both succeed, but
> > streaming doesn't start. What could be the problem and how can I fix it?
> >
> > Thanks
> > Guennadi
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Toshick]

Hello Guennadi,

From your attachment, it is not clear what are you planning to achieve. Is RPi the 'send only' side, or you wanna get a stream from the browser at it?

The first log said that you have the wrong SDP (empty one). The second one has errors in the decoder (but it is not clear why you need a decoder at the RPi side.).

Can you describe, what result are you expecting?

Best regards,

Anton.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Guennadi Liakhovetski]

Hi Anton,

On Sat, 10 Oct 2020, Anton Pryima wrote:

> Hello Guennadi,
>
> >From your attachment, it is not clear what are you planning to achieve. Is
> RPi the 'send only' side, or you wanna get a stream from the browser at it?
> The first log said that you have the wrong SDP (empty one). The second one
> has errors in the decoder (but it is not clear why you need a decoder at
> the RPi side.).
>
> Can you describe, what result are you expecting?

In your previous email you asked me to verify whether and how well
unmodified webrtc examples run on my RPi to check whether all plugins are
available etc. So, my previous email contained exactly that - results of
running two unmodified (well, the sources are modified, but I ran them in
case of sendrecv in the original mode, in case of sendonly modified for
videotestsrc) webrtc examples. That means, all the plugins are available,
and the result is successful, even if I'm getting errors and the RPi
reboots sporadically...

The end-goal would be a "telepresence" type of a device: the RPi should
send video and audio and should be able to receive and play back audio.

Thanks
Guennadi

> Best regards,
> Anton.
>
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Toshick]

Hi Guennadi,

In such a case, you do not need a video decoder on the RPi side. 

What is your receiving side? If it is a browser, please, ensure that the required H264 profile is supported on the receiving side.

(profile you can check in SDP you've got from RPI). This can cause video freezing or absence at all.

To receive just audio at RPi, you need to comment out 

handle_media_stream (pad, pipe, "videoconvert", "autovideosink"); at line 167 of sendrecv/gst/webrtc-sendrecv.c.

Then, at line 383 of the same file, you need to change:

"videotestsrc is-live=true pattern=ball 

to your:

"raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000

(same change for your audio stream)

Leave everything as is.

Best regards,

Anton.

On Sat, Oct 10, 2020 at 2:26 PM Guennadi Liakhovetski <[hidden email]> wrote:

Hi Anton,

On Sat, 10 Oct 2020, Anton Pryima wrote:

> Hello Guennadi,
>
> >From your attachment, it is not clear what are you planning to achieve. Is
> RPi the 'send only' side, or you wanna get a stream from the browser at it?
> The first log said that you have the wrong SDP (empty one). The second one
> has errors in the decoder (but it is not clear why you need a decoder at
> the RPi side.).
>
> Can you describe, what result are you expecting?

In your previous email you asked me to verify whether and how well
unmodified webrtc examples run on my RPi to check whether all plugins are
available etc. So, my previous email contained exactly that - results of
running two unmodified (well, the sources are modified, but I ran them in
case of sendrecv in the original mode, in case of sendonly modified for
videotestsrc) webrtc examples. That means, all the plugins are available,
and the result is successful, even if I'm getting errors and the RPi
reboots sporadically...

The end-goal would be a "telepresence" type of a device: the RPi should
send video and audio and should be able to receive and play back audio.

Thanks
Guennadi

> Best regards,
> Anton.
>
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Guennadi Liakhovetski]

On Sat, 10 Oct 2020, Anton Pryima wrote:

> Hi Guennadi,
>
> In such a case, you do not need a video decoder on the RPi side.
> What is your receiving side?

I plan to use a browser, yes.

> If it is a browser, please, ensure that the
> required H264 profile is supported on the receiving side.
> (profile you can check in SDP you've got from RPI). This can cause video
> freezing or absence at all.

I see on the RPi:

Received SDP:
...
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96
...
a=mid:video0
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000

Does that mean that H264 is supported? I compared that with a browser,
where the video is frozen, it also contains the same H264 line, but there
are some differences too, e.g. it also has

a=sendrecv

in addition to

a=recvonly

> To receive just audio at RPi, you need to comment out
> handle_media_stream (pad, pipe, "videoconvert", "autovideosink"); at line
> 167 of sendrecv/gst/webrtc-sendrecv.c.

Yes, I've done that already, that's how I got rid of incoming video on the
RPi.

> Then, at line 383 of the same file, you need to change:
> "videotestsrc is-live=true pattern=ball
> to your:
> "raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000
> (same change for your audio stream)

Hm, how is this supposed to work? "videotestsrc" is a gstreamer pipeline
element, "raspivid" is a standalone application. You cannot just put it in
the string for gst_parse_launch()? That's why I tried to get them to
stream data via a named pipe, but that didn't work.

Thanks
Guennadi

> Leave everything as is.
>
> Best regards,
> Anton.
>
> On Sat, Oct 10, 2020 at 2:26 PM Guennadi Liakhovetski <[hidden email]>
> wrote:
>
> > Hi Anton,
> >
> > On Sat, 10 Oct 2020, Anton Pryima wrote:
> >
> > > Hello Guennadi,
> > >
> > > >From your attachment, it is not clear what are you planning to achieve.
> > Is
> > > RPi the 'send only' side, or you wanna get a stream from the browser at
> > it?
> > > The first log said that you have the wrong SDP (empty one). The second
> > one
> > > has errors in the decoder (but it is not clear why you need a decoder at
> > > the RPi side.).
> > >
> > > Can you describe, what result are you expecting?
> >
> > In your previous email you asked me to verify whether and how well
> > unmodified webrtc examples run on my RPi to check whether all plugins are
> > available etc. So, my previous email contained exactly that - results of
> > running two unmodified (well, the sources are modified, but I ran them in
> > case of sendrecv in the original mode, in case of sendonly modified for
> > videotestsrc) webrtc examples. That means, all the plugins are available,
> > and the result is successful, even if I'm getting errors and the RPi
> > reboots sporadically...
> >
> > The end-goal would be a "telepresence" type of a device: the RPi should
> > send video and audio and should be able to receive and play back audio.
> >
> > Thanks
> > Guennadi
> >
> > > Best regards,
> > > Anton.
> > >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Guennadi Liakhovetski]

On Sat, 10 Oct 2020, Guennadi Liakhovetski wrote:

> On Sat, 10 Oct 2020, Anton Pryima wrote:

[snip]

> > Then, at line 383 of the same file, you need to change:
> > "videotestsrc is-live=true pattern=ball
> > to your:
> > "raspivid -t 0 -h 480 -w 640 -fps 25 -hf -b 2000000
> > (same change for your audio stream)
>
> Hm, how is this supposed to work? "videotestsrc" is a gstreamer pipeline
> element, "raspivid" is a standalone application. You cannot just put it in
> the string for gst_parse_launch()? That's why I tried to get them to
> stream data via a named pipe, but that didn't work.

Now I know how that is supposed to work. It turns out there's already a
rpicamsrc plugin in gstreamer! With that one I'm getting a whole lot
closer to the goal!

Thanks
Guennadi
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel