gstreamer / Farstream / reSIProcate interoperability

[Daniel Pocock]

Farstream and Gstreamer both look like interesting partners for reSIProcate

In reSIProcate, we can do

a) high level programming with librecon (Conversation Manager API) or,

b) more fine-grained control over SIP using the libdum API

My initial thought is to focus on (a) and create a very basic binary,
similar to rtpproxy and mediaproxy but based on gstreamer+farstream.

The high-level classes in librecon almost complement the classes in
Farstream:


FsConference — Interface for farstream conference elements
- librecon: Conversation

FsParticipant — A participant in a conference
FsSession — A session in a conference
- librecon: RemoteParticipant, but only supports one session, so it is
like a combination of FsParticipant and FsSession
- librecon: has other types of Participant, e.g. LocalParticipant for
webcam/screen/mic/speakers

FsStream — A stream in a session in a conference
- librecon: MediaStream
- it looks like FsStream represents both the RTP and RTCP, same as
librecon MediaStream


To make this work in reSIProcate, it looks like we need to split the
Participant class into Participant and Session so that we have the same
class hierarchy as Farstream, support for video, etc.

librecon also has
- MediaResourceParticipant (for playing a file into the Conversation) and
- LocalParticipant (to represent hardware like the webcam, screen, mic
and speakers)

Sometimes the RemoteParticipant is regular SIP and sometimes it is
WebRTC (SIP over WebSocket).  The reSIProcate stack supports both of
these very well at the SIP level.  As Gstreamer webrtcbin is not part of
Farstream, how does it relate to FsSession and FsStream?  Or can I just
manage webrtcbin elements on my own and link the pads together from my
own code?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Olivier Crête-3]

Hi,

I think your model is the right one.

The local devices and file playback are intentionally  out of scope for Farstream.

The fsrtpconference element is largely a wrapper around rtpbin just like webrtcbin, the main functional difference is that Farstream also handles the encoding and decoding, while webrtcbin doesn't. You can definitely mix both in the same pipeline if we want. You may also want to have separate pipelines and use inter{audio,video}{sink,src} to transfer the frames between the pipelines if you want to decouple them a little.

Olivier

On April 13, 2021 11:26:57 a.m. EDT, Daniel Pocock <[hidden email]> wrote:

>
>Farstream and Gstreamer both look like interesting partners for
>reSIProcate
>
>In reSIProcate, we can do
>
>a) high level programming with librecon (Conversation Manager API) or,
>
>b) more fine-grained control over SIP using the libdum API
>
>My initial thought is to focus on (a) and create a very basic binary,
>similar to rtpproxy and mediaproxy but based on gstreamer+farstream.
>
>The high-level classes in librecon almost complement the classes in
>Farstream:
>
>
>FsConference — Interface for farstream conference elements
>- librecon: Conversation
>
>FsParticipant — A participant in a conference
>FsSession — A session in a conference
>- librecon: RemoteParticipant, but only supports one session, so it is
>like a combination of FsParticipant and FsSession
>- librecon: has other types of Participant, e.g. LocalParticipant for
>webcam/screen/mic/speakers
>
>FsStream — A stream in a session in a conference
>- librecon: MediaStream
>- it looks like FsStream represents both the RTP and RTCP, same as
>librecon MediaStream
>
>
>To make this work in reSIProcate, it looks like we need to split the
>Participant class into Participant and Session so that we have the same
>class hierarchy as Farstream, support for video, etc.
>
>librecon also has
>- MediaResourceParticipant (for playing a file into the Conversation)
>and
>- LocalParticipant (to represent hardware like the webcam, screen, mic
>and speakers)
>
>Sometimes the RemoteParticipant is regular SIP and sometimes it is
>WebRTC (SIP over WebSocket).  The reSIProcate stack supports both of
>these very well at the SIP level.  As Gstreamer webrtcbin is not part
>of
>Farstream, how does it relate to FsSession and FsStream?  Or can I just
>manage webrtcbin elements on my own and link the pads together from my
>own code?
>
>_______________________________________________
>gstreamer-devel mailing list
>[hidden email]
>https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Olivier Crête
[hidden email]
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel