RTP FEC + RTX: Using both rtpulpfecenc and rtprtxqueue

[GStreamer-devel mailing list]

Hello,

Thank you for the awesome project.

I'm trying to enable both RTP ULP-based FEC and RTX on a pipeline
which is using rtpbin.

They are both somewhat working independently*, but RTX breaks down
when FEC is enabled. This is most likely due to this comment in the
rtpulpfecenc doc:
> This element rewrites packets' seqnums, which means that when combined with retransmission elements such as GstRtpRtxSend, it *must* be placed upstream of those, otherwise retransmission requests will request incorrect seqnums.

Currently rtpulpfecenc is inside the rtpbin ("request-fec-encoder"
signal) and rtprtxqueue is placed right before rtpbin.send_rtp_sink_0
which upstream of rtpulpfecenc.

Any idea how I can switch this order considering rtpbin is doing the
linking of the rtpulpfecenc?

* rtpulpfecenc is not working that great so far. Dropping 1% of
packets it can only recover <50% of the packets no matter what the FEC
percentage I use (even 100%), is this expected?

Thank you,
Pablo
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Hi,

You must place the rtprtxsend inside rtpbin too, you can do this by
inserting using the "request-aux-sender" signal that was designed for
this.

As for the FEC's success, if you do random drops and you drop multiple
packets in a sequence, it's expected that it won't be that great. For
those cases, we would need to implement 2D FEC and that's not
implemented yet for ULP. Ideally, we'd actually implement flexfec is
which the future of FEC, but that's also in the future.

Olivier

On Tue, 2021-06-08 at 17:42 +0200, Pablo Odorico via gstreamer-devel
wrote:

> Hello,
>
> Thank you for the awesome project.
>
> I'm trying to enable both RTP ULP-based FEC and RTX on a pipeline
> which is using rtpbin.
>
> They are both somewhat working independently*, but RTX breaks down
> when FEC is enabled. This is most likely due to this comment in the
> rtpulpfecenc doc:
> > This element rewrites packets' seqnums, which means that when combined with retransmission elements such as GstRtpRtxSend, it *must* be placed upstream of those, otherwise retransmission requests will request incorrect seqnums.
>
> Currently rtpulpfecenc is inside the rtpbin ("request-fec-encoder"
> signal) and rtprtxqueue is placed right before rtpbin.send_rtp_sink_0
> which upstream of rtpulpfecenc.
>
> Any idea how I can switch this order considering rtpbin is doing the
> linking of the rtpulpfecenc?
>
> * rtpulpfecenc is not working that great so far. Dropping 1% of
> packets it can only recover <50% of the packets no matter what the FEC
> percentage I use (even 100%), is this expected?
>
> Thank you,
> Pablo
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
Olivier Crête
[hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Worth mentioning http://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html,
we can probably use this as the basis for flex fec, I second Olivier's
comment that we want that :)

On 6/8/21 7:04 PM, Olivier Crête via gstreamer-devel wrote:

> Hi,
>
> You must place the rtprtxsend inside rtpbin too, you can do this by
> inserting using the "request-aux-sender" signal that was designed for
> this.
>
> As for the FEC's success, if you do random drops and you drop multiple
> packets in a sequence, it's expected that it won't be that great. For
> those cases, we would need to implement 2D FEC and that's not
> implemented yet for ULP. Ideally, we'd actually implement flexfec is
> which the future of FEC, but that's also in the future.
>
> Olivier
>
> On Tue, 2021-06-08 at 17:42 +0200, Pablo Odorico via gstreamer-devel
> wrote:
>> Hello,
>>
>> Thank you for the awesome project.
>>
>> I'm trying to enable both RTP ULP-based FEC and RTX on a pipeline
>> which is using rtpbin.
>>
>> They are both somewhat working independently*, but RTX breaks down
>> when FEC is enabled. This is most likely due to this comment in the
>> rtpulpfecenc doc:
>>> This element rewrites packets' seqnums, which means that when combined with retransmission elements such as GstRtpRtxSend, it *must* be placed upstream of those, otherwise retransmission requests will request incorrect seqnums.
>> Currently rtpulpfecenc is inside the rtpbin ("request-fec-encoder"
>> signal) and rtprtxqueue is placed right before rtpbin.send_rtp_sink_0
>> which upstream of rtpulpfecenc.
>>
>> Any idea how I can switch this order considering rtpbin is doing the
>> linking of the rtpulpfecenc?
>>
>> * rtpulpfecenc is not working that great so far. Dropping 1% of
>> packets it can only recover <50% of the packets no matter what the FEC
>> percentage I use (even 100%), is this expected?
>>
>> Thank you,
>> Pablo
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Thank you Olivier and Mathieu,

I just tried moving the rtprtxqueues to rtpbin using
request-aux-sender but I'm encountering an issue that rtpbin expects
the pads to be named "src_%u" and "sink_%u" but they are instead named
"src" and "sink".
This may be a beginner question, but is there a way I can rename or
mirror those pads to match the expected name? Maybe there's a way to
put the rtprtxqueue in a bin and pass that to request-aux-sender?

Mathieu, I did read that article when researching this and I'm looking
forward to try rtpst2022-1-fecenc or flex once it makes it to a
release.

Regards,
Pablo

On Tue, Jun 8, 2021 at 8:15 PM Mathieu Duponchelle via gstreamer-devel
<[hidden email]> wrote:

>
> Worth mentioning http://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html,
> we can probably use this as the basis for flex fec, I second Olivier's
> comment that we want that :)
>
> On 6/8/21 7:04 PM, Olivier Crête via gstreamer-devel wrote:
> > Hi,
> >
> > You must place the rtprtxsend inside rtpbin too, you can do this by
> > inserting using the "request-aux-sender" signal that was designed for
> > this.
> >
> > As for the FEC's success, if you do random drops and you drop multiple
> > packets in a sequence, it's expected that it won't be that great. For
> > those cases, we would need to implement 2D FEC and that's not
> > implemented yet for ULP. Ideally, we'd actually implement flexfec is
> > which the future of FEC, but that's also in the future.
> >
> > Olivier
> >
> > On Tue, 2021-06-08 at 17:42 +0200, Pablo Odorico via gstreamer-devel
> > wrote:
> >> Hello,
> >>
> >> Thank you for the awesome project.
> >>
> >> I'm trying to enable both RTP ULP-based FEC and RTX on a pipeline
> >> which is using rtpbin.
> >>
> >> They are both somewhat working independently*, but RTX breaks down
> >> when FEC is enabled. This is most likely due to this comment in the
> >> rtpulpfecenc doc:
> >>> This element rewrites packets' seqnums, which means that when combined with retransmission elements such as GstRtpRtxSend, it *must* be placed upstream of those, otherwise retransmission requests will request incorrect seqnums.
> >> Currently rtpulpfecenc is inside the rtpbin ("request-fec-encoder"
> >> signal) and rtprtxqueue is placed right before rtpbin.send_rtp_sink_0
> >> which upstream of rtpulpfecenc.
> >>
> >> Any idea how I can switch this order considering rtpbin is doing the
> >> linking of the rtpulpfecenc?
> >>
> >> * rtpulpfecenc is not working that great so far. Dropping 1% of
> >> packets it can only recover <50% of the packets no matter what the FEC
> >> percentage I use (even 100%), is this expected?
> >>
> >> Thank you,
> >> Pablo
> >> _______________________________________________
> >> gstreamer-devel mailing list
> >> [hidden email]
> >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Let me answer my own question. I managed to make it work by creating
an RtpBin and using ghost pads to expose the required "src_%u" and
"sink_%u" pad names.

It looks like both RTX and FEC are working together now, but so far in
my tests more packets are dropped when they are both enabled than by
just using RTX.
According to the rtpjitterbuffer "stats" property almost all the lost
packets are late. I'm only using 10% FEC so I don't think this is
caused by the additional traffic. I will do my own research but if you
have any thoughts please let me know.

Thank you,
Pablo

On Wed, Jun 9, 2021 at 9:38 AM Pablo Odorico <[hidden email]> wrote:

>
> Thank you Olivier and Mathieu,
>
> I just tried moving the rtprtxqueues to rtpbin using
> request-aux-sender but I'm encountering an issue that rtpbin expects
> the pads to be named "src_%u" and "sink_%u" but they are instead named
> "src" and "sink".
> This may be a beginner question, but is there a way I can rename or
> mirror those pads to match the expected name? Maybe there's a way to
> put the rtprtxqueue in a bin and pass that to request-aux-sender?
>
> Mathieu, I did read that article when researching this and I'm looking
> forward to try rtpst2022-1-fecenc or flex once it makes it to a
> release.
>
> Regards,
> Pablo
>
> On Tue, Jun 8, 2021 at 8:15 PM Mathieu Duponchelle via gstreamer-devel
> <[hidden email]> wrote:
> >
> > Worth mentioning http://mathieuduponchelle.github.io/2020-10-09-SMPTE-2022-1-2D-Forward-Error-Correction-in-GStreamer.html,
> > we can probably use this as the basis for flex fec, I second Olivier's
> > comment that we want that :)
> >
> > On 6/8/21 7:04 PM, Olivier Crête via gstreamer-devel wrote:
> > > Hi,
> > >
> > > You must place the rtprtxsend inside rtpbin too, you can do this by
> > > inserting using the "request-aux-sender" signal that was designed for
> > > this.
> > >
> > > As for the FEC's success, if you do random drops and you drop multiple
> > > packets in a sequence, it's expected that it won't be that great. For
> > > those cases, we would need to implement 2D FEC and that's not
> > > implemented yet for ULP. Ideally, we'd actually implement flexfec is
> > > which the future of FEC, but that's also in the future.
> > >
> > > Olivier
> > >
> > > On Tue, 2021-06-08 at 17:42 +0200, Pablo Odorico via gstreamer-devel
> > > wrote:
> > >> Hello,
> > >>
> > >> Thank you for the awesome project.
> > >>
> > >> I'm trying to enable both RTP ULP-based FEC and RTX on a pipeline
> > >> which is using rtpbin.
> > >>
> > >> They are both somewhat working independently*, but RTX breaks down
> > >> when FEC is enabled. This is most likely due to this comment in the
> > >> rtpulpfecenc doc:
> > >>> This element rewrites packets' seqnums, which means that when combined with retransmission elements such as GstRtpRtxSend, it *must* be placed upstream of those, otherwise retransmission requests will request incorrect seqnums.
> > >> Currently rtpulpfecenc is inside the rtpbin ("request-fec-encoder"
> > >> signal) and rtprtxqueue is placed right before rtpbin.send_rtp_sink_0
> > >> which upstream of rtpulpfecenc.
> > >>
> > >> Any idea how I can switch this order considering rtpbin is doing the
> > >> linking of the rtpulpfecenc?
> > >>
> > >> * rtpulpfecenc is not working that great so far. Dropping 1% of
> > >> packets it can only recover <50% of the packets no matter what the FEC
> > >> percentage I use (even 100%), is this expected?
> > >>
> > >> Thank you,
> > >> Pablo
> > >> _______________________________________________
> > >> gstreamer-devel mailing list
> > >> [hidden email]
> > >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel