How to send every comes source video buffer to the udp sink without of a buffering?

[Denis Shienkov]

Hi all,

I have implemented the own sender application, which uses the appsrc
element. This source receive the image buffer from the some external
"entity" and calls the gst_app_src_push_buffer() function with the
current timestamp (sets this timestamp to the buffer's property).

My resulting pipeline contains the following elements:

m_appsrc ! videoconvert ! x264enc ! rtph264pay ! udpsink

(with default caps and properties of each element)

so, the resulting frames are sends to the udpsink.

But the problem is that the udpsink begin to send the frames only
after the  "push-buffers" was called more than ~48 times.  So, the
receiver receives the first frame with a delay ~48 frames. So, if
the sender will call the gst_app_src_push_buffer() every second,
then the receiver got a first frame only after the ~48 seconds! So,
I need to call the gst_app_src_push_buffer() to fastest as possible,
to minimise the delay, but, it is nonsense, as it eats  a RAM and CPU!

Is it possible to configure the x264enc, or rtph264pay (I don't know
what I need), to that the udpsink will send the *every encoded frame*,
immediatelly after a *source frame* comes to the appsrc? And do not
wait for 48 source buffers for this?

BR,
Denis
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2017-12-25 at 16:38 +0300, Denis Shienkov wrote:

Hi,

> Is it possible to configure the x264enc, or rtph264pay (I don't know
> what I need), to that the udpsink will send the *every encoded
> frame*, immediatelly after a *source frame* comes to the appsrc? And
> do not wait for 48 source buffers for this?

The easiest way to enable this is to set  x264enc tune=zerolatency

Cheers
-Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Denis Shienkov]

Hi Tim,

I already have tried this, but in this case ths delay decreases
up to ~10-11 frames.

Also I have tried many different options:

    ::g_object_set(G_OBJECT(videoenc),
//                   "pass", "qual",
//                   "quantizer", 20,
                   "tune", "zerolatency",
                   "speed-preset", "ultrafast",
                   "byte-stream", true,
                   "key-int-max", 1,
                   "intra-refresh", true,
//                   "bitrate", 500,
                   "rc-lookahead" , 5,
                   nullptr);

but, the resulting minimum delay stays ~10-11 frames.

It is too big delay anyway...

I tries to do workaround for every new present frame,
to stimulate the "real-time" sending, by pushing the
same frame with > 10 times, e.g. as following:

void Foo::presentImage(const QImage &image)
{
    ...

    auto nsecs = m_timestamp.nsecsElapsed();

    for (int i = 0; i < 15; ++i) {
        const auto buffer = gst_buffer_new_and_alloc(m_image.byteCount());

        const auto bytesCopied = ::gst_buffer_fill(
                    buffer, 0, m_image.constBits(), m_image.byteCount());
        Q_UNUSED(bytesCopied)

        GST_BUFFER_PTS(buffer) = nsecs;
        nsecs += 10; // to add a fake stamp

        const auto result = ::gst_app_src_push_buffer(
                    GST_APP_SRC(m_appsrc), buffer);
        if (result != GST_FLOW_OK) {
            qCCritical(msProducerControl) << "Unable to push buffer";
        }
    }
}

But, it is ugly, too. I don't know, how to fix it. :(

25.12.2017 17:03, Tim Müller пишет:

On Mon, 2017-12-25 at 16:38 +0300, Denis Shienkov wrote:

Hi,

Is it possible to configure the x264enc, or rtph264pay (I don't know
what I need), to that the udpsink will send the *every encoded
frame*, immediatelly after a *source frame* comes to the appsrc? And
do not wait for 48 source buffers for this?

The easiest way to enable this is to set  x264enc tune=zerolatency

Cheers
-Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2017-12-25 at 18:18 +0300, Denis Shienkov wrote:

Hi,

> I already have tried this, but in this case ths delay decreases
> up to ~10-11 frames.
>
> Also I have tried many different options:
>
>     ::g_object_set(G_OBJECT(videoenc),
>                    "tune", "zerolatency",
>                    nullptr);

This won't work. It's a flags property so you'll have to pass an int.

Try either:

  g_objecet_set (videoenc, "tune", 0x04, NULL);

or

  gst_util_set_object_arg (G_OBJECT (videoenc), "tune", "zerolatency");

Cheers
-Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Denis Shienkov]

Hi Tim,

many thanks for your answer.

Seems, this

     ::g_object_set(videoenc, "tune", 0x04, nullptr);

works better. The consumer shows a received images
from the producer after ~3 pushed buffers of producer.

BUT, the consumer shows nothing at all, if the consumer
has been started after the producer, even if the producer
continue sends a frames. E.g. when the consumer (by some
reasons) skip a first one-two the udp frames from the
producer. The consumer shows then an images again after
~150-200 pushed buffers of producer.

How to avoid this problem? I need to show on a consumer
a frames, at least of ~3 pushed buffers from the producer
(It will be fine in case the ~1 pushed buffer).

25.12.2017 23:12, Tim Müller пишет:

> On Mon, 2017-12-25 at 18:18 +0300, Denis Shienkov wrote:
>
> Hi,
>
>> I already have tried this, but in this case ths delay decreases
>> up to ~10-11 frames.
>>
>> Also I have tried many different options:
>>
>>      ::g_object_set(G_OBJECT(videoenc),
>>                     "tune", "zerolatency",
>>                     nullptr);
> This won't work. It's a flags property so you'll have to pass an int.
>
> Try either:
>
>    g_objecet_set (videoenc, "tune", 0x04, NULL);
>
> or
>
>    gst_util_set_object_arg (G_OBJECT (videoenc), "tune", "zerolatency");
>
> Cheers
> -Tim
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Arjen Veenhuizen]

You miss the key frame if the first frame got lost somehow. The decoder must
wait for the next key frame before it can start decoding. In static scenes
(and when using default x264 settings) this can take up to 250 frames. You
can tune the key-int-max properly of x264enc to reduce this latency at the
cost of compression efficiency. Also, do not forget to insert SPS/PPS at
every key frame (e.g. set config-interval = -1 on h264parse)



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Denis Shienkov]

Ahh.. OK, many thanks.

28.12.2017 17:32, Arjen Veenhuizen пишет:

You miss the key frame if the first frame got lost somehow. The decoder must
wait for the next key frame before it can start decoding. In static scenes
(and when using default x264 settings) this can take up to 250 frames. You
can tune the key-int-max properly of x264enc to reduce this latency at the
cost of compression efficiency. Also, do not forget to insert SPS/PPS at
every key frame (e.g. set config-interval = -1 on h264parse)



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel