GStreamer-devel

vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Classic

List

Threaded

10 messages Options

Timtchenko, Michael

vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi all,

i'm currently facing a very strange behaviour within the gstreamer module vaapipostproc in version 1.16.2. I'm using this module for postprocessing of my video stream. Especially for adjustment of brightness, saturation and contrast, by changing the modules properties.

The decoding of the stream with vaapi, as well as the gl-texture-upload by glupload works like a charm. But when the stream is running and I’m trying to change the properties for brightness, saturation or contrast many times (in a calibration manner), I’m running out of memory.

Is it possible that there is a huge memory leak? When I’m steadily calibrating the mentioned properties for about 30 secs, I get an increase of memory about 2 GB. (I know from 14.4.4, that there were some issues related to a mutual exclusive color calibration, which were fixed by a patch.)

Could someone verify this issue?

Below I’ve attached the relevant code sequences:

Construction:

…

auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );

auto upload = gst_element_factory_make( "glupload", "upload" );

GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );

if( !gst_element_link_filtered( postproc, upload, caps ) )

…

Setter for Properties (the lines below are executed very often, while stream is running):

...

gfloat tmp = 0.5

g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );

...

Not sure whether i’m doing wrong, or there is sth broken in vaapipostproc.

Kind regards
Michael

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Victor Jaquez

Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:

> Hi all,
> i'm currently facing a very strange behaviour within the gstreamer module
> vaapipostproc in version 1.16.2. I'm using this module for postprocessing of
> my video stream. Especially for adjustment of brightness, saturation and
> contrast, by changing the modules properties.
> The decoding of the stream with vaapi, as well as the gl-texture-upload by
> glupload works like a charm. But when the stream is running and I'm trying to
> change the properties for brightness, saturation or contrast many times (in a
> calibration manner), I'm running out of memory.
> Is it possible that there is a huge memory leak? When I'm steadily calibrating
> the mentioned properties for about 30 secs, I get an increase of memory about
> 2 GB. (I know from 14.4.4, that there were some issues related to a mutual
> exclusive color calibration, which were fixed by a patch.) Could someone
> verify this issue?
> Below I've attached the relevant code sequences:
> Construction:
> ...
> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
> auto upload = gst_element_factory_make( "glupload", "upload" );
> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
> if( !gst_element_link_filtered( postproc, upload, caps ) )
> ...
> Setter for Properties (the lines below are executed very often, while stream is running):
> ...
> gfloat tmp = 0.5
> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
> ...
> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
> Kind regards
> Michael

Timtchenko, Michael

AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Victor,

thanks for your reply. Unfortunately I do not have a small executable to demonstrate my problem. (Maybe I can provide it later)

Further information:

- I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.

- I use vaapih264dec as a decoder

- I detect the leak by use of "free" and "htop" with my target process (there is a significant increase of memory usage while changing calibration)
- There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.

My full pipeline looks like this:

auto pipeline = gst_pipeline_new( "pipeline" );

auto src = gst_element_factory_make( "udpsrc", "source" );
auto depay = gst_element_factory_make( "rtph264depay", "depay" );
auto dec = gst_element_factory_make( "vaapih264dec", "dec" );
auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
auto upload = gst_element_factory_make( "glupload", "upload" );
auto sink = gst_element_factory_make( "appsink", "sink" );

gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc, upload, sink, NULL );

if( !gst_element_link_many( src, depay, dec, postproc, NULL ) )
{
return -1;
}

GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );

if( !gst_element_link_filtered( postproc, upload, caps ) )
{
return -1;
}

if( !gst_element_link( upload, sink ) )
{
return -1;
}

// Setup Caps, Ports of src

// Inject GlContext

// Register Callbacks etc.

Thanks in advance!

BR
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Víctor Jáquez
Gesendet: Mittwoch, 22. April 2020 08:38
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:

> Hi all,
> i'm currently facing a very strange behaviour within the gstreamer
> module vaapipostproc in version 1.16.2. I'm using this module for
> postprocessing of my video stream. Especially for adjustment of
> brightness, saturation and contrast, by changing the modules properties.
> The decoding of the stream with vaapi, as well as the
> gl-texture-upload by glupload works like a charm. But when the stream
> is running and I'm trying to change the properties for brightness,
> saturation or contrast many times (in a calibration manner), I'm running out of memory.
> Is it possible that there is a huge memory leak? When I'm steadily
> calibrating the mentioned properties for about 30 secs, I get an
> increase of memory about
> 2 GB. (I know from 14.4.4, that there were some issues related to a
> mutual exclusive color calibration, which were fixed by a patch.)
> Could someone verify this issue?
> Below I've attached the relevant code sequences:
> Construction:
> ...
> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
> auto upload = gst_element_factory_make( "glupload", "upload" );
> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
> Setter for Properties (the lines below are executed very often, while stream is running):
> ...
> gfloat tmp = 0.5
> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
> ...
> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
> Kind regards
> Michael

Hi Michael,

Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/27

How do you measure the memory leak?

You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?

cheers

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

junyan.he

Re: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Do you use DMB buffer and EGL?

Can you try to use VASurface and GLX to have a test?

We really have latent issue in EGL's DMA buffer.

On 2020年04月22日 22:18, Timtchenko, Michael wrote:

> Hi Victor,
>
> thanks for your reply. Unfortunately I do not have a small executable to demonstrate my problem. (Maybe I can provide it later)
>
> Further information:
>
> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>
> - I use vaapih264dec as a decoder
>
> - I detect the leak by use of "free" and "htop" with my target process (there is a significant increase of memory usage while changing calibration)
> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>
> My full pipeline looks like this:
>
> auto pipeline = gst_pipeline_new( "pipeline" );
>
> auto src = gst_element_factory_make( "udpsrc", "source" );
> auto depay = gst_element_factory_make( "rtph264depay", "depay" );
> auto dec = gst_element_factory_make( "vaapih264dec", "dec" );
> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
> auto upload = gst_element_factory_make( "glupload", "upload" );
> auto sink = gst_element_factory_make( "appsink", "sink" );
>
> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc, upload, sink, NULL );
>
> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) )
> {
> return -1;
> }
>
> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>
> if( !gst_element_link_filtered( postproc, upload, caps ) )
> {
> return -1;
> }
>
> if( !gst_element_link( upload, sink ) )
> {
> return -1;
> }
>
> // Setup Caps, Ports of src
>
> // Inject GlContext
>
> // Register Callbacks etc.
>
> Thanks in advance!
>
> BR
> Michael
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Víctor Jáquez
> Gesendet: Mittwoch, 22. April 2020 08:38
> An: Discussion of the development of and with GStreamer <[hidden email]>
> Betreff: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak
>
> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>> Hi all,
>> i'm currently facing a very strange behaviour within the gstreamer
>> module vaapipostproc in version 1.16.2. I'm using this module for
>> postprocessing of my video stream. Especially for adjustment of
>> brightness, saturation and contrast, by changing the modules properties.
>> The decoding of the stream with vaapi, as well as the
>> gl-texture-upload by glupload works like a charm. But when the stream
>> is running and I'm trying to change the properties for brightness,
>> saturation or contrast many times (in a calibration manner), I'm running out of memory.
>> Is it possible that there is a huge memory leak? When I'm steadily
>> calibrating the mentioned properties for about 30 secs, I get an
>> increase of memory about
>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>> mutual exclusive color calibration, which were fixed by a patch.)
>> Could someone verify this issue?
>> Below I've attached the relevant code sequences:
>> Construction:
>> ...
>> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>> auto upload = gst_element_factory_make( "glupload", "upload" );
>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>> Setter for Properties (the lines below are executed very often, while stream is running):
>> ...
>> gfloat tmp = 0.5
>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>> ...
>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>> Kind regards
>> Michael
> Hi Michael,
>
> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/27
>
> How do you measure the memory leak?
>
> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>
> cheers
>
> vmjl
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> Bitte beachten / Please note!
>
> *******************************************************************************
>
> AGCO GmbH
> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
> Registergericht Amtsgericht Kempten HRB 10327
> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
> Vorsitzender des Aufsichtsrates: Torsten Dehner
>
> *******************************************************************************
>
> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
> bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
> oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
> Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
> erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
> Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
> Dateien (sofern vorhanden) auf Viren zu überprüfen.
>
> *******************************************************************************
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain
> confidential and/or privileged material. Any views or opinions presented are solely those of the
> author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
> advised that you have received this email in error and that any use, dissemination, forwarding,
> printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
> responsibility for viruses and it is your responsibility to scan and virus check the email and its
> attachment(s) (if any).
>
> *******************************************************************************
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Timtchenko, Michael

AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Junyan,

thx 4 ur reply!

I'm working on an EGL-System without a window manager. Therefore a GLX test is not possible. (It's an embedded device)

Nonetheless, I tried already "video/x-raw(memory:VASurface)" but had no success in transforming the EGLImage to a GL-Texture manually. That’s why I'm using glupload to achieve this goal.
Do you think, that using the VASurface instead, would solve the problem?

I'm quite sure that there is an issue. Without changing the mentioned properties, the whole pipeline works very nice and efficient.

Maybe this is helpful for further analysis:

When trying to run the pipeline, without "video/x-raw(memory:DMABuf)" and streaming in an kmlsink, there's no increase of memory at all (of course in the same scenario with changing property values etc.). In other words, the following pipeline allows property manipulation without memory issues:

udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
! rtpjitterbuffer latency=20
! rtph264depay
! vaapih264dec
! vaapipostproc
! kmssink

So I guess you are right, that there is sth wrong related to the DMABuf code.

Any further advice?

BR
Michael

-----Ursprüngliche Nachricht-----
Von: Junyan He [mailto:[hidden email]]
Gesendet: Mittwoch, 22. April 2020 17:10
An: Discussion of the development of and with GStreamer <[hidden email]>; Timtchenko, Michael <[hidden email]>
Betreff: Re: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Do you use DMB buffer and EGL?

Can you try to use VASurface and GLX to have a test?

We really have latent issue in EGL's DMA buffer.

On 2020年04月22日 22:18, Timtchenko, Michael wrote:

> Hi Victor,
>
> thanks for your reply. Unfortunately I do not have a small executable
> to demonstrate my problem. (Maybe I can provide it later)
>
> Further information:
>
> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>
> - I use vaapih264dec as a decoder
>
> - I detect the leak by use of "free" and "htop" with my target process
> (there is a significant increase of memory usage while changing
> calibration)
> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>
> My full pipeline looks like this:
>
> auto pipeline = gst_pipeline_new( "pipeline" );
>
> auto src = gst_element_factory_make( "udpsrc", "source" ); auto depay
> = gst_element_factory_make( "rtph264depay", "depay" ); auto dec =
> gst_element_factory_make( "vaapih264dec", "dec" ); auto postproc =
> gst_element_factory_make( "vaapipostproc", "postproc" ); auto upload =
> gst_element_factory_make( "glupload", "upload" ); auto sink =
> gst_element_factory_make( "appsink", "sink" );
>
> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc,
> upload, sink, NULL );
>
> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) ) {
> return -1; }
>
> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf),
> format=RGBA" );
>
> if( !gst_element_link_filtered( postproc, upload, caps ) ) { return
> -1; }
>
> if( !gst_element_link( upload, sink ) ) { return -1; }
>
> // Setup Caps, Ports of src
>
> // Inject GlContext
>
> // Register Callbacks etc.
>
> Thanks in advance!
>
> BR
> Michael
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel
> [mailto:[hidden email]] Im Auftrag von
> Víctor Jáquez
> Gesendet: Mittwoch, 22. April 2020 08:38
> An: Discussion of the development of and with GStreamer
> <[hidden email]>
> Betreff: [External] Re: vaapipostproc +
> (brightness|saturation|contrast) + huge memory leak
>
> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>> Hi all,
>> i'm currently facing a very strange behaviour within the gstreamer
>> module vaapipostproc in version 1.16.2. I'm using this module for
>> postprocessing of my video stream. Especially for adjustment of
>> brightness, saturation and contrast, by changing the modules properties.
>> The decoding of the stream with vaapi, as well as the
>> gl-texture-upload by glupload works like a charm. But when the stream
>> is running and I'm trying to change the properties for brightness,
>> saturation or contrast many times (in a calibration manner), I'm running out of memory.
>> Is it possible that there is a huge memory leak? When I'm steadily
>> calibrating the mentioned properties for about 30 secs, I get an
>> increase of memory about
>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>> mutual exclusive color calibration, which were fixed by a patch.)
>> Could someone verify this issue?
>> Below I've attached the relevant code sequences:
>> Construction:
>> ...
>> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>> auto upload = gst_element_factory_make( "glupload", "upload" );
>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>> Setter for Properties (the lines below are executed very often, while stream is running):
>> ...
>> gfloat tmp = 0.5
>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>> ...
>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>> Kind regards
>> Michael
> Hi Michael,
>
> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_reque
> sts/27
>
> How do you measure the memory leak?
>
> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>
> cheers
>
> vmjl
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> Bitte beachten / Please note!
>
> **********************************************************************
> *********
>
> AGCO GmbH
> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf,
> Germany Registergericht Amtsgericht Kempten HRB 10327
> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin,
> Dr. Heribert Reiter, Ekkehart Glaeser Vorsitzender des Aufsichtsrates:
> Torsten Dehner
>
> **********************************************************************
> *********
>
> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet
> ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes
> Material enthalten. Jegliche darin enthaltene Ansicht oder
> Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise
> die Ansicht oder Meinung von AGCO dar. Sind Sie nicht der Empfänger,
> so haben Sie diese E-Mail irrtümlich erhalten und jegliche Verwendung,
> Veröffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser
> E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte Dateien (sofern vorhanden) auf Viren zu überprüfen.
>
> **********************************************************************
> *********
>
> This email is intended solely for the use of the individual to whom it
> is addressed and may contain confidential and/or privileged material.
> Any views or opinions presented are solely those of the author and do
> not necessarily represent those of AGCO. If you are not the intended
> recipient, be advised that you have received this email in error and
> that any use, dissemination, forwarding, printing or copying of this
> email is strictly prohibited. Neither AGCO nor the sender accepts any
> responsibility for viruses and it is your responsibility to scan and
> virus check the email and its
> attachment(s) (if any).
>
> **********************************************************************
> ********* _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

junyan.he

Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

If KMS sink, the vaapipostproc should copy the data back to sys memory,
not using DMA or VA surface.

If that does not leak memory, it seems that the internal logic of
postproc or decoder may not leak memory.

The DMA buffer logic really has problems, and we already do something to
fix it.

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303

but they are not included in 1.16.2

On 2020年04月23日 14:50, Timtchenko, Michael wrote:

> Hi Junyan,
>
> thx 4 ur reply!
>
> I'm working on an EGL-System without a window manager. Therefore a GLX test is not possible. (It's an embedded device)
>
> Nonetheless, I tried already "video/x-raw(memory:VASurface)" but had no success in transforming the EGLImage to a GL-Texture manually. That’s why I'm using glupload to achieve this goal.
> Do you think, that using the VASurface instead, would solve the problem?
>
> I'm quite sure that there is an issue. Without changing the mentioned properties, the whole pipeline works very nice and efficient.
>
> Maybe this is helpful for further analysis:
>
> When trying to run the pipeline, without "video/x-raw(memory:DMABuf)" and streaming in an kmlsink, there's no increase of memory at all (of course in the same scenario with changing property values etc.). In other words, the following pipeline allows property manipulation without memory issues:
>
> udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
> ! rtpjitterbuffer latency=20
> ! rtph264depay
> ! vaapih264dec
> ! vaapipostproc
> ! kmssink
>
> So I guess you are right, that there is sth wrong related to the DMABuf code.
>
> Any further advice?
>
> BR
> Michael
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Junyan He [mailto:[hidden email]]
> Gesendet: Mittwoch, 22. April 2020 17:10
> An: Discussion of the development of and with GStreamer <[hidden email]>; Timtchenko, Michael <[hidden email]>
> Betreff: Re: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak
>
> Do you use DMB buffer and EGL?
>
> Can you try to use VASurface and GLX to have a test?
>
> We really have latent issue in EGL's DMA buffer.
>
>
>
> On 2020年04月22日 22:18, Timtchenko, Michael wrote:
>> Hi Victor,
>>
>> thanks for your reply. Unfortunately I do not have a small executable
>> to demonstrate my problem. (Maybe I can provide it later)
>>
>> Further information:
>>
>> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>>
>> - I use vaapih264dec as a decoder
>>
>> - I detect the leak by use of "free" and "htop" with my target process
>> (there is a significant increase of memory usage while changing
>> calibration)
>> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>>
>> My full pipeline looks like this:
>>
>> auto pipeline = gst_pipeline_new( "pipeline" );
>>
>> auto src = gst_element_factory_make( "udpsrc", "source" ); auto depay
>> = gst_element_factory_make( "rtph264depay", "depay" ); auto dec =
>> gst_element_factory_make( "vaapih264dec", "dec" ); auto postproc =
>> gst_element_factory_make( "vaapipostproc", "postproc" ); auto upload =
>> gst_element_factory_make( "glupload", "upload" ); auto sink =
>> gst_element_factory_make( "appsink", "sink" );
>>
>> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc,
>> upload, sink, NULL );
>>
>> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) ) {
>> return -1; }
>>
>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf),
>> format=RGBA" );
>>
>> if( !gst_element_link_filtered( postproc, upload, caps ) ) { return
>> -1; }
>>
>> if( !gst_element_link( upload, sink ) ) { return -1; }
>>
>> // Setup Caps, Ports of src
>>
>> // Inject GlContext
>>
>> // Register Callbacks etc.
>>
>> Thanks in advance!
>>
>> BR
>> Michael
>>
>> -----Ursprüngliche Nachricht-----
>> Von: gstreamer-devel
>> [mailto:[hidden email]] Im Auftrag von
>> Víctor Jáquez
>> Gesendet: Mittwoch, 22. April 2020 08:38
>> An: Discussion of the development of and with GStreamer
>> <[hidden email]>
>> Betreff: [External] Re: vaapipostproc +
>> (brightness|saturation|contrast) + huge memory leak
>>
>> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>>> Hi all,
>>> i'm currently facing a very strange behaviour within the gstreamer
>>> module vaapipostproc in version 1.16.2. I'm using this module for
>>> postprocessing of my video stream. Especially for adjustment of
>>> brightness, saturation and contrast, by changing the modules properties.
>>> The decoding of the stream with vaapi, as well as the
>>> gl-texture-upload by glupload works like a charm. But when the stream
>>> is running and I'm trying to change the properties for brightness,
>>> saturation or contrast many times (in a calibration manner), I'm running out of memory.
>>> Is it possible that there is a huge memory leak? When I'm steadily
>>> calibrating the mentioned properties for about 30 secs, I get an
>>> increase of memory about
>>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>>> mutual exclusive color calibration, which were fixed by a patch.)
>>> Could someone verify this issue?
>>> Below I've attached the relevant code sequences:
>>> Construction:
>>> ...
>>> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>>> auto upload = gst_element_factory_make( "glupload", "upload" );
>>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>>> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>>> Setter for Properties (the lines below are executed very often, while stream is running):
>>> ...
>>> gfloat tmp = 0.5
>>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>>> ...
>>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>>> Kind regards
>>> Michael
>> Hi Michael,
>>
>> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_reque
>> sts/27
>>
>> How do you measure the memory leak?
>>
>> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>>
>> cheers
>>
>> vmjl
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>> Bitte beachten / Please note!
>>
>> **********************************************************************
>> *********
>>
>> AGCO GmbH
>> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf,
>> Germany Registergericht Amtsgericht Kempten HRB 10327
>> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin,
>> Dr. Heribert Reiter, Ekkehart Glaeser Vorsitzender des Aufsichtsrates:
>> Torsten Dehner
>>
>> **********************************************************************
>> *********
>>
>> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet
>> ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes
>> Material enthalten. Jegliche darin enthaltene Ansicht oder
>> Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise
>> die Ansicht oder Meinung von AGCO dar. Sind Sie nicht der Empfänger,
>> so haben Sie diese E-Mail irrtümlich erhalten und jegliche Verwendung,
>> Veröffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser
>> E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte Dateien (sofern vorhanden) auf Viren zu überprüfen.
>>
>> **********************************************************************
>> *********
>>
>> This email is intended solely for the use of the individual to whom it
>> is addressed and may contain confidential and/or privileged material.
>> Any views or opinions presented are solely those of the author and do
>> not necessarily represent those of AGCO. If you are not the intended
>> recipient, be advised that you have received this email in error and
>> that any use, dissemination, forwarding, printing or copying of this
>> email is strictly prohibited. Neither AGCO nor the sender accepts any
>> responsibility for viruses and it is your responsibility to scan and
>> virus check the email and its
>> attachment(s) (if any).
>>
>> **********************************************************************
>> ********* _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Timtchenko, Michael

AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Junyan,

we'll try to integrate the merge-requests and evaluate the issue again. I'll share the results.

I would like to try the VASurface variant. Do you have some example code, how to transform a eglimage to a gltexture? If so, I could easily provide some feedback. Maybe this would lead to further isolation of the problem source.

BR,
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Junyan He
Gesendet: Donnerstag, 23. April 2020 09:04
An: Timtchenko, Michael <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

If KMS sink, the vaapipostproc should copy the data back to sys memory, not using DMA or VA surface.

If that does not leak memory, it seems that the internal logic of postproc or decoder may not leak memory.

The DMA buffer logic really has problems, and we already do something to
fix it.

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303

but they are not included in 1.16.2

On 2020年04月23日 14:50, Timtchenko, Michael wrote:

Timtchenko, Michael

AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Or maybe an example how to convert a VASurface to a GLTexture. This would be very helpful.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Timtchenko, Michael
Gesendet: Donnerstag, 23. April 2020 10:04
An: Discussion of the development of and with GStreamer <[hidden email]>; Junyan He <[hidden email]>
Betreff: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Junyan,

we'll try to integrate the merge-requests and evaluate the issue again. I'll share the results.

I would like to try the VASurface variant. Do you have some example code, how to transform a eglimage to a gltexture? If so, I could easily provide some feedback. Maybe this would lead to further isolation of the problem source.

BR,
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Junyan He
Gesendet: Donnerstag, 23. April 2020 09:04
An: Timtchenko, Michael <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

If KMS sink, the vaapipostproc should copy the data back to sys memory, not using DMA or VA surface.

If that does not leak memory, it seems that the internal logic of postproc or decoder may not leak memory.

The DMA buffer logic really has problems, and we already do something to fix it.

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303

but they are not included in 1.16.2

On 2020年04月23日 14:50, Timtchenko, Michael wrote:

> Hi Junyan,
>
> thx 4 ur reply!
>
> I'm working on an EGL-System without a window manager. Therefore a GLX
> test is not possible. (It's an embedded device)
>
> Nonetheless, I tried already "video/x-raw(memory:VASurface)" but had no success in transforming the EGLImage to a GL-Texture manually. That’s why I'm using glupload to achieve this goal.
> Do you think, that using the VASurface instead, would solve the problem?
>
> I'm quite sure that there is an issue. Without changing the mentioned properties, the whole pipeline works very nice and efficient.
>
> Maybe this is helpful for further analysis:
>
> When trying to run the pipeline, without "video/x-raw(memory:DMABuf)" and streaming in an kmlsink, there's no increase of memory at all (of course in the same scenario with changing property values etc.). In other words, the following pipeline allows property manipulation without memory issues:
>
> udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
> ! rtpjitterbuffer latency=20
> ! rtph264depay
> ! vaapih264dec
> ! vaapipostproc
> ! kmssink
>
> So I guess you are right, that there is sth wrong related to the DMABuf code.
>
> Any further advice?
>
> BR
> Michael
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Junyan He [mailto:[hidden email]]
> Gesendet: Mittwoch, 22. April 2020 17:10
> An: Discussion of the development of and with GStreamer
> <[hidden email]>; Timtchenko, Michael
> <[hidden email]>
> Betreff: Re: AW: [External] Re: vaapipostproc +
> (brightness|saturation|contrast) + huge memory leak
>
> Do you use DMB buffer and EGL?
>
> Can you try to use VASurface and GLX to have a test?
>
> We really have latent issue in EGL's DMA buffer.
>
>
>
> On 2020年04月22日 22:18, Timtchenko, Michael wrote:
>> Hi Victor,
>>
>> thanks for your reply. Unfortunately I do not have a small executable
>> to demonstrate my problem. (Maybe I can provide it later)
>>
>> Further information:
>>
>> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>>
>> - I use vaapih264dec as a decoder
>>
>> - I detect the leak by use of "free" and "htop" with my target
>> process (there is a significant increase of memory usage while
>> changing
>> calibration)
>> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>>
>> My full pipeline looks like this:
>>
>> auto pipeline = gst_pipeline_new( "pipeline" );
>>
>> auto src = gst_element_factory_make( "udpsrc", "source" ); auto depay
>> = gst_element_factory_make( "rtph264depay", "depay" ); auto dec =
>> gst_element_factory_make( "vaapih264dec", "dec" ); auto postproc =
>> gst_element_factory_make( "vaapipostproc", "postproc" ); auto upload
>> = gst_element_factory_make( "glupload", "upload" ); auto sink =
>> gst_element_factory_make( "appsink", "sink" );
>>
>> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc,
>> upload, sink, NULL );
>>
>> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) ) {
>> return -1; }
>>
>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf),
>> format=RGBA" );
>>
>> if( !gst_element_link_filtered( postproc, upload, caps ) ) { return
>> -1; }
>>
>> if( !gst_element_link( upload, sink ) ) { return -1; }
>>
>> // Setup Caps, Ports of src
>>
>> // Inject GlContext
>>
>> // Register Callbacks etc.
>>
>> Thanks in advance!
>>
>> BR
>> Michael
>>
>> -----Ursprüngliche Nachricht-----
>> Von: gstreamer-devel
>> [mailto:[hidden email]] Im Auftrag von
>> Víctor Jáquez
>> Gesendet: Mittwoch, 22. April 2020 08:38
>> An: Discussion of the development of and with GStreamer
>> <[hidden email]>
>> Betreff: [External] Re: vaapipostproc +
>> (brightness|saturation|contrast) + huge memory leak
>>
>> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>>> Hi all,
>>> i'm currently facing a very strange behaviour within the gstreamer
>>> module vaapipostproc in version 1.16.2. I'm using this module for
>>> postprocessing of my video stream. Especially for adjustment of
>>> brightness, saturation and contrast, by changing the modules properties.
>>> The decoding of the stream with vaapi, as well as the
>>> gl-texture-upload by glupload works like a charm. But when the
>>> stream is running and I'm trying to change the properties for
>>> brightness, saturation or contrast many times (in a calibration manner), I'm running out of memory.
>>> Is it possible that there is a huge memory leak? When I'm steadily
>>> calibrating the mentioned properties for about 30 secs, I get an
>>> increase of memory about
>>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>>> mutual exclusive color calibration, which were fixed by a patch.)
>>> Could someone verify this issue?
>>> Below I've attached the relevant code sequences:
>>> Construction:
>>> ...
>>> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>>> auto upload = gst_element_factory_make( "glupload", "upload" );
>>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>>> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>>> Setter for Properties (the lines below are executed very often, while stream is running):
>>> ...
>>> gfloat tmp = 0.5
>>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>>> ...
>>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>>> Kind regards
>>> Michael
>> Hi Michael,
>>
>> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requ
>> e
>> sts/27
>>
>> How do you measure the memory leak?
>>
>> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>>
>> cheers
>>
>> vmjl
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>> Bitte beachten / Please note!
>>
>> *********************************************************************
>> *
>> *********
>>
>> AGCO GmbH
>> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf,
>> Germany Registergericht Amtsgericht Kempten HRB 10327
>> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin,
>> Dr. Heribert Reiter, Ekkehart Glaeser Vorsitzender des Aufsichtsrates:
>> Torsten Dehner
>>
>> *********************************************************************
>> *
>> *********
>>
>> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet
>> ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes
>> Material enthalten. Jegliche darin enthaltene Ansicht oder
>> Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise
>> die Ansicht oder Meinung von AGCO dar. Sind Sie nicht der Empfänger,
>> so haben Sie diese E-Mail irrtümlich erhalten und jegliche
>> Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
>> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte Dateien (sofern vorhanden) auf Viren zu überprüfen.
>>
>> *********************************************************************
>> *
>> *********
>>
>> This email is intended solely for the use of the individual to whom
>> it is addressed and may contain confidential and/or privileged material.
>> Any views or opinions presented are solely those of the author and do
>> not necessarily represent those of AGCO. If you are not the intended
>> recipient, be advised that you have received this email in error and
>> that any use, dissemination, forwarding, printing or copying of this
>> email is strictly prohibited. Neither AGCO nor the sender accepts any
>> responsibility for viruses and it is your responsibility to scan and
>> virus check the email and its
>> attachment(s) (if any).
>>
>> *********************************************************************
>> *
>> ********* _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

junyan.he

Re: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

In reply to this post by Timtchenko, Michael

Can you run valgrind to get some info?

If not available, maybe you can add some trace in
gst_vaapi_surface_free, to make sure whether there are some un-freed
surface.

On 2020年04月23日 16:03, Timtchenko, Michael wrote:

> Hi Junyan,
>
> we'll try to integrate the merge-requests and evaluate the issue again. I'll share the results.
>
> I would like to try the VASurface variant. Do you have some example code, how to transform a eglimage to a gltexture? If so, I could easily provide some feedback. Maybe this would lead to further isolation of the problem source.
>
> BR,
> Michael
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Junyan He
> Gesendet: Donnerstag, 23. April 2020 09:04
> An: Timtchenko, Michael <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
> Betreff: Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak
>
> If KMS sink, the vaapipostproc should copy the data back to sys memory, not using DMA or VA surface.
>
> If that does not leak memory, it seems that the internal logic of postproc or decoder may not leak memory.
>
>
> The DMA buffer logic really has problems, and we already do something to
> fix it.
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303
>
> but they are not included in 1.16.2
>
>
>
> On 2020年04月23日 14:50, Timtchenko, Michael wrote:
>> Hi Junyan,
>>
>> thx 4 ur reply!
>>
>> I'm working on an EGL-System without a window manager. Therefore a GLX test is not possible. (It's an embedded device)
>>
>> Nonetheless, I tried already "video/x-raw(memory:VASurface)" but had no success in transforming the EGLImage to a GL-Texture manually. That’s why I'm using glupload to achieve this goal.
>> Do you think, that using the VASurface instead, would solve the problem?
>>
>> I'm quite sure that there is an issue. Without changing the mentioned properties, the whole pipeline works very nice and efficient.
>>
>> Maybe this is helpful for further analysis:
>>
>> When trying to run the pipeline, without "video/x-raw(memory:DMABuf)" and streaming in an kmlsink, there's no increase of memory at all (of course in the same scenario with changing property values etc.). In other words, the following pipeline allows property manipulation without memory issues:
>>
>> udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
>> ! rtpjitterbuffer latency=20
>> ! rtph264depay
>> ! vaapih264dec
>> ! vaapipostproc
>> ! kmssink
>>
>> So I guess you are right, that there is sth wrong related to the DMABuf code.
>>
>> Any further advice?
>>
>> BR
>> Michael
>>
>>
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Junyan He [mailto:[hidden email]]
>> Gesendet: Mittwoch, 22. April 2020 17:10
>> An: Discussion of the development of and with GStreamer <[hidden email]>; Timtchenko, Michael <[hidden email]>
>> Betreff: Re: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak
>>
>> Do you use DMB buffer and EGL?
>>
>> Can you try to use VASurface and GLX to have a test?
>>
>> We really have latent issue in EGL's DMA buffer.
>>
>>
>>
>> On 2020年04月22日 22:18, Timtchenko, Michael wrote:
>>> Hi Victor,
>>>
>>> thanks for your reply. Unfortunately I do not have a small executable
>>> to demonstrate my problem. (Maybe I can provide it later)
>>>
>>> Further information:
>>>
>>> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>>>
>>> - I use vaapih264dec as a decoder
>>>
>>> - I detect the leak by use of "free" and "htop" with my target process
>>> (there is a significant increase of memory usage while changing
>>> calibration)
>>> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>>>
>>> My full pipeline looks like this:
>>>
>>> auto pipeline = gst_pipeline_new( "pipeline" );
>>>
>>> auto src = gst_element_factory_make( "udpsrc", "source" ); auto depay
>>> = gst_element_factory_make( "rtph264depay", "depay" ); auto dec =
>>> gst_element_factory_make( "vaapih264dec", "dec" ); auto postproc =
>>> gst_element_factory_make( "vaapipostproc", "postproc" ); auto upload =
>>> gst_element_factory_make( "glupload", "upload" ); auto sink =
>>> gst_element_factory_make( "appsink", "sink" );
>>>
>>> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc,
>>> upload, sink, NULL );
>>>
>>> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) ) {
>>> return -1; }
>>>
>>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf),
>>> format=RGBA" );
>>>
>>> if( !gst_element_link_filtered( postproc, upload, caps ) ) { return
>>> -1; }
>>>
>>> if( !gst_element_link( upload, sink ) ) { return -1; }
>>>
>>> // Setup Caps, Ports of src
>>>
>>> // Inject GlContext
>>>
>>> // Register Callbacks etc.
>>>
>>> Thanks in advance!
>>>
>>> BR
>>> Michael
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: gstreamer-devel
>>> [mailto:[hidden email]] Im Auftrag von
>>> Víctor Jáquez
>>> Gesendet: Mittwoch, 22. April 2020 08:38
>>> An: Discussion of the development of and with GStreamer
>>> <[hidden email]>
>>> Betreff: [External] Re: vaapipostproc +
>>> (brightness|saturation|contrast) + huge memory leak
>>>
>>> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>>>> Hi all,
>>>> i'm currently facing a very strange behaviour within the gstreamer
>>>> module vaapipostproc in version 1.16.2. I'm using this module for
>>>> postprocessing of my video stream. Especially for adjustment of
>>>> brightness, saturation and contrast, by changing the modules properties.
>>>> The decoding of the stream with vaapi, as well as the
>>>> gl-texture-upload by glupload works like a charm. But when the stream
>>>> is running and I'm trying to change the properties for brightness,
>>>> saturation or contrast many times (in a calibration manner), I'm running out of memory.
>>>> Is it possible that there is a huge memory leak? When I'm steadily
>>>> calibrating the mentioned properties for about 30 secs, I get an
>>>> increase of memory about
>>>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>>>> mutual exclusive color calibration, which were fixed by a patch.)
>>>> Could someone verify this issue?
>>>> Below I've attached the relevant code sequences:
>>>> Construction:
>>>> ...
>>>> auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>>>> auto upload = gst_element_factory_make( "glupload", "upload" );
>>>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>>>> if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>>>> Setter for Properties (the lines below are executed very often, while stream is running):
>>>> ...
>>>> gfloat tmp = 0.5
>>>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>>>> ...
>>>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>>>> Kind regards
>>>> Michael
>>> Hi Michael,
>>>
>>> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
>>> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_reque
>>> sts/27
>>>
>>> How do you measure the memory leak?
>>>
>>> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>>>
>>> cheers
>>>
>>> vmjl
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> [hidden email]
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>>>
>>> Bitte beachten / Please note!
>>>
>>> **********************************************************************
>>> *********
>>>
>>> AGCO GmbH
>>> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf,
>>> Germany Registergericht Amtsgericht Kempten HRB 10327
>>> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin,
>>> Dr. Heribert Reiter, Ekkehart Glaeser Vorsitzender des Aufsichtsrates:
>>> Torsten Dehner
>>>
>>> **********************************************************************
>>> *********
>>>
>>> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet
>>> ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes
>>> Material enthalten. Jegliche darin enthaltene Ansicht oder
>>> Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise
>>> die Ansicht oder Meinung von AGCO dar. Sind Sie nicht der Empfänger,
>>> so haben Sie diese E-Mail irrtümlich erhalten und jegliche Verwendung,
>>> Veröffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser
>>> E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte Dateien (sofern vorhanden) auf Viren zu überprüfen.
>>>
>>> **********************************************************************
>>> *********
>>>
>>> This email is intended solely for the use of the individual to whom it
>>> is addressed and may contain confidential and/or privileged material.
>>> Any views or opinions presented are solely those of the author and do
>>> not necessarily represent those of AGCO. If you are not the intended
>>> recipient, be advised that you have received this email in error and
>>> that any use, dissemination, forwarding, printing or copying of this
>>> email is strictly prohibited. Neither AGCO nor the sender accepts any
>>> responsibility for viruses and it is your responsibility to scan and
>>> virus check the email and its
>>> attachment(s) (if any).
>>>
>>> **********************************************************************
>>> ********* _______________________________________________
>>> gstreamer-devel mailing list
>>> [hidden email]
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Timtchenko, Michael

AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

In reply to this post by Timtchenko, Michael

Hi all,

after further tests we decided to open an issue:

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/253

There seems to be a bug in the DMABuf-Handling.

The issue #253 also contains a small application to demonstrate the problem.

Best regards
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Timtchenko, Michael
Gesendet: Donnerstag, 23. April 2020 10:28
An: Discussion of the development of and with GStreamer <[hidden email]>; Junyan He <[hidden email]>
Betreff: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Or maybe an example how to convert a VASurface to a GLTexture. This would be very helpful.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Timtchenko, Michael
Gesendet: Donnerstag, 23. April 2020 10:04
An: Discussion of the development of and with GStreamer <[hidden email]>; Junyan He <[hidden email]>
Betreff: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Junyan,

we'll try to integrate the merge-requests and evaluate the issue again. I'll share the results.

I would like to try the VASurface variant. Do you have some example code, how to transform a eglimage to a gltexture? If so, I could easily provide some feedback. Maybe this would lead to further isolation of the problem source.

BR,
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Junyan He
Gesendet: Donnerstag, 23. April 2020 09:04
An: Timtchenko, Michael <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

If KMS sink, the vaapipostproc should copy the data back to sys memory, not using DMA or VA surface.

If that does not leak memory, it seems that the internal logic of postproc or decoder may not leak memory.

The DMA buffer logic really has problems, and we already do something to fix it.

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303

but they are not included in 1.16.2

On 2020年04月23日 14:50, Timtchenko, Michael wrote:

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel