memory leak when reconnection to camera using gstreamer on TX1 platform

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

memory leak when reconnection to camera using gstreamer on TX1 platform

oona

Hi all,

 

This is Oona, a junior software engineer. I send this email to ask your help about my gstreamer application:

Recently, I met a memory leak problem on TX1 platform, when my gstreamer application tried to reconnect to the camera by rtsp. 

Moreover, the tegra I’m using is:

# R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t210ref, EABI: aarch64, DATE: Fri Mar  2 04:58:16 UTC 2018

Gstreamer is:

gst-inspect-1.0 version 1.8.1

GStreamer 1.8.3

https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

 

Problem Details:

1.      The application will call gst_init() at the very first, then builds pipleline as “rtspsrc location=rtsp://${usrname}:${pwd}@${ip}:${port}/cam/realmonitor?channel=1&subtype=0 protocols=tcp ! rtph264depay ! h264parse ! omxh264dec ! video/x-raw, format=NV12 ! appsink sync=false async=false name=appsink1”. As expected, it gets data from camera. By `top` commands, I can see its occupied-memory is ~108M;

disable ${ip}

2.      Of course, the application fails to get data. And the application is trying to reconnect the pipeline periodically. It is still occupied ~108M;

enable ${ip}

3.      The application succeeds to get video again. However, the memory rises to ~180M!

After trying 2,3 for times, I finds that: the memory will increase by -80M each time when it succeed in reconnection.

 

The following code segment is the reconnection (first close() then open()). I think I have unref all the resources in close(), in addition, I unref gst-sample, gst-message in time.

```

int GSTCameraMini::open() {
  this->m_camera_state = CAM_INIT;
  this -> build_pipeline();
  gst_element_set_state(this -> pipeline, GST_STATE_PLAYING);
  return 0;
}

int GSTCameraMini::close() {
  this->m_camera_state = CAM_OFFLINE;
  gst_element_set_state(this -> pipeline, GST_STATE_NULL);
  // unref appsink
  gst_object_unref(G_OBJECT(this -> appsink));
  // unref bus
  gst_object_unref(G_OBJECT(this -> bus));
  // unref pipeline
  gst_object_unref(G_OBJECT(this -> pipeline));
  return 0;
}

```

I have no idea where goes wrong, and this problem makes me headache for a long time. It is a bug from gstreamer or other?

Sincerely looking forward to your reply.

 

Best & Regards,

Oona

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: memory leak when reconnection to camera using gstreamer on TX1 platform

Nicolas Dufresne-5
Le jeudi 29 novembre 2018 à 03:59 +0000, oonamao(毛江云) a écrit :

> Hi all,
>  
> This is Oona, a junior software engineer. I send this email to ask your help about my gstreamer application:
> Recently, I met a memory leak problem on TX1 platform, when my gstreamer application tried to reconnect to the camera by rtsp.
> Moreover, the tegra I’m using is:
> # R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t210ref, EABI: aarch64, DATE: Fri Mar  2 04:58:16 UTC 2018
> Gstreamer is:
> gst-inspect-1.0 version 1.8.1
> GStreamer 1.8.3
> https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
This is relatively old, if it's an upstream bug, it is likely fixed.

>  
> Problem Details:
> 1.      The application will call gst_init() at the very first, then builds pipleline as “rtspsrc location=rtsp://${usrname}:${pwd}@${ip}:${port}/cam/realmonitor?channel=1&subtype=0 protocols=tcp ! rtph264depay ! h264parse ! omxh264dec ! video/x-raw, format=NV12 ! appsink sync=false async=false name=appsink1”. As expected, it gets data from camera. By `top` commands, I can see its occupied-memory is ~108M;
> (disable ${ip})

Though, most of these report are related to miss-used of appsink.
appsink is non leak and have inifit queues by default. Make sure you
limite that queue and pop and unref buffers properly.

> 2.      Of course, the application fails to get data. And the application is trying to reconnect the pipeline periodically. It is still occupied ~108M;
> (enable ${ip})
> 3.      The application succeeds to get video again. However, the memory rises to ~180M!
> After trying 2,3 for times, I finds that: the memory will increase by -80M each time when it succeed in reconnection.
>  
> The following code segment is the reconnection (first close() then open()). I think I have unref all the resources in close(), in addition, I unref gst-sample, gst-message in time.
> ```
> int GSTCameraMini::open() {
>   this->m_camera_state = CAM_INIT;
>   this -> build_pipeline();
>   gst_element_set_state(this -> pipeline, GST_STATE_PLAYING);
>   return 0;
> }
>
> int GSTCameraMini::close() {
>   this->m_camera_state = CAM_OFFLINE;
>   gst_element_set_state(this -> pipeline, GST_STATE_NULL);
>   // unref appsink
>   gst_object_unref(G_OBJECT(this -> appsink));
>   // unref bus
>   gst_object_unref(G_OBJECT(this -> bus));
>   // unref pipeline
>   gst_object_unref(G_OBJECT(this -> pipeline));
>   return 0;
> }
> ```
> I have no idea where goes wrong, and this problem makes me headache for a long time. It is a bug from gstreamer or other?
> Sincerely looking forward to your reply.
>  
> Best & Regards,
> Oona
>  
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: memory leak when reconnection to camera using gstreamer on TX1 platform

oona
Hi,

Thanks for your reply.

I refactor my application, limiting the appsink as:
    gst_app_sink_set_emit_signals((GstAppSink*)g_appsink, true);
    gst_app_sink_set_drop((GstAppSink*)g_appsink, true);
    gst_app_sink_set_max_buffers((GstAppSink*)g_appsink, 1);

unrefing gstSample in time like function NewFrame() in:  github
<https://github.com/rticommunity/rticonnextdds-videodemo-android/blob/master/jni/ConnextGstreamer.cxx>
.

Now, the memory does not increase for each reconnection. However, for the
first time, the memory will rise from 80M to 120M, later it is floating
around 120M.

Moreover, I use valgrind to test the memory leakage, the report is:
==813== LEAK SUMMARY:
==813==    definitely lost: 20,854 bytes in 49 blocks
==813==    indirectly lost: 5,281 bytes in 80 blocks
==813==      possibly lost: 8,572 bytes in 73 blocks
==813==    still reachable: 135,825,332 bytes in 23,583 blocks
==813==                       of which reachable via heuristic:
==813==                         stdstring          : 97 bytes in 1 blocks
==813==                         length64           : 2,208 bytes in 51
blocks
==813==                         newarray           : 2,352 bytes in 36
blocks
==813==         suppressed: 0 bytes in 0 blocks

without "definitely" details in the report, we guess that the memory leakage
happens in the gstreamer lib.

Could you help us? thanks a lot!


Nicolas Dufresne-5 wrote

> Le jeudi 29 novembre 2018 à 03:59 +0000, oonamao(毛江云) a écrit :
>> Hi all,
>>  
>> This is Oona, a junior software engineer. I send this email to ask your
>> help about my gstreamer application:
>> Recently, I met a memory leak problem on TX1 platform, when my gstreamer
>> application tried to reconnect to the camera by rtsp.
>> Moreover, the tegra I’m using is:
>> # R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t210ref, EABI:
>> aarch64, DATE: Fri Mar  2 04:58:16 UTC 2018
>> Gstreamer is:
>> gst-inspect-1.0 version 1.8.1
>> GStreamer 1.8.3
>> https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
>
> This is relatively old, if it's an upstream bug, it is likely fixed.
>
>>  
>> Problem Details:
>> 1.      The application will call gst_init() at the very first, then
>> builds pipleline as “rtspsrc
>> location=rtsp://${usrname}:${pwd}@${ip}:${port}/cam/realmonitor?channel=1&subtype=0
>> protocols=tcp ! rtph264depay ! h264parse ! omxh264dec ! video/x-raw,
>> format=NV12 ! appsink sync=false async=false name=appsink1”. As expected,
>> it gets data from camera. By `top` commands, I can see its
>> occupied-memory is ~108M;
>> (disable ${ip})
>
> Though, most of these report are related to miss-used of appsink.
> appsink is non leak and have inifit queues by default. Make sure you
> limite that queue and pop and unref buffers properly.
>
>> 2.      Of course, the application fails to get data. And the application
>> is trying to reconnect the pipeline periodically. It is still occupied
>> ~108M;
>> (enable ${ip})
>> 3.      The application succeeds to get video again. However, the memory
>> rises to ~180M!
>> After trying 2,3 for times, I finds that: the memory will increase by
>> -80M each time when it succeed in reconnection.
>>  
>> The following code segment is the reconnection (first close() then
>> open()). I think I have unref all the resources in close(), in addition,
>> I unref gst-sample, gst-message in time.
>> ```
>> int GSTCameraMini::open() {
>>   this->m_camera_state = CAM_INIT;
>>   this -> build_pipeline();
>>   gst_element_set_state(this -> pipeline, GST_STATE_PLAYING);
>>   return 0;
>> }
>>
>> int GSTCameraMini::close() {
>>   this->m_camera_state = CAM_OFFLINE;
>>   gst_element_set_state(this -> pipeline, GST_STATE_NULL);
>>   // unref appsink
>>   gst_object_unref(G_OBJECT(this -> appsink));
>>   // unref bus
>>   gst_object_unref(G_OBJECT(this -> bus));
>>   // unref pipeline
>>   gst_object_unref(G_OBJECT(this -> pipeline));
>>   return 0;
>> }
>> ```
>> I have no idea where goes wrong, and this problem makes me headache for a
>> long time. It is a bug from gstreamer or other?
>> Sincerely looking forward to your reply.
>>  
>> Best & Regards,
>> Oona
>>  
>> _______________________________________________
>> gstreamer-devel mailing list
>>

> gstreamer-devel@.freedesktop

>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list

> gstreamer-devel@.freedesktop

> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> signature.asc (201 bytes)
> &lt;http://gstreamer-devel.966125.n4.nabble.com/attachment/4689075/0/signature.asc&gt;





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: memory leak when reconnection to camera using gstreamer on TX1 platform

Nicolas Dufresne-5
Le mardi 11 décembre 2018 à 20:44 -0600, oona a écrit :

> Hi,
>
> Thanks for your reply.
>
> I refactor my application, limiting the appsink as:
>     gst_app_sink_set_emit_signals((GstAppSink*)g_appsink, true);
>     gst_app_sink_set_drop((GstAppSink*)g_appsink, true);
>     gst_app_sink_set_max_buffers((GstAppSink*)g_appsink, 1);
>
> unrefing gstSample in time like function NewFrame() in:  github
> <https://github.com/rticommunity/rticonnextdds-videodemo-android/blob/master/jni/ConnextGstreamer.cxx>
> .
>
> Now, the memory does not increase for each reconnection. However, for the
> first time, the memory will rise from 80M to 120M, later it is floating
> around 120M.
>
> Moreover, I use valgrind to test the memory leakage, the report is:
> ==813== LEAK SUMMARY:
> ==813==    definitely lost: 20,854 bytes in 49 blocks
> ==813==    indirectly lost: 5,281 bytes in 80 blocks
> ==813==      possibly lost: 8,572 bytes in 73 blocks
> ==813==    still reachable: 135,825,332 bytes in 23,583 blocks
> ==813==                       of which reachable via heuristic:
> ==813==                         stdstring          : 97 bytes in 1 blocks
> ==813==                         length64           : 2,208 bytes in 51
> blocks
> ==813==                         newarray           : 2,352 bytes in 36
> blocks
> ==813==         suppressed: 0 bytes in 0 blocks
>
> without "definitely" details in the report, we guess that the memory leakage
> happens in the gstreamer lib.
>
> Could you help us? thanks a lot!

You can use GStreamer leak tracer to get the list of still allocated
blocks, might give you a hint, though it's more likely an application
issue here.

GST_TRACERS=leak GST_DEBUG=GST_TRACER:7 ./your-app

>
>
> Nicolas Dufresne-5 wrote
> > Le jeudi 29 novembre 2018 à 03:59 +0000, oonamao(毛江云) a écrit :
> > > Hi all,
> > >  
> > > This is Oona, a junior software engineer. I send this email to ask your
> > > help about my gstreamer application:
> > > Recently, I met a memory leak problem on TX1 platform, when my gstreamer
> > > application tried to reconnect to the camera by rtsp.
> > > Moreover, the tegra I’m using is:
> > > # R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t210ref, EABI:
> > > aarch64, DATE: Fri Mar  2 04:58:16 UTC 2018
> > > Gstreamer is:
> > > gst-inspect-1.0 version 1.8.1
> > > GStreamer 1.8.3
> > > https://launchpad.net/distros/ubuntu/+source/gstreamer1.0
> >
> > This is relatively old, if it's an upstream bug, it is likely fixed.
> >
> > >  
> > > Problem Details:
> > > 1.      The application will call gst_init() at the very first, then
> > > builds pipleline as “rtspsrc
> > > location=rtsp://${usrname}:${pwd}@${ip}:${port}/cam/realmonitor?channel=1&subtype=0
> > > protocols=tcp ! rtph264depay ! h264parse ! omxh264dec ! video/x-raw,
> > > format=NV12 ! appsink sync=false async=false name=appsink1”. As expected,
> > > it gets data from camera. By `top` commands, I can see its
> > > occupied-memory is ~108M;
> > > (disable ${ip})
> >
> > Though, most of these report are related to miss-used of appsink.
> > appsink is non leak and have inifit queues by default. Make sure you
> > limite that queue and pop and unref buffers properly.
> >
> > > 2.      Of course, the application fails to get data. And the application
> > > is trying to reconnect the pipeline periodically. It is still occupied
> > > ~108M;
> > > (enable ${ip})
> > > 3.      The application succeeds to get video again. However, the memory
> > > rises to ~180M!
> > > After trying 2,3 for times, I finds that: the memory will increase by
> > > -80M each time when it succeed in reconnection.
> > >  
> > > The following code segment is the reconnection (first close() then
> > > open()). I think I have unref all the resources in close(), in addition,
> > > I unref gst-sample, gst-message in time.
> > > ```
> > > int GSTCameraMini::open() {
> > >   this->m_camera_state = CAM_INIT;
> > >   this -> build_pipeline();
> > >   gst_element_set_state(this -> pipeline, GST_STATE_PLAYING);
> > >   return 0;
> > > }
> > >
> > > int GSTCameraMini::close() {
> > >   this->m_camera_state = CAM_OFFLINE;
> > >   gst_element_set_state(this -> pipeline, GST_STATE_NULL);
> > >   // unref appsink
> > >   gst_object_unref(G_OBJECT(this -> appsink));
> > >   // unref bus
> > >   gst_object_unref(G_OBJECT(this -> bus));
> > >   // unref pipeline
> > >   gst_object_unref(G_OBJECT(this -> pipeline));
> > >   return 0;
> > > }
> > > ```
> > > I have no idea where goes wrong, and this problem makes me headache for a
> > > long time. It is a bug from gstreamer or other?
> > > Sincerely looking forward to your reply.
> > >  
> > > Best & Regards,
> > > Oona
> > >  
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > >
> > gstreamer-devel@.freedesktop
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel@.freedesktop
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> >
> > signature.asc (201 bytes)
> > &lt;http://gstreamer-devel.966125.n4.nabble.com/attachment/4689075/0/signature.asc&gt;
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: memory leak when reconnection to camera using gstreamer on TX1 platform

oona
Thanks for your kind hint, I will try it later.

But here I saw a sentence from GStreamer reference manual:
"""
All rendered buffers will be put in a queue so that the application can pull
samples at its own rate. Note that when the application does not pull
samples fast enough, the queued buffers could consume a lot of memory,
especially when dealing with raw video frames.
"""

If I limit the appsink as:
```
    gst_app_sink_set_emit_signals((GstAppSink *)appsink, true);
    gst_app_sink_set_drop((GstAppSink *)appsink, true);
    gst_app_sink_set_max_buffers((GstAppSink *)appsink, 1);
```
can problem mentioned above be solved?


Looking forward to your reply, thanks!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: memory leak when reconnection to camera using gstreamer on TX1 platform

oona
In reply to this post by Nicolas Dufresne-5
Sorry for the letter again, I test the performance of gstreamer of
reconnection. Method is:  I enable the iptables for camera-ip, and waiting
for 300s, I disable the iptables, waiting for 300s, enable...(loop)

the result please see:
https://github.com/yuna90/gstreamer_streamer/blob/master/stream_1camera_6fps.png

*
It is obvious that the memory is rising for the first 5 period. I'm afraid
that later it might rise again...
*

you can see the demo code in the same repository.


Looking forward to your reply





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel