Memory Leak?

[Tarun Tej K]

Hi,

Have you checked the memory consumption with this latest pipeline? Is it under control? Are you still using '--disable-passthrough' and gst_base_parse_set_pts_interpolation(GST_BASE_PARSE(parse), TRUE)   H264parse and 'leaky' in queues for the latest pipeline also?

We are also facing similar issue on iMX6UL. When we run multiple pipelines simultaneously, the memory gets exhausted very quickly.

Here is the sample pipeline we are trying:

gst-launch-1.0 -e rtspsrc location="rtsp://<username>:<password>@<Url>" name = d 
! rtph264depay ! h264parse ! splitmuxsink name = mux location=a1_video%02d.mp4 max-size-time=10000000000 max-files=3

On Sat 6 Oct, 2018, 5:28 AM killerrats, <[hidden email]> wrote:

gst-launch-1.0 -e rtspsrc location=[IP] latency=66 name=source !
rtpjitterbuffer do-lost=true ! rtph264depay ! identity ! h264parse
config-interval=1 ! avdec_h264 ! openh264enc ! h264parse ! tee name=vtee !
mq.sink_1 multiqueue name=mq ! mux.video splitmuxsink async-handling=true
max-size-bytes=4194304 max-files=360 muxer=mp4mux location=video%06d.avi
name=mux mq.src_2 ! mux.audio_1 source. ! rtpjitterbuffer do-lost=true !
rtpmp4gdepay ! identity ! aacparse ! avdec_aac ! avenc_aac ! aacparse ! tee
name=atee ! mq.sink_2

found out if I do this for each side of the branches for audio and video.

depay -> parse -> decode -> encode -> parse -> tee

if i leave the decode -> encode -> parse out of it the pipeline won't do
that stopping point like it did. even the sync of audio and video works.



-----
------------------------------
Gstreamer 1.14.2
------------------------------
Windows
--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[killerrats]

I didn't use the pipeline without the decode encode parse on the new version.

I have used the pipeline for 5 days continuously with 30mb of memory
averagely with no hick-ups. I actually didn't use the disable-passthrough
but i did use the
gst_base_parse_set_pts_interpolation(GST_BASE_PARSE(parse), TRUE). long as I
did decode encode parse way it worked long term. That one was for the mp4mux
and qtmux because they want to use pts.

the queue leak i actually modified the splitmuxsink source code but I used
the splitmuxsink that comes with the gstreamer and works fine, at least the
5 days did well.



-----
------------------------------
Gstreamer 1.14.4
------------------------------
Windows
--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tarun Tej K]

Thanks. I will give  a try and update here

On Mon 8 Oct, 2018, 9:57 AM killerrats, <[hidden email]> wrote:

I didn't use the pipeline without the decode encode parse on the new version.

I have used the pipeline for 5 days continuously with 30mb of memory
averagely with no hick-ups. I actually didn't use the disable-passthrough
but i did use the
gst_base_parse_set_pts_interpolation(GST_BASE_PARSE(parse), TRUE). long as I
did decode encode parse way it worked long term. That one was for the mp4mux
and qtmux because they want to use pts.

the queue leak i actually modified the splitmuxsink source code but I used
the splitmuxsink that comes with the gstreamer and works fine, at least the
5 days did well.



-----
------------------------------
Gstreamer 1.14.4
------------------------------
Windows
--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[killerrats]

Here are the files that are splitmuxsink modified that I did

gstSplitMuxSink.h gstSplitMuxSink.cpp

---

Sent from the GStreamer-devel mailing list archive at Nabble.com.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tarun Tej K]

OK, I have run the below pipeline with extra decode -> encode -> parse as you have been using. Additionally I have added gst_base_parse_set_pts_interpolation(GST_BASE_PARSE(parser), 1); in the C program.

gst-launch-1.0  -e rtspsrc location="rtsp://<usrnm>:<pass>@<URL>" name = d !  rtph264depay ! h264parse config-interval=1 ! avdec_h264  ! x264enc ! splitmuxsink name = mux location=a1_video%02d.mp4 max-size-time=2000000000 max-files=5

But it turned only worse than before, the OOM killer gets invoked even with single pipeline. (Previously: Without the extra 'decode-> encode -> parse' I was able to run 3 pipelines without any issues. Once i add a 4th pipeline, the memory keeps growing.)

I have not tried with modification in splitmuxsink's queue yet.

On Mon, Oct 8, 2018 at 12:12 PM killerrats <[hidden email]> wrote:

Here are the files that are splitmuxsink modified that I did

gstSplitMuxSink.h gstSplitMuxSink.cpp

---

Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tarun Tej K]

Sorry, copied the pipeline wrong in the previous email. Here is the correct one:

gst-launch-1.0  -e rtspsrc location="rtsp://<usrnm>:<pass>@<URL>" name = d !  rtph264depay ! h264parse config-interval=1 ! avdec_h264  ! x264enc !  h264parse ! splitmuxsink name = mux location=a1_video%02d.mp4 max-size-time=2000000000 max-files=5  

On Fri, Oct 12, 2018 at 5:25 PM Tarun Tej K <[hidden email]> wrote:

OK, I have run the below pipeline with extra decode -> encode -> parse as you have been using. Additionally I have added gst_base_parse_set_pts_interpolation(GST_BASE_PARSE(parser), 1); in the C program.

gst-launch-1.0  -e rtspsrc location="rtsp://<usrnm>:<pass>@<URL>" name = d !  rtph264depay ! h264parse config-interval=1 ! avdec_h264  ! x264enc ! splitmuxsink name = mux location=a1_video%02d.mp4 max-size-time=2000000000 max-files=5

But it turned only worse than before, the OOM killer gets invoked even with single pipeline. (Previously: Without the extra 'decode-> encode -> parse' I was able to run 3 pipelines without any issues. Once i add a 4th pipeline, the memory keeps growing.)

I have not tried with modification in splitmuxsink's queue yet.

On Mon, Oct 8, 2018 at 12:12 PM killerrats <[hidden email]> wrote:

Here are the files that are splitmuxsink modified that I did

gstSplitMuxSink.h gstSplitMuxSink.cpp

---

Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[killerrats]

My co-worker said once that using the decode / encode on the raspberry pi
that it seem to use software and not hardware. which might actually involve
the high memory maybe.



-----
------------------------------
Gstreamer 1.14.4
------------------------------
Windows
--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[killerrats]

So far what I found is that if there is a same buffer for period of time I
have to stop the process to get rid of the memory.

start the pipeline again in same process:
still have references in memory. All I do null the pipeline and unref the
pipeline then start all over again. The memory will increase each time it
gets stuck and restarts the pipeline again.



-----
------------------------------
Gstreamer 1.14.4
------------------------------
Windows
--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel