GStreamer-devel

Rethinking --disable-gst-debug option in gstreamer configure script

Classic

List

Threaded

6 messages Options

Crane, Matt

Rethinking --disable-gst-debug option in gstreamer configure script

Gstreamer developers:

I am an engineer in a group that has developed a gstreamer application that handles the recording of dozens of video streams in a single server. Recent load tests we’ve run with our application have revealed that disabling debug logging during compile time via use of the –disable-gst-debug option in the gstreamer configure script frees up about 10% of the CPU cycles on our target hardware configuration (CPU idle time with debugging enabled = ~50%; CPU idle time with debugging disabled = ~60%). While we would like to realize the performance improvement that would come from disabling (much of the) debug logging, we would still like to log ERROR and WARNING level messages on deployed production systems.

Toward this end, we are thinking about changing the compile-time configuration of debug logging to allow different levels of debug enablement. Instead of having a single –disable-gst-debug option, we might like to have a configure option like: –gst-debug-logging=[disabled, warning, full]. In looking at the current codebase, –disable-gst-debug results in setting almost all of the gstreamer debugging system function call macros to No-Op using G_STMT_START{ }G_STMT_END. In our proposed solution, we would selectively set appropriate macros to No-Op to match the desired gst-debug-logging parameter. Going this route, it is thought that we could maintain the availability of ERROR and WARNING messages without keeping the overhead of higher level debug messages INFO, DEBUG, etc…

I wanted to float this proposal here on the devel list because, ideally, we would like whatever idea we implement to be incorporated upstream into the gstreamer project. If any of the key contributors would like to volunteer feedback on this idea (is it something you’d consider incorporating into the project? is there something about how you’d like the scheme implemented?), we would greatly appreciate it.

Thank you for your time…

-Matt

------------------------------------------------------------------------------
Virtualization is moving to the mainstream and overtaking non-virtualized
environment for deploying applications. Does it make network security
easier or more difficult to achieve? Read this whitepaper to separate the
two and get a better understanding.
http://p.sf.net/sfu/hp-phase2-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Felipe Contreras

Re: Rethinking --disable-gst-debug option in gstreamer configure script

Hi,

On Mon, Oct 4, 2010 at 8:49 PM, Crane, Matt <[hidden email]> wrote:

> I am an engineer in a group that has developed a gstreamer application that
> handles the recording of dozens of video streams in a single server. Recent
> load tests we’ve run with our application have revealed that disabling debug
> logging during compile time via use of the –disable-gst-debug option in the
> gstreamer configure script frees up about 10% of the CPU cycles on our
> target hardware configuration (CPU idle time with debugging enabled = ~50%;
> CPU idle time with debugging disabled = ~60%). While we would like to
> realize the performance improvement that would come from disabling (much of
> the) debug logging, we would still like to log ERROR and WARNING level
> messages on deployed production systems.

Stefan Sauer

Re: Rethinking --disable-gst-debug option in gstreamer configure script

In reply to this post by Crane, Matt

Am 04.10.2010 20:49, schrieb Crane, Matt:

> Gstreamer developers:
>
>
>
> I am an engineer in a group that has developed a gstreamer application that
> handles the recording of dozens of video streams in a single server. Recent
> load tests we’ve run with our application have revealed that disabling debug
> logging during compile time via use of the –disable-gst-debug option in the
> gstreamer configure script frees up about 10% of the CPU cycles on our target
> hardware configuration (CPU idle time with debugging enabled = ~50%; CPU idle
> time with debugging disabled = ~60%). While we would like to realize the
> performance improvement that would come from disabling (much of the) debug
> logging, we would still like to log ERROR and WARNING level messages on deployed
> production systems.
>
>
>
> Toward this end, we are thinking about changing the compile-time configuration
> of debug logging to allow different levels of debug enablement. Instead of
> having a single –disable-gst-debug option, we might like to have a configure
> option like: –gst-debug-logging=[disabled, warning, full]. In looking at the
> current codebase, –disable-gst-debug results in setting almost all of the
> gstreamer debugging system function call macros to No-Op using G_STMT_START{
> }G_STMT_END. In our proposed solution, we would selectively set appropriate
> macros to No-Op to match the desired gst-debug-logging parameter. Going this
> route, it is thought that we could maintain the availability of ERROR and
> WARNING messages without keeping the overhead of higher level debug messages
> INFO, DEBUG, etc…
>

I wanted if some uses of GST_ERROR should become a GST_ELEMENT_ERROR instead
(likewise for the warnings). Do you have some example of messages that you would
like to still get with debug logging turned off.

Stefan

>
>
> I wanted to float this proposal here on the devel list because, ideally, we
> would like whatever idea we implement to be incorporated upstream into the
> gstreamer project. If any of the key contributors would like to volunteer
> feedback on this idea (is it something you’d consider incorporating into the
> project? is there something about how you’d like the scheme implemented?), we
> would greatly appreciate it.
>
>
>
> Thank you for your time…
>
>
>
> -Matt
>
>
>
>
>
> ------------------------------------------------------------------------------
> Virtualization is moving to the mainstream and overtaking non-virtualized
> environment for deploying applications. Does it make network security
> easier or more difficult to achieve? Read this whitepaper to separate the
> two and get a better understanding.
> http://p.sf.net/sfu/hp-phase2-d2d
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Gruenke, Matt

Re: Rethinking --disable-gst-debug option in gstreamer configure script

In reply to this post by Felipe Contreras

Felipe, gst_debug_add_log_function() enables you to configure a custom log handler with the following signature:

void (*GstLogFunction) (GstDebugCategory *category,
GstDebugLevel level,
const gchar *file,
const gchar *function,
gint line,
GObject *object,
GstDebugMessage *message,
gpointer data);

If you want to alter behavior of your handler based on origin, you can use category and object (if this is non-null, you could check its type ID). From what I can tell, you can use this existing mechanism to satisfy all of your requirements. See http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstInfo.html#gst-debug-add-log-function for more info.

The issue that we're having is that simply calling gst_debug_log()/gst_debug_log_valist() for the messages logged at certain levels uses a significant amount of CPU time. We just want to be able to compile out some levels, while keeping others. We've found this simple and straight-forward to implement, and I'd prefer to keep it separate from discussions of other changes to the logging system.

Thanks,
Matt

-----Original Message-----
From: Felipe Contreras [mailto:[hidden email]]
Sent: Monday, October 04, 2010 16:13
To: Discussion of the development of GStreamer
Subject: Re: [gst-devel] Rethinking --disable-gst-debug option in gstreamer configure script

Hi,

On Mon, Oct 4, 2010 at 8:49 PM, Crane, Matt <[hidden email]> wrote:

> I am an engineer in a group that has developed a gstreamer application that
> handles the recording of dozens of video streams in a single server. Recent
> load tests we've run with our application have revealed that disabling debug
> logging during compile time via use of the -disable-gst-debug option in the
> gstreamer configure script frees up about 10% of the CPU cycles on our
> target hardware configuration (CPU idle time with debugging enabled = ~50%;
> CPU idle time with debugging disabled = ~60%). While we would like to
> realize the performance improvement that would come from disabling (much of
> the) debug logging, we would still like to log ERROR and WARNING level
> messages on deployed production systems.

Totally agree, that's why I wrote my own:
http://github.com/felipec/gst-dsp/blob/master/log.c
http://github.com/felipec/gst-dsp/blob/master/log.h

Not only do I want the errors are warning to be printed, but stored in
the syslog. And more over, even if the systems's debugging
capabilities are disabled, I still would like to enable debugging on
my plug-in (override). And finally, I want a higher level that all
others that *always* print messages (not on the syslog though), in
order add logging only in certain areas (I call this pr_test).

Right now there's nothing like that on GStreamer, which is not ideal
for embedded systems.

Cheers.

--
Felipe Contreras

------------------------------------------------------------------------------
Virtualization is moving to the mainstream and overtaking non-virtualized
environment for deploying applications. Does it make network security
easier or more difficult to achieve? Read this whitepaper to separate the
two and get a better understanding.
http://p.sf.net/sfu/hp-phase2-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Crane, Matt

Re: Rethinking --disable-gst-debug option in gstreamer configure script

In reply to this post by Stefan Sauer

> From: Stefan Kost [mailto:[hidden email]]

> I wanted if some uses of GST_ERROR should become a GST_ELEMENT_ERROR
> instead
> (likewise for the warnings). Do you have some example of messages that
> you would
> like to still get with debug logging turned off.
>
> Stefan

Stefan:

We don't have anything specific in mind. We simply figured that having
ERROR and WARNING level messages would be a good idea and wanted to
implement our change in whatever way would maximize the probability of
it being accepted by the project.

-Matt

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Stefan Sauer

Re: Rethinking --disable-gst-debug option in gstreamer configure script

hi Matt,

On 06.10.2010 17:53, Crane, Matt wrote:

>> From: Stefan Kost [mailto:[hidden email]]
>>
>
>> I wanted if some uses of GST_ERROR should become a GST_ELEMENT_ERROR
>> instead
>> (likewise for the warnings). Do you have some example of messages that
>> you would
>> like to still get with debug logging turned off.
>>
>> Stefan
>>
> Stefan:
>
> We don't have anything specific in mind. We simply figured that having
> ERROR and WARNING level messages would be a good idea and wanted to
> implement our change in whatever way would maximize the probability of
> it being accepted by the project.
>

I would start by publishing your patch in bugzilla. I definitely
understand the point and see benefits of it. So if the patch is not too
intrusive, why not.

Stefan

> -Matt
>
>
> ------------------------------------------------------------------------------
> Beautiful is writing same markup. Internet Explorer 9 supports
> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
> Spend less time writing and rewriting code and more time creating great
> experiences on the web. Be a part of the beta today.
> http://p.sf.net/sfu/beautyoftheweb
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>