Naming threads ? Other performance debugging helpers ?

[Florent THIERY-2]

Hi

I was wondering if it would be feasible that elements that launch
threads (e.g. queues) could name threads (using prctl(PR_SET_NAME? ),
with the name being constructed after their name, or the neighbour
elements' name. After a quick search [1], i'm not sure the Linux
kernel offers this feature

This would help a lot performance debugging : so far if you want to
know what thread is consuming the most power, you make assumptions on
the nature of processing, and study partial pipelines to estimate the
proportion taken by what, but this approach is very limiting for
detecting/understanding side effects.

Also, any other idea about how to "map" the CPU cycles consumption of
a pipeline or other performance indicators would be welcome. I am
considering developing a tool which would display states/hints of a
pipeline's behaviour in a graphical fashion. So far i experimented
with polling queue filling states, but it does not always prove
helpful. I also don't know how i can access QoS data from python.

Regards,

FLorent

[1] http://www.gossamer-threads.com/lists/engine?list=linux&do=search_results&search_forum=forum_1&search_string=thread+name&search_type=AND

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

Hi Florent,

Florent schrieb:
> Hi
>
> I was wondering if it would be feasible that elements that launch
> threads (e.g. queues) could name threads (using prctl(PR_SET_NAME? ),
> with the name being constructed after their name, or the neighbour
> elements' name. After a quick search [1], i'm not sure the Linux
> kernel offers this feature

https://bugzilla.gnome.org/show_bug.cgi?id=580505

I'd like to have this too, but I don't think it can be done in a crossplatform,
non hackish way.
>
> This would help a lot performance debugging : so far if you want to
> know what thread is consuming the most power, you make assumptions on
> the nature of processing, and study partial pipelines to estimate the
> proportion taken by what, but this approach is very limiting for
> detecting/understanding side effects.

What you could do is to add some thread-id name association to gstreamer. The
you could write gdb extension to query it? Or the logging could print the name
as well. Where would you need the name?

>
> Also, any other idea about how to "map" the CPU cycles consumption of
> a pipeline or other performance indicators would be welcome. I am
> considering developing a tool which would display states/hints of a
> pipeline's behaviour in a graphical fashion. So far i experimented
> with polling queue filling states, but it does not always prove
> helpful. I also don't know how i can access QoS data from python.

As my time permits I hack on gst-tracelib [1]. One plan is to intergrate the
graphing (which is done by gnuplot) into gst-debug-viewer [2]. The later is
written in python. WOuld be cool if you would consider helping on that one.

Stefan

[1] http://cgit.freedesktop.org/~ensonic/gst-tracelib/
[2] http://cgit.freedesktop.org/~cymacs/gst-debug-viewer/

>
> Regards,
>
> FLorent
>
> [1] http://www.gossamer-threads.com/lists/engine?list=linux&do=search_results&search_forum=forum_1&search_string=thread+name&search_type=AND
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Florent THIERY-2]

Hello Stefan,

> https://bugzilla.gnome.org/show_bug.cgi?id=580505
>
> I'd like to have this too, but I don't think it can be done in a crossplatform,
> non hackish way.
> What you could do is to add some thread-id name association to gstreamer. The
> you could write gdb extension to query it? Or the logging could print the name
> as well. Where would you need the name?

Indeed, naming threads is a non-standard feature. The elements which
spawn a thread could report their new PID in the gst logs, and/or emit
a gst message ("new-thread", with {"pid", "gst_element_name"} as
contents). This would be cross-platform, right ?

> As my time permits I hack on gst-tracelib [1]. One plan is to intergrate the
> graphing (which is done by gnuplot) into gst-debug-viewer [2]. The later is
> written in python. Would be cool if you would consider helping on that one.

I quickly tried them, this is awesome work :) I agree, integrating
plotting into gst-debug-viewer would be great. However, it also means
that log generation (and dynamic linking) should be unified, so that
both gstreamer log and performance logs are generated in the same step
? As for integrating the plots, some kind of image viewer could do the
trick, right (scrollable pane with png/svg inside)?

Sadly, performance tracing doesn't work with python-based gstreamer
apps... Any idea how to let python also use them ?

My question was however more related to quick/real time performance
debugging. I ended up coding http://code.google.com/p/gst-viewperf/,
which uses queue filling states to graphically point out performance
starvations. This is merely a prototype, but it works and helped me
already. I am planning on integrating python-gtop (not before Ubuntu
9.04 though), which will enable the use of libgtop (aka
gnome-system-monitor's lib) so that system performance plotting would
be integrable (cpu, mem, disk, network) in python apps. If onlt i
could get the gst element's spawned PIDs, i could also embed the
plugin-related CPU usage.

In this scope, i am asking for help regarding getting the graph
structure of a pipeline: how can i get a graph representation of the
pipeline ? Here, i merely parse the pipeline description... Works for
linearly-defined pipelines, but not in all cases.

PS: sorry for the delay, i don't have much time for spare projects...

Florent

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

Florent schrieb:

> Hello Stefan,
>
>> https://bugzilla.gnome.org/show_bug.cgi?id=580505
>>
>> I'd like to have this too, but I don't think it can be done in a crossplatform,
>> non hackish way.
>> What you could do is to add some thread-id name association to gstreamer. The
>> you could write gdb extension to query it? Or the logging could print the name
>> as well. Where would you need the name?
>
> Indeed, naming threads is a non-standard feature. The elements which
> spawn a thread could report their new PID in the gst logs, and/or emit
> a gst message ("new-thread", with {"pid", "gst_element_name"} as
> contents). This would be cross-platform, right ?
>
>> As my time permits I hack on gst-tracelib [1]. One plan is to intergrate the
>> graphing (which is done by gnuplot) into gst-debug-viewer [2]. The later is
>> written in python. Would be cool if you would consider helping on that one.
>
> I quickly tried them, this is awesome work :) I agree, integrating
> plotting into gst-debug-viewer would be great. However, it also means
> that log generation (and dynamic linking) should be unified, so that
> both gstreamer log and performance logs are generated in the same step
> ? As for integrating the plots, some kind of image viewer could do the
> trick, right (scrollable pane with png/svg inside)?

Yep, you need to run tracelib and generate a debug log at the same time. In the
viewer we would draw the curves ourself, so that we can zoom. I spent too much
with gnuplot already.
>
> Sadly, performance tracing doesn't work with python-based gstreamer
> apps... Any idea how to let python also use them ?

It should work with python apps too, its just hooking into the underlying
gstreamer. What happens?

>
> My question was however more related to quick/real time performance
> debugging. I ended up coding http://code.google.com/p/gst-viewperf/,
> which uses queue filling states to graphically point out performance
> starvations. This is merely a prototype, but it works and helped me
> already.

Looks neat. So one would need to insert a queue after each element? You are
aware that this modifies the runtime behaviour heavily. Anyway we have the same
goals - somehow shing graphically bottlenecks.

> I am planning on integrating python-gtop (not before Ubuntu
> 9.04 though), which will enable the use of libgtop (aka
> gnome-system-monitor's lib) so that system performance plotting would
> be integrable (cpu, mem, disk, network) in python apps. If onlt i
> could get the gst element's spawned PIDs, i could also embed the
> plugin-related CPU usage.

I was also thinking of a plugin api for tracelib to allow plugging more sensors,
e.g. a dsp-cpu-load sensor.

>
> In this scope, i am asking for help regarding getting the graph
> structure of a pipeline: how can i get a graph representation of the
> pipeline ? Here, i merely parse the pipeline description... Works for
> linearly-defined pipelines, but not in all cases.

Have you looked at gstreamer/gst/debugutils.c &
GST_DEBUG_BIN_TO_DOT_FILE? This is dumping the current pipeline as a dot file
and one can render that via graphviz.

One idea would be to use the graphviz api directly to dynamically draw the
pipelines. Unfortunately the docs are in doxygen - good luck with finding where
to start :/
http://www.graphviz.org/pub/graphviz/development/doxygen/html/index.html
I also found
http://www.graphviz.org/pdf/cgraph.3.pdf
http://www.graphviz.org/pdf/libguide.pdf

now you would need python bindings :/
>
> PS: sorry for the delay, i don't have much time for spare projects...

I know. mee too :/

Stefan

>
> Florent
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel