Support for Massive Multicore Machines?

[Stirling Westrup]

I know that GStreamer is multi-threaded, but I'm worried that it
doesn't make efficient use of CPUs like the new AMD 64-core
processors. Are there plans to better support such devices in the
future? Are there any recommendations for folks targeting such
equipment today?

--
Stirling Westrup
Programmer, Entrepreneur.
https://www.linkedin.com/e/fpf/77228
http://www.linkedin.com/in/swestrup
http://technaut.livejournal.com
http://sourceforge.net/users/stirlingwestrup
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Thu, 2019-10-10 at 14:03 -0400, Stirling Westrup wrote:
> I know that GStreamer is multi-threaded, but I'm worried that it
> doesn't make efficient use of CPUs like the new AMD 64-core
> processors. Are there plans to better support such devices in the
> future? Are there any recommendations for folks targeting such
> equipment today?

What potential problems do you see? That would probably allow to give a
more useful answer to your question :)

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stirling Westrup]

Well, as I see it, there is generally one thread spawned per queue
device, and one per encode/decode operation, and several for command
and control functions. This means that on a 64-core machine a simple
video player will use something like 8 or 10 threads, leaving the vast
majority of the machine idle. When one is displaying 8K video, that
might matter.

On Thu, Oct 10, 2019 at 2:13 PM Sebastian Dröge
<[hidden email]> wrote:

>
> On Thu, 2019-10-10 at 14:03 -0400, Stirling Westrup wrote:
> > I know that GStreamer is multi-threaded, but I'm worried that it
> > doesn't make efficient use of CPUs like the new AMD 64-core
> > processors. Are there plans to better support such devices in the
> > future? Are there any recommendations for folks targeting such
> > equipment today?
>
> What potential problems do you see? That would probably allow to give a
> more useful answer to your question :)
>
> --
> Sebastian Dröge, Centricular Ltd · https://www.centricular.com

--
Stirling Westrup
Programmer, Entrepreneur.
https://www.linkedin.com/e/fpf/77228
http://www.linkedin.com/in/swestrup
http://technaut.livejournal.com
http://sourceforge.net/users/stirlingwestrup
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Olivier Crête-3]

Hi,

The video decoder can be internally multi-threaded. We also have a
multi-threaded video scaler now, so that can use multiple cores.

That's really all that uses the CPU for those. There is a limit to what
GStreamer can do for you.

Olivier

On Thu, 2019-10-10 at 14:22 -0400, Stirling Westrup wrote:

> Well, as I see it, there is generally one thread spawned per queue
> device, and one per encode/decode operation, and several for command
> and control functions. This means that on a 64-core machine a simple
> video player will use something like 8 or 10 threads, leaving the vast
> majority of the machine idle. When one is displaying 8K video, that
> might matter.
>
> On Thu, Oct 10, 2019 at 2:13 PM Sebastian Dröge
> <[hidden email]> wrote:
> > On Thu, 2019-10-10 at 14:03 -0400, Stirling Westrup wrote:
> > > I know that GStreamer is multi-threaded, but I'm worried that it
> > > doesn't make efficient use of CPUs like the new AMD 64-core
> > > processors. Are there plans to better support such devices in the
> > > future? Are there any recommendations for folks targeting such
> > > equipment today?
> >
> > What potential problems do you see? That would probably allow to give a
> > more useful answer to your question :)
> >
> > --
> > Sebastian Dröge, Centricular Ltd · https://www.centricular.com
>
>

--
Olivier Crête
[hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le jeu. 10 oct. 2019 14 h 11, Stirling Westrup <[hidden email]> a écrit :

I know that GStreamer is multi-threaded, but I'm worried that it
doesn't make efficient use of CPUs like the new AMD 64-core
processors. Are there plans to better support such devices in the
future? Are there any recommendations for folks targeting such
equipment today?

Do you have numbers to support this ? I've done some test on ThreadRipper 2990, and many things scale very well after configuring it (tried software encoding and software color conversion). Of course, just adding queues as a parallel method won't work, but many element already have ability to use multiple threads internally.

For servers, where the 64 threads can easily be exhausted, I believe the best way forward is using the rust threadshare, looks a bit experimental, but sounds like the right approach.

--
Stirling Westrup
Programmer, Entrepreneur.
https://www.linkedin.com/e/fpf/77228
http://www.linkedin.com/in/swestrup
http://technaut.livejournal.com
http://sourceforge.net/users/stirlingwestrup
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel