Several questions regarding the Python bindings for GStreamer (Asyncio compat, mixing of streams, reading directly from BytesIO)

[Hadrien Titeux]

Hello,

It's kind of my first time posting on a dev mailing list so please, be gentle. If the questions that follow are totally out of place, I'm really sorry for wasting your time (but could you please redirect me to some place where people my answer might answer my questions)

My question stems from a a particular small usecase of Gstreamer. The problematic is the following: I want to build a client for some sort of audio chat (it's a bit uncommon, don't ask) that sends wav audio bytes over websocket. I've already built most of the interface using the GTK3+ bindings for Python, and now the sound-playing part remains. I could very well use the portaudio bindings (and i've done it on another terminal-based client) to do this, but I'm in a mood where I want to tackle this problem using the gst python bindings (because portaudio kind of sucks).

So, here are what I'd like to know:

- Is it possible to play a .wav file without saving it to disk first? Like, directly from the byte file stored in the python memory (as portaudio can do it)

- The websocket receives sounds asynchronously with the asyncio module. I've seen that Gbulb permits the use of the Gtk event loop with asyncio. I'll be thus receiving sound bites that i'll have to play right at the moment I receive them through my websocket client, sometimes overlapping each other when they're being played. This requires asynchronously mixing several audio files. Do you think this is feasible with the gst bindings? 

To give you an idea of what i'd expect, here's the portaudio implementation of what I need: https://github.com/loult-elte-fwere/termiloult/blob/master/tools/audiosink.py 

I can't find any documentation on this kind of usage of gstreamer, apart from the cold  PyGObject API reference.

Thanks in advance for your help,

Hadrien

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Olivier Crête-3]

Hello,

On Fri, 2018-08-24 at 19:37 +0200, Hadrien Titeux wrote:

So, here are what I'd like to know:

- Is it possible to play a .wav file without saving it to disk first? Like, directly from the byte file stored in the python memory (as portaudio can do it)

Totally possible, you can use "appsrc" instead of a filesrc and then just give it buffers.

- The websocket receives sounds asynchronously with the asyncio module. I've seen that Gbulb permits the use of the Gtk event loop with asyncio. I'll be thus receiving sound bites that i'll have to play right at the moment I receive them through my websocket client, sometimes overlapping each other when they're being played. This requires asynchronously mixing several audio files. Do you think this is feasible with the gst bindings? 

Yes, you would do that with multiple appsrcs, you want to make it a livesrc and set "do-timestamp" to true on them. Then mix them with the new "audiomixer", but make sure there is a "live" source connected when it starts. You should probably set all of your appsrc instances to be live.

To give you an idea of what i'd expect, here's the portaudio implementation of what I need: https://github.com/loult-elte-fwere/termiloult/blob/master/tools/audiosink.py 

I can't find any documentation on this kind of usage of gstreamer, apart from the cold  PyGObject API reference.

Your usecase is indeed a bit special, so I don't think you'll find any written documentation.

Olivier

https://twitter.com/Bodegacats_/status/1034042708074135553

Thanks in advance for your help,

Hadrien

_______________________________________________

gstreamer-devel mailing list

[hidden email]

https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Olivier Crête
[hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mathieu Duponchelle]

On 8/24/18 7:37 PM, Hadrien Titeux wrote:
> - The websocket receives sounds asynchronously with the asyncio module. I've seen that Gbulb permits the use of the Gtk event loop with asyncio. I'll be thus receiving sound bites that i'll have to play right at the moment I receive them through my websocket client, sometimes overlapping each other when they're being played. This requires asynchronously mixing several audio files. Do you think this is feasible with the gst bindings?

You can run a gstreamer pipeline without involving GMainLoop, by monitoring the bus
with GstBus.getpollfd().

Here [1] is an example pull request in a webrtcbin example, the
caveat being that this will only work with master currently, the PR will probably be merged
once the next release is out

[1]: https://github.com/centricular/gstwebrtc-demos/pull/34/files

--
Mathieu Duponchelle · https://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel