Threading in callbacks

[moritz.vieli]

Hi there

I set up an app sink for MIDI files to be played in sync with audio files
and it all works fine. However, when I put some CPU load on the system and
add many MIDI files, the following error appears often and audio playback
stutters:
0:00:43.604328917  3140 0x34d6b350 WARN                    alsa
gstalsasink.c:983:xrun_recovery:<sink0> xrun recovery -32: Broken pipe

The problem seems to be the MIDI app sink, or more general, callback
processing from Gstreamer. When I add a level meter and process the produced
messages in my host, the same errors appear.

I am using a Java host and built it exactly on the example from the repo
gstreamer-java. It seems not to be different from the C examples in the
documentation. I also tried wrapping the processing in own threads, added
queues on different places but nothing worked. Somehow it seems like my host
is interrupting the Gstreamer pipe, but I have no idea why and how I could
work around it. A little delay for the processing on my side is no problem,
but audio should not interrupt.

Any help is highly welcome! :)

Best,
Moritz





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[moritz.vieli]

I tried pulling the buffers from a completely different thread instead of
getting called but still the same problem. It seems like *getting* the MIDI
buffers from Gstreamer interrupts the audio streaming. Only registering the
callback and doing in it what I want, is no problem. But as soon as I get
the buffers, Gstreamer has issues under high load. Is there somehow a
bottleneck here?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Sun, 2018-12-30 at 16:55 -0600, moritz.vieli wrote:

Hi Moritz,

> I tried pulling the buffers from a completely different thread
> instead of getting called but still the same problem. It seems like
> *getting* the MIDI buffers from Gstreamer interrupts the audio
> streaming. Only registering the callback and doing in it what I want,
> is no problem. But as soon as I get the buffers, Gstreamer has issues
> under high load. Is there somehow a bottleneck here?

It would be useful if you could post/describe your pipeline and perhaps
even make available your code or a small sample program that
demonstrates the issue you're seeing.

Cheers
 Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[moritz.vieli]

Hi Tim

Thanks a lot for your fast reply!

The pipe looks like this:
[uridecodebin] -> [audioconvert] -|
[uridecodebin] -> [audioconvert] -> [audiomixer] -> [queue] -> [alsasink]
[uridecodebin] -> [audioconvert] -|
...
[filesrc] -> [midiparse] -> [queue] -> [appsink]
[filesrc] -> [midiparse] -> [queue] -> [appsink]
...

The AppSink already produces the errors with this code:
/Sample sample = element.pullSample();
logger.info(sample.getBuffer().map(false),
midiCompositionFile.getMidiRoutingList());
sample.dispose();
return FlowReturn.OK;/

There is no problem, if the AppSink does not read the buffers from the
samples:
/Sample sample = element.pullSample();
sample.dispose();
return FlowReturn.OK;/

The more MIDI events there are, the more ALSA xruns I get. However, already
small amounts of MIDI events, like e.g. 5 per second already introduce the
problem. But some events may be thrown at exactly the same time. Maybe this
is a problem?

Best,
Moritz





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[moritz.vieli]

The whole code is available here (lines 329 to 361 should be interesting):
https://github.com/moritzvieli/rocketshow/blob/a6cb508c1e08c083357e4bbae12a9521890d219a/src/main/java/com/ascargon/rocketshow/composition/CompositionPlayer.java

But it's in Java and still a work-in-progress. I am getting these errors on
a Raspberry Pi, which makes it not that easy to reproduce. But I'll check.
Just let me know, if you need more information.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[moritz.vieli]

I just made another test and pulled all samples from a different thread so I
could leave the app sinks property "emit-signals" to false. This works much
better although it should not in my opinion. According to the documentation,
also the callbacks are in own threads and should not block the streaming
thread. But maybe it's also something different, I really have no clue...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel