audio speech recognition and recording at the same time with tee
Locked
 
|
Hello all,
I am using gstreamer plugin for kaldi <https://github.com/alumae/gst-kaldi-nnet2-online> (speech recognition tool). This is the pipeline <https://github.com/alumae/gst-kaldi-nnet2-online/blob/master/demo/gui-demo.py> for kaldi plugi to do speech-to-text ( microphone input -> text transcript): self.pulsesrc.link(self.audioconvert) self.audioconvert.link(self.audioresample) self.audioresample.link(self.asr) self.asr.link(self.fakesink) Now, I'd like to record the audio from microphone into .ogg file at the same time. I made the following changes to the pipeline codes: # get audio from mic device self.pulsesrc.link(self.audioconvert1) self.audioconvert1.link(self.tee) # save to ogg self.audio_queue.link(self.vorbisenc) self.vorbisenc.link(self.oggmux) self.oggmux.link(self.filesink) # send to asr self.asr_queue.link(self.audioconvert) self.audioconvert.link(self.audioresample) self.audioresample.link(self.asr) self.asr.link(self.fakesink) # tied up tee and queues self.tee.link(self.audio_queue) self.tee.link(self.asr_queue) However, the pipeline doesn't work, There is no gst errors but audio is not saved and text doesn't come out. I suspect something is blocking the pipeline and I couldn't figure it out. Would like to hear suggestions ! -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: audio speech recognition and recording at the same time with tee
|
|
Le lun. 4 nov. 2019 15 h 40, kyle111 <[hidden email]> a écrit : Hello all, Check your return values. Something must be failing. The error is returned by GstPad.link() call directly, and may not appear on the bus.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Nicolas Dufresne-5 wrote
> Le lun. 4 nov. 2019 15 h 40, kyle111 < > purehavoc77@ > > a écrit : > >> Hello all, >> >> I am using gstreamer plugin for kaldi >> <https://github.com/alumae/gst-kaldi-nnet2-online> (speech >> recognition >> tool). This is the pipeline >> < >> https://github.com/alumae/gst-kaldi-nnet2-online/blob/master/demo/gui-demo.py> >> >> for kaldi plugi to do speech-to-text ( microphone input -> text >> transcript): >> >> self.pulsesrc.link(self.audioconvert) >> self.audioconvert.link(self.audioresample) >> self.audioresample.link(self.asr) >> self.asr.link(self.fakesink) >> >> >> Now, I'd like to record the audio from microphone into .ogg file at the >> same >> time. I made the following changes to the pipeline codes: >> >> # get audio from mic device >> self.pulsesrc.link(self.audioconvert1) >> self.audioconvert1.link(self.tee) >> >> # save to ogg >> self.audio_queue.link(self.vorbisenc) >> self.vorbisenc.link(self.oggmux) >> self.oggmux.link(self.filesink) >> >> # send to asr >> self.asr_queue.link(self.audioconvert) >> self.audioconvert.link(self.audioresample) >> self.audioresample.link(self.asr) >> self.asr.link(self.fakesink) >> >> # tied up tee and queues >> self.tee.link(self.audio_queue) >> self.tee.link(self.asr_queue) >> >> However, the pipeline doesn't work, There is no gst errors but audio is >> not >> saved and text doesn't come out. >> I suspect something is blocking the pipeline and I couldn't figure it >> out. >> Would like to hear suggestions ! >> > > Check your return values. Something must be failing. The error is returned > by GstPad.link() call directly, and may not appear on the bus. Hello Nicolas, thanks again for lending a hand. Not sure what you mean by 'check return values'. You mean I should return the above pipelin w/ gst-launch in terminal instead of python lib? I could try that out. so I found 1 interesting thing. > self.tee.link(self.audio_queue) > self.tee.link(self.asr_queue) whenever, I disable of the one of 'tee' links to queue, the other queue works (can record audio or can get speech-2-text ) but not both at the same time. -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: audio speech recognition and recording at the same time with tee
|
|
Le jeu. 7 nov. 2019 03 h 40, kyle111 <[hidden email]> a écrit : Nicolas Dufresne-5 wrote No, I simply mean that function like link() have a return value, in this case true or false, false would mean that the link failed.
If links didn't fail, check the queue level, maybe one end accumulate more then the queue capacity, preventing preroll. You may also chose to disable preroll using async property you let's say the speech to text sink.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |