GStreamer-devel

Gstreamer audio mixer issue

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
dananjaya
Reply | Threaded
Open this post in threaded view
♦
♦
|

Gstreamer audio mixer issue

dananjaya
Hi,
I am trying the below pipeline to capture 2 1080p usb video streams while
previewing, with 2 usb audio sources mixed together.

The issue is, the audio mixed and captured is containing lots of artifacts
such as click sounds. How can I resolve this issue.

gst-launch-1.0 -e mp4mux name=mux ! filesink location=feed1mp1080HS.mp4
v4l2src device=/dev/video1 ! tee name=t1 t1. ! queue ! video/x-raw,
width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! nvv4l2h264enc
maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse ! queue !
mux.video_0 audiomixer name=mixer ! audioconvert ! voaacenc ! aacparse !
mux.audio_0 pulsesrc
device="alsa_input.usb-VXIS_Inc_ezcap_U3_capture-02.analog-stereo" ! queue !
audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue ! mixer.sink_0
pulsesrc
device="alsa_input.usb-C-Media_Electronics_Inc._USB_Advanced_Audio_Device-00.analog-stereo"
! queue ! audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue !
mixer.sink_1 t1. ! queue ! video/x-raw, width=1920, height=1080,
framerate=30/1 !  nvvidconv ! queue !
"video/x-raw(memory:NVMM),width=959,height=540,framerate=30/1,format=NV12" !
queue ! nvoverlaysink overlay-x=0 overlay-y=270 overlay-w=959 overlay-h=540
overlay=1 v4l2src device=/dev/video2 ! tee name=t2 t2. ! queue !
video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse
! queue ! mp4mux ! filesink location=feed2mp1080HS.mp4 t2. ! queue !
video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
"video/x-raw(memory:NVMM),width=960,height=540,framerate=30/1,format=NV12" !
queue ! nvoverlaysink overlay-x=960 overlay-y=270 overlay-w=960
overlay-h=540 overlay=2

Thanks in advance



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Guillaume Cartier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Gstreamer audio mixer issue

Guillaume Cartier
Hi dananjaya, we are also using GStreamer for a project that involves mixing multiple live sources together.

For us, the audiomixer would very often generate what we called "gritches" that upon looking at the output in Audacity were revealed to be small silence gaps in the output. Reading the source code and playing with it has led to the gst_aggregator_pad_has_space function that has a test that I believe is incorrect (in all humbleness being fairly new to GStreamer) :

if (self->priv->latency == 0)
  return FALSE;

I think this test should only apply in the non-live case, but in any case the good news is that by setting latency to any value in our audiomixer element (even 1), this has solved most of our problems.

PS: You could also replace audiomixer with liveadder which is a subclass of audiomixer that by default sets latency to 30ms.

On Tue, Jan 5, 2021 at 6:51 AM dananjaya <[hidden email]> wrote:
Hi,
I am trying the below pipeline to capture 2 1080p usb video streams while
previewing, with 2 usb audio sources mixed together.

The issue is, the audio mixed and captured is containing lots of artifacts
such as click sounds. How can I resolve this issue.

gst-launch-1.0 -e mp4mux name=mux ! filesink location=feed1mp1080HS.mp4
v4l2src device=/dev/video1 ! tee name=t1 t1. ! queue ! video/x-raw,
width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! nvv4l2h264enc
maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse ! queue !
mux.video_0 audiomixer name=mixer ! audioconvert ! voaacenc ! aacparse !
mux.audio_0 pulsesrc
device="alsa_input.usb-VXIS_Inc_ezcap_U3_capture-02.analog-stereo" ! queue !
audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue ! mixer.sink_0
pulsesrc
device="alsa_input.usb-C-Media_Electronics_Inc._USB_Advanced_Audio_Device-00.analog-stereo"
! queue ! audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue !
mixer.sink_1 t1. ! queue ! video/x-raw, width=1920, height=1080,
framerate=30/1 !  nvvidconv ! queue !
"video/x-raw(memory:NVMM),width=959,height=540,framerate=30/1,format=NV12" !
queue ! nvoverlaysink overlay-x=0 overlay-y=270 overlay-w=959 overlay-h=540
overlay=1 v4l2src device=/dev/video2 ! tee name=t2 t2. ! queue !
video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse
! queue ! mp4mux ! filesink location=feed2mp1080HS.mp4 t2. ! queue !
video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
"video/x-raw(memory:NVMM),width=960,height=540,framerate=30/1,format=NV12" !
queue ! nvoverlaysink overlay-x=960 overlay-y=270 overlay-w=960
overlay-h=540 overlay=2

Thanks in advance



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
jim nualart
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Gstreamer audio mixer issue

jim nualart
In reply to this post by dananjaya
I faced similar issues a while back. A search in the group for something like "audiomixer gaps" should pull up the related posts.

If memory serves, I found that using "adder" instead of "audiomixer" eliminated the gaps. As did setting a non-zero latency in the audiomixer properties. However, in my case, the latency value only seemed to have any effect at all if I also placed a queue after the audiomixer.

 

Message: 1
Date: Tue, 5 Jan 2021 09:37:25 -0500
From: Guillaume Cartier <[hidden email]>
To: Discussion of the development of and with GStreamer
        <[hidden email]>
Subject: Re: Gstreamer audio mixer issue
Message-ID:
        <CAN4e_LmOgJ6_q4yQOSaaL2jD_LZi80hjM=PJR1rC=[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi dananjaya, we are also using GStreamer for a project that involves
mixing multiple live sources together.

For us, the audiomixer would very often generate what we called "gritches"
that upon looking at the output in Audacity were revealed to be small
silence gaps in the output. Reading the source code and playing with it has
led to the gst_aggregator_pad_has_space function that has a test that I
believe is incorrect (in all humbleness being fairly new to GStreamer) :

if (self->priv->latency == 0)
  return FALSE;

I think this test should only apply in the non-live case, but in any case
the good news is that by setting latency to any value in our audiomixer
element (even 1), this has solved most of our problems.

PS: You could also replace audiomixer with liveadder which is a subclass of
audiomixer that by default sets latency to 30ms.

On Tue, Jan 5, 2021 at 6:51 AM dananjaya <[hidden email]> wrote:

> Hi,
> I am trying the below pipeline to capture 2 1080p usb video streams while
> previewing, with 2 usb audio sources mixed together.
>
> The issue is, the audio mixed and captured is containing lots of artifacts
> such as click sounds. How can I resolve this issue.
>
> gst-launch-1.0 -e mp4mux name=mux ! filesink location=feed1mp1080HS.mp4
> v4l2src device=/dev/video1 ! tee name=t1 t1. ! queue ! video/x-raw,
> width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! nvv4l2h264enc
> maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse ! queue !
> mux.video_0 audiomixer name=mixer ! audioconvert ! voaacenc ! aacparse !
> mux.audio_0 pulsesrc
> device="alsa_input.usb-VXIS_Inc_ezcap_U3_capture-02.analog-stereo" ! queue
> !
> audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue ! mixer.sink_0
> pulsesrc
>
> device="alsa_input.usb-C-Media_Electronics_Inc._USB_Advanced_Audio_Device-00.analog-stereo"
> ! queue ! audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue !
> mixer.sink_1 t1. ! queue ! video/x-raw, width=1920, height=1080,
> framerate=30/1 !  nvvidconv ! queue !
> "video/x-raw(memory:NVMM),width=959,height=540,framerate=30/1,format=NV12"
> !
> queue ! nvoverlaysink overlay-x=0 overlay-y=270 overlay-w=959 overlay-h=540
> overlay=1 v4l2src device=/dev/video2 ! tee name=t2 t2. ! queue !
> video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
> nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! queue !
> h264parse
> ! queue ! mp4mux ! filesink location=feed2mp1080HS.mp4 t2. ! queue !
> video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue !
> "video/x-raw(memory:NVMM),width=960,height=540,framerate=30/1,format=NV12"
> !
> queue ! nvoverlaysink overlay-x=960 overlay-y=270 overlay-w=960
> overlay-h=540 overlay=2
>
> Thanks in advance
>
>
>


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
dananjaya
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Gstreamer audio mixer issue

dananjaya
Placing a queue after audiomixer solved the issue for me. Appreciate your
help. Thank you





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page