Ref Interpipe caps negociation.

[Frederic Turmel]

Question for interpipe expert. Should interpipe be able to deal with incoming stream change?

If I really specify the entire caps it works but when I leave the caps more open on the receiving end It never picks up the right match and it just stop.

Example logs below:

0:00:03.460903000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)1/1, format=(string)YUY2
0:00:03.460912000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)1/1
0:00:03.460921000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:513:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Listeners caps updated
0:00:03.461096000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:838:gst_adv_video_src_start:<advvideosrc> start
0:00:03.641902000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:858:gst_adv_video_src_start:<advvideosrc> Capture is already started. No need to start
0:00:03.641918000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:860:gst_adv_video_src_start: Successfully started capture
0:00:03.641942000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:889:gst_adv_video_src_start: Updating Caps to: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.642030000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:898:gst_adv_video_src_start: gst_adv_video_src_start Exit!
0:00:03.645274000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:390:gst_inter_pipe_sink_intersect_listener_caps:<webrtc_video_src> Listener intervideosrc caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.645299000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:444:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Caps negotiated: EMPTY
0:00:03.645306000  1910 0x5564ee97d450 ERROR          interpipesink gstinterpipesink.c:448:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Failed to obtain an intersection between upstream elements and listeners
0:00:03.645314000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:856:gst_inter_pipe_sink_remove_listener:<webrtc_video_src> Removing listener intervideosrc
0:00:03.647752000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.648435000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649131000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649153000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649167000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.649173000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: (NULL)

The receiving side has this pipeline for a webrtc application. toggling allow-renegotiation does not really help. 

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2 ! queue leaky=1 ! videorate max-rate=30 average-period=2 ! videoscale ! video/x-raw,width=1280,height=720 ! queue ! videoconvert ! video/x-raw,format=I420 ! queue ! x264enc bitrate=1200 speed-preset=veryfast tune=zerolatency key-int-max=60 ! rtph264pay pt=96 ssrc=2 ! application/x-rtp,media=video,encoding-name=H264,payload=96

With the issue above it will work if specify I fully the receiving end like below:

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=60000/1001 !

Is this the expected behavior? My source framerate can change so interpipe may not work if everything has to be defined in a static way.

Thanks

FredT

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Gruner]

Hi,

Yes, interpipes should allow you to renegotiate the caps if you set the “allow-renegotiation=true”. Let me test this on my side and get back to you. Meanwhile, as a workaround, can you try leaving the framerate out of the interpipesrc caps?

Do you have more that one interpipesrc listening to the interpipesink?

Michael

On Jun 18, 2020, at 18:05, Frederic Turmel <[hidden email]> wrote:

Question for interpipe expert. Should interpipe be able to deal with incoming stream change?

If I really specify the entire caps it works but when I leave the caps more open on the receiving end It never picks up the right match and it just stop.

Example logs below:

0:00:03.460903000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)1/1, format=(string)YUY2
0:00:03.460912000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)1/1
0:00:03.460921000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:513:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Listeners caps updated
0:00:03.461096000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:838:gst_adv_video_src_start:<advvideosrc> start
0:00:03.641902000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:858:gst_adv_video_src_start:<advvideosrc> Capture is already started. No need to start
0:00:03.641918000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:860:gst_adv_video_src_start: Successfully started capture
0:00:03.641942000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:889:gst_adv_video_src_start: Updating Caps to: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.642030000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:898:gst_adv_video_src_start: gst_adv_video_src_start Exit!
0:00:03.645274000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:390:gst_inter_pipe_sink_intersect_listener_caps:<webrtc_video_src> Listener intervideosrc caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.645299000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:444:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Caps negotiated: EMPTY
0:00:03.645306000  1910 0x5564ee97d450 ERROR          interpipesink gstinterpipesink.c:448:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Failed to obtain an intersection between upstream elements and listeners
0:00:03.645314000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:856:gst_inter_pipe_sink_remove_listener:<webrtc_video_src> Removing listener intervideosrc
0:00:03.647752000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.648435000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649131000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649153000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649167000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.649173000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: (NULL)

The receiving side has this pipeline for a webrtc application. toggling allow-renegotiation does not really help. 

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2 ! queue leaky=1 ! videorate max-rate=30 average-period=2 ! videoscale ! video/x-raw,width=1280,height=720 ! queue ! videoconvert ! video/x-raw,format=I420 ! queue ! x264enc bitrate=1200 speed-preset=veryfast tune=zerolatency key-int-max=60 ! rtph264pay pt=96 ssrc=2 ! application/x-rtp,media=video,encoding-name=H264,payload=96

With the issue above it will work if specify I fully the receiving end like below:

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=60000/1001 !

Is this the expected behavior? My source framerate can change so interpipe may not work if everything has to be defined in a static way.

Thanks

FredT

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Frederic Turmel]

Only one receiver. If I remove the frame rate, I get the error pasted earlier.

On Thu, Jun 18, 2020, 5:10 PM Michael Gruner <[hidden email]> wrote:

Hi,

Yes, interpipes should allow you to renegotiate the caps if you set the “allow-renegotiation=true”. Let me test this on my side and get back to you. Meanwhile, as a workaround, can you try leaving the framerate out of the interpipesrc caps?

Do you have more that one interpipesrc listening to the interpipesink?

Michael

On Jun 18, 2020, at 18:05, Frederic Turmel <[hidden email]> wrote:

Question for interpipe expert. Should interpipe be able to deal with incoming stream change?

If I really specify the entire caps it works but when I leave the caps more open on the receiving end It never picks up the right match and it just stop.

Example logs below:

0:00:03.460903000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)1/1, format=(string)YUY2
0:00:03.460912000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)1/1
0:00:03.460921000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:513:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Listeners caps updated
0:00:03.461096000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:838:gst_adv_video_src_start:<advvideosrc> start
0:00:03.641902000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:858:gst_adv_video_src_start:<advvideosrc> Capture is already started. No need to start
0:00:03.641918000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:860:gst_adv_video_src_start: Successfully started capture
0:00:03.641942000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:889:gst_adv_video_src_start: Updating Caps to: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.642030000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:898:gst_adv_video_src_start: gst_adv_video_src_start Exit!
0:00:03.645274000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:390:gst_inter_pipe_sink_intersect_listener_caps:<webrtc_video_src> Listener intervideosrc caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.645299000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:444:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Caps negotiated: EMPTY
0:00:03.645306000  1910 0x5564ee97d450 ERROR          interpipesink gstinterpipesink.c:448:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Failed to obtain an intersection between upstream elements and listeners
0:00:03.645314000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:856:gst_inter_pipe_sink_remove_listener:<webrtc_video_src> Removing listener intervideosrc
0:00:03.647752000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.648435000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649131000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649153000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649167000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.649173000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: (NULL)

The receiving side has this pipeline for a webrtc application. toggling allow-renegotiation does not really help. 

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2 ! queue leaky=1 ! videorate max-rate=30 average-period=2 ! videoscale ! video/x-raw,width=1280,height=720 ! queue ! videoconvert ! video/x-raw,format=I420 ! queue ! x264enc bitrate=1200 speed-preset=veryfast tune=zerolatency key-int-max=60 ! rtph264pay pt=96 ssrc=2 ! application/x-rtp,media=video,encoding-name=H264,payload=96

With the issue above it will work if specify I fully the receiving end like below:

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=60000/1001 !

Is this the expected behavior? My source framerate can change so interpipe may not work if everything has to be defined in a static way.

Thanks

FredT

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Frederic Turmel]

I think I figured out the problem. My source element was not prescriptive enough for the video format. I was not forcing interlace-mode.  Which was causing problems during caps negotiation with all my downstream elements. It is working fine now. 

Thanks

FredT

On Thu, Jun 18, 2020 at 5:53 PM Frederic Turmel <[hidden email]> wrote:

Only one receiver. If I remove the frame rate, I get the error pasted earlier.

On Thu, Jun 18, 2020, 5:10 PM Michael Gruner <[hidden email]> wrote:

Hi,

Yes, interpipes should allow you to renegotiate the caps if you set the “allow-renegotiation=true”. Let me test this on my side and get back to you. Meanwhile, as a workaround, can you try leaving the framerate out of the interpipesrc caps?

Do you have more that one interpipesrc listening to the interpipesink?

Michael

On Jun 18, 2020, at 18:05, Frederic Turmel <[hidden email]> wrote:

Question for interpipe expert. Should interpipe be able to deal with incoming stream change?

If I really specify the entire caps it works but when I leave the caps more open on the receiving end It never picks up the right match and it just stop.

Example logs below:

0:00:03.460903000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)1/1, format=(string)YUY2
0:00:03.460912000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)1/1
0:00:03.460921000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:513:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Listeners caps updated
0:00:03.461096000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:838:gst_adv_video_src_start:<advvideosrc> start
0:00:03.641902000  1910 0x5564ee97d370 DEBUG            advvideosrc gstadvvideosrc.cpp:858:gst_adv_video_src_start:<advvideosrc> Capture is already started. No need to start
0:00:03.641918000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:860:gst_adv_video_src_start: Successfully started capture
0:00:03.641942000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:889:gst_adv_video_src_start: Updating Caps to: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.642030000  1910 0x5564ee97d370 INFO             advvideosrc gstadvvideosrc.cpp:898:gst_adv_video_src_start: gst_adv_video_src_start Exit!
0:00:03.645274000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:390:gst_inter_pipe_sink_intersect_listener_caps:<webrtc_video_src> Listener intervideosrc caps: video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.645299000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:444:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Caps negotiated: EMPTY
0:00:03.645306000  1910 0x5564ee97d450 ERROR          interpipesink gstinterpipesink.c:448:gst_inter_pipe_sink_get_caps:<webrtc_video_src> Failed to obtain an intersection between upstream elements and listeners
0:00:03.645314000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:856:gst_inter_pipe_sink_remove_listener:<webrtc_video_src> Removing listener intervideosrc
0:00:03.647752000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.648435000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649131000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649153000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:420:gst_inter_pipe_sink_get_caps:<webrtc_video_src> No listeners yet, accepting any caps
0:00:03.649167000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:492:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Incoming Caps: video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60000/1001, format=(string)YUY2
0:00:03.649173000  1910 0x5564ee97d450 INFO           interpipesink gstinterpipesink.c:494:gst_inter_pipe_sink_set_caps:<webrtc_video_src> Negotiated Caps: (NULL)

The receiving side has this pipeline for a webrtc application. toggling allow-renegotiation does not really help. 

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2 ! queue leaky=1 ! videorate max-rate=30 average-period=2 ! videoscale ! video/x-raw,width=1280,height=720 ! queue ! videoconvert ! video/x-raw,format=I420 ! queue ! x264enc bitrate=1200 speed-preset=veryfast tune=zerolatency key-int-max=60 ! rtph264pay pt=96 ssrc=2 ! application/x-rtp,media=video,encoding-name=H264,payload=96

With the issue above it will work if specify I fully the receiving end like below:

interpipesrc name=intervideosrc stream-sync=1 listen-to=webrtc_video_src allow-renegotiation=1 format=3 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=60000/1001 !

Is this the expected behavior? My source framerate can change so interpipe may not work if everything has to be defined in a static way.

Thanks

FredT

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel