gst_parse_launch vs programatic webrtc implementation

[eadan]

Hi,
I'm trying to figure out why the following works reliably, but the programmatic (not sure of the correct terminology but see below) version works, but not reliably.

This is an excerpt from a modified webrtc-unidirectional-h264

      gst_parse_launch ("webrtcbin name=webrtcbin stun-server=stun://"
      STUN_SERVER " "
      " shmsrc socket-path=/tmp/shmsink.h264 is-live=true do-timestamp=true ! queue max-size-buffers=1 ! h264parse ! "
      "rtph264pay config-interval=-1 aggregate-mode=zero-latency ! "
      "application/x-rtp,media=video,encoding-name=H264,payload="
      RTP_PAYLOAD_TYPE " ! queue max-size-time=100000000 ! webrtcbin. ", &error);

With this i can refresh the browser page over and over and it will always establish a video stream.

In our application we need to be able to dynamically add and remove streams from an ongoing session, I have reproduced the bad behaviour with another modified  webrtc-unidirectional-h264 where the above becomes :

  receiver_entry->pipeline = gst_pipeline_new ("webrtc-pipeline");
  receiver_entry->webrtcbin = gst_element_factory_make ("webrtcbin", NULL);
  receiver_entry->shmsrc = gst_element_factory_make ("shmsrc", NULL);
  receiver_entry->queue1 = gst_element_factory_make ("queue", NULL);
  receiver_entry->h264parse = gst_element_factory_make ("h264parse", NULL);
  receiver_entry->rtph264pay = gst_element_factory_make ("rtph264pay", NULL);
  receiver_entry->queue2 = gst_element_factory_make ("queue", NULL);

  // Check all elements
  if(!(receiver_entry->pipeline && receiver_entry->webrtcbin && receiver_entry->shmsrc && receiver_entry->queue1 && receiver_entry->h264parse &&
      receiver_entry->rtph264pay && receiver_entry->queue2)) {
    g_error("Failed to create one or more elemenets.");
  }

  // Add webrtc element to the pipeline
  gst_bin_add (GST_BIN (receiver_entry->pipeline), receiver_entry->webrtcbin);

  // Configure webrtcbin
  g_object_set(receiver_entry->webrtcbin, "bundle-policy", 3, NULL);  // max bundle
  g_object_set(receiver_entry->webrtcbin, "async_handling", 1, NULL);
  g_object_set(receiver_entry->webrtcbin, "stun-server", "stun://" STUN_SERVER, NULL);

  // connect signal handlers
  g_signal_connect (receiver_entry->webrtcbin, "on-negotiation-needed",
      G_CALLBACK (on_negotiation_needed_cb), (gpointer) receiver_entry);

  g_signal_connect (receiver_entry->webrtcbin, "on-ice-candidate",
      G_CALLBACK (on_ice_candidate_cb), (gpointer) receiver_entry);

  // Set pipeline ready
  if (gst_element_set_state (receiver_entry->pipeline, GST_STATE_READY) ==
      GST_STATE_CHANGE_FAILURE)
    g_error ("Could not set pipeline to ready state.");

  bus = gst_pipeline_get_bus (GST_PIPELINE (receiver_entry->pipeline));
  gst_bus_add_watch (bus, bus_watch_cb, NULL);
  gst_object_unref (bus);

  // Start pipeline
  if (gst_element_set_state (receiver_entry->pipeline, GST_STATE_PLAYING) ==
      GST_STATE_CHANGE_FAILURE)
    g_error ("Could not start pipeline");

  // Add remaining elements to the pipeline
  gst_bin_add_many (GST_BIN (receiver_entry->pipeline), receiver_entry->shmsrc,
      receiver_entry->queue1, receiver_entry->h264parse, receiver_entry->rtph264pay, receiver_entry->queue2, NULL);

  // Configure shmsrc
  g_object_set (receiver_entry->shmsrc, "socket-path", "/tmp/shmsink.h264", NULL);
  g_object_set (receiver_entry->shmsrc, "is-live", 1, NULL);
  g_object_set (receiver_entry->shmsrc, "do-timestamp", 1, NULL);

  // Configure queue1
  g_object_set(receiver_entry->queue1,"max-size-buffers", 1, NULL);
  g_object_set(receiver_entry->queue1,"leaky", 2 /* downstream - drop old buffers */, NULL);

  // Link shmsrc to queue1
  if(!gst_element_link(receiver_entry->shmsrc, receiver_entry->queue1))
    g_error("Failed to link shmsrc");

  // Configure rtph264pay
  g_object_set(receiver_entry->rtph264pay, "config-interval", -1, NULL);
  g_object_set(receiver_entry->rtph264pay, "aggregate-mode", 1 /* "zero-latency" */, NULL);

  // Link queue1 to h264parse
  if(!gst_element_link(receiver_entry->queue1, receiver_entry->h264parse))
    g_error("Failed to link queue1");

  // Link h264parse to rtph264pay
  if(!gst_element_link(receiver_entry->h264parse, receiver_entry->rtph264pay))
    g_error("Failed to link h264parse");

  // Link rtph264pay to queue2
  if(!gst_element_link(receiver_entry->rtph264pay, receiver_entry->queue2))
    g_error("Failed to link rtph264pay");

  // Configure queue2
  g_object_set(receiver_entry->queue1,"max-size-time", 100000000, NULL);

  // Create video caps
  GstCaps *videocaps = gst_caps_from_string("application/x-rtp,media=video,encoding-name=H264,payload=96");
  if(!videocaps)
    g_error("Failed creating video caps");

  // Sync element states
  if(!gst_element_sync_state_with_parent(receiver_entry->shmsrc))
    g_error("gst_element_sync_state_with_parent failed for shmsrc.");

  if(!gst_element_sync_state_with_parent(receiver_entry->queue1))
    g_error("gst_element_sync_state_with_parent failed for queue1.");

  if(!gst_element_sync_state_with_parent(receiver_entry->h264parse))
    g_error("gst_element_sync_state_with_parent failed for h264parse.");

  if(!gst_element_sync_state_with_parent(receiver_entry->rtph264pay))
    g_error("gst_element_sync_state_with_parent failed for rtph264pay.");

  if(!gst_element_sync_state_with_parent(receiver_entry->queue2))
    g_error("gst_element_sync_state_with_parent failed for queue2.");

  // Link queue2 element to webrtcbin
  if(!gst_element_link_filtered(receiver_entry->queue2, receiver_entry->webrtcbin, videocaps))
    g_error("Failed to link queue2");

With this version the video stream sometimes does not start when loading the page, in fact it does not get as far as "Creating negotiation offer". refresh and it will often then work, but not always.

Somehow in our real application there is also some different behaviour whereby we sometimes get an error such as:

0:00:23.074054192 14054   0x7f5400e460 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<shmsrc0> error: Internal data stream error.
0:00:23.074170134 14054   0x7f5400e460 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<shmsrc0> error: streaming stopped, reason not-linked (-1)


Another point of note is that our application works more reliably if the webrtc-unidirectional-h264 is currently running and browser connected. In fact our application hardly ever succeeds in setting up a video stream if webrtc-unidirectional-h264 is not also running.

Is anybody able to shed any light on any of this?

[eadan]

Setting the property wait-for-connection to false on the shmsink seems to help with this.