Strange problem with remuxing RTSP streams to Matroska

[Carlos Rafael Giani]

Hello, I am trying to remux a h.264/AAC stream that is transmitted over
RTSP to Matroska. So far, the result is a Matroska file with one
elementary stream, but not both. Sometimes, it contains h.264, sometimes
AAC, never both.

It seems as though somehow the stream starts after the first pad is
linked, and subsequent linking results in pads that aren't set up
properly. But I can't see what exactly is wrong.

This can be tested by using gst-rtsp-server's test-mp4 example. Pass an
mp4 file to it with one h.264 and one AAC stream. I tried the following
gst-launch line, and it works properly - the resulting Matroska data
contains both audio and video:


gst-launch-1.0 uridecodebin caps="video/x-h264, parsed=true; audio/mpeg"
uri="rtsp://127.0.0.1:8554/test" name=d   matroskamux name=e ! filesink
location=test.mkv  d. ! identity ! e.   d. ! aacparse ! e.


However, I wrote this example for doing it programmatically, and the
aforementioned error occurs - only one elementary stream is present in
the Matroska data. (I know the request pads are not released at the end,
I ignored this for now to keep the example small). So, what is the
difference? What is the GstParse functionality doing what I am not?
Anybody can spot anything I missed (especially in the pad_added_cb)?


Here is the test code:


#include <gst/gst.h>
#include <glib-unix.h>


static gboolean sigint(gpointer ptr)
{
     g_main_loop_quit((GMainLoop *)ptr);
     return TRUE;
}


GstElement *pipeline;


static void pad_added_cb(GstElement *decodebin, GstPad *pad, gpointer data)
{
     GstCaps *caps;
     GstStructure *str;
     gchar const *media_type;
     gchar const *req_pad_name;
     gchar const *parser_name = "";
     GstElement *mux;
     GstElement *parser;
     GstPad *req_pad;
     GstPad *parser_pad;

     if (GST_PAD_IS_LINKED(pad))
         return;

     mux = GST_ELEMENT_CAST(data);

     caps = gst_pad_query_caps(pad, NULL);
     str = gst_caps_get_structure(caps, 0);
     media_type = gst_structure_get_name(str);

     if (g_str_has_prefix(media_type, "video/x-h264"))
     {
         req_pad_name = "video_%u";
         parser_name = "identity";
     }
     else if (g_str_has_prefix(media_type, "audio/mpeg"))
     {
         req_pad_name = "audio_%u";
         parser_name = "aacparse";
     }
     else
         return;

     parser = gst_element_factory_make(parser_name, NULL);
     gst_bin_add(GST_BIN(pipeline), parser);
     gst_element_sync_state_with_parent(parser);

     parser_pad = gst_element_get_static_pad(parser, "sink");
     gst_pad_link(pad, parser_pad);
     gst_object_unref(GST_OBJECT(parser_pad));

     parser_pad = gst_element_get_static_pad(parser, "src");
     req_pad = gst_element_get_request_pad(mux, req_pad_name);
     gst_pad_link(parser_pad, req_pad);
     gst_object_unref(GST_OBJECT(parser_pad));
}


int main(int argc, char *argv[])
{
     GMainLoop *loop;
     GstCaps *dec_caps;
     GstElement *uridecodebin, *matroskamux, *filesink;

     gst_init(&argc, &argv);

     if (argc < 2)
         return -1;

     loop = g_main_loop_new(NULL, TRUE);
     g_unix_signal_add(SIGINT, sigint, loop);

     pipeline = gst_pipeline_new(NULL);
     uridecodebin = gst_element_factory_make("uridecodebin", NULL);
     matroskamux = gst_element_factory_make("matroskamux", NULL);
     filesink = gst_element_factory_make("filesink", NULL);

     gst_bin_add_many(GST_BIN(pipeline), uridecodebin, matroskamux,
filesink, NULL);

     gst_element_link(matroskamux, filesink);

     dec_caps = gst_caps_from_string("video/x-h264, parsed=true;
audio/mpeg");
     g_object_set(G_OBJECT(uridecodebin), "caps", dec_caps, "uri",
argv[1], NULL);

     g_object_set(G_OBJECT(filesink), "location", "test.mkv", NULL);

     g_signal_connect(G_OBJECT(uridecodebin), "pad-added",
G_CALLBACK(pad_added_cb), matroskamux);

     gst_element_set_state(pipeline, GST_STATE_PLAYING);

     g_main_loop_run(loop);

     gst_object_unref(GST_OBJECT(pipeline));

     return 0;
}

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Carlos Rafael Giani]

Partially solved it thanks to thaytan and slomo. The important
difference between the gst-launch-1.0 line and my code is that GstParse
(which is what drives gst-launch-1.0) looks at the pipeline description
and *first* request *both* video_%u and audio_%u request-pads. My code
doesn't - it requests on-demand in the pad-added callback. The result is
indeed that after linking the first pad, the stream starts, and the
second one can't be linked anymore.

Unfortunately, remuxing is apparently tricky to get right in general. In
my case, what can work is to block the pads in the pad-added callback
and then unblock them all in the no-more-pads callback, to make sure the
stream doesn't start until *all* are linked. In the general case, this
would not work if for example a subtitle stream starts later.



On 2017-07-09 20:44, Carlos Rafael Giani wrote:

> Hello, I am trying to remux a h.264/AAC stream that is transmitted
> over RTSP to Matroska. So far, the result is a Matroska file with one
> elementary stream, but not both. Sometimes, it contains h.264,
> sometimes AAC, never both.
>
> It seems as though somehow the stream starts after the first pad is
> linked, and subsequent linking results in pads that aren't set up
> properly. But I can't see what exactly is wrong.
>
> This can be tested by using gst-rtsp-server's test-mp4 example. Pass
> an mp4 file to it with one h.264 and one AAC stream. I tried the
> following gst-launch line, and it works properly - the resulting
> Matroska data contains both audio and video:
>
>
> gst-launch-1.0 uridecodebin caps="video/x-h264, parsed=true;
> audio/mpeg" uri="rtsp://127.0.0.1:8554/test" name=d   matroskamux
> name=e ! filesink location=test.mkv  d. ! identity ! e.   d. !
> aacparse ! e.
>
>
> However, I wrote this example for doing it programmatically, and the
> aforementioned error occurs - only one elementary stream is present in
> the Matroska data. (I know the request pads are not released at the
> end, I ignored this for now to keep the example small). So, what is
> the difference? What is the GstParse functionality doing what I am
> not? Anybody can spot anything I missed (especially in the pad_added_cb)?
>
>
> Here is the test code:
>
>
> #include <gst/gst.h>
> #include <glib-unix.h>
>
>
> static gboolean sigint(gpointer ptr)
> {
>     g_main_loop_quit((GMainLoop *)ptr);
>     return TRUE;
> }
>
>
> GstElement *pipeline;
>
>
> static void pad_added_cb(GstElement *decodebin, GstPad *pad, gpointer
> data)
> {
>     GstCaps *caps;
>     GstStructure *str;
>     gchar const *media_type;
>     gchar const *req_pad_name;
>     gchar const *parser_name = "";
>     GstElement *mux;
>     GstElement *parser;
>     GstPad *req_pad;
>     GstPad *parser_pad;
>
>     if (GST_PAD_IS_LINKED(pad))
>         return;
>
>     mux = GST_ELEMENT_CAST(data);
>
>     caps = gst_pad_query_caps(pad, NULL);
>     str = gst_caps_get_structure(caps, 0);
>     media_type = gst_structure_get_name(str);
>
>     if (g_str_has_prefix(media_type, "video/x-h264"))
>     {
>         req_pad_name = "video_%u";
>         parser_name = "identity";
>     }
>     else if (g_str_has_prefix(media_type, "audio/mpeg"))
>     {
>         req_pad_name = "audio_%u";
>         parser_name = "aacparse";
>     }
>     else
>         return;
>
>     parser = gst_element_factory_make(parser_name, NULL);
>     gst_bin_add(GST_BIN(pipeline), parser);
>     gst_element_sync_state_with_parent(parser);
>
>     parser_pad = gst_element_get_static_pad(parser, "sink");
>     gst_pad_link(pad, parser_pad);
>     gst_object_unref(GST_OBJECT(parser_pad));
>
>     parser_pad = gst_element_get_static_pad(parser, "src");
>     req_pad = gst_element_get_request_pad(mux, req_pad_name);
>     gst_pad_link(parser_pad, req_pad);
>     gst_object_unref(GST_OBJECT(parser_pad));
> }
>
>
> int main(int argc, char *argv[])
> {
>     GMainLoop *loop;
>     GstCaps *dec_caps;
>     GstElement *uridecodebin, *matroskamux, *filesink;
>
>     gst_init(&argc, &argv);
>
>     if (argc < 2)
>         return -1;
>
>     loop = g_main_loop_new(NULL, TRUE);
>     g_unix_signal_add(SIGINT, sigint, loop);
>
>     pipeline = gst_pipeline_new(NULL);
>     uridecodebin = gst_element_factory_make("uridecodebin", NULL);
>     matroskamux = gst_element_factory_make("matroskamux", NULL);
>     filesink = gst_element_factory_make("filesink", NULL);
>
>     gst_bin_add_many(GST_BIN(pipeline), uridecodebin, matroskamux,
> filesink, NULL);
>
>     gst_element_link(matroskamux, filesink);
>
>     dec_caps = gst_caps_from_string("video/x-h264, parsed=true;
> audio/mpeg");
>     g_object_set(G_OBJECT(uridecodebin), "caps", dec_caps, "uri",
> argv[1], NULL);
>
>     g_object_set(G_OBJECT(filesink), "location", "test.mkv", NULL);
>
>     g_signal_connect(G_OBJECT(uridecodebin), "pad-added",
> G_CALLBACK(pad_added_cb), matroskamux);
>
>     gst_element_set_state(pipeline, GST_STATE_PLAYING);
>
>     g_main_loop_run(loop);
>
>     gst_object_unref(GST_OBJECT(pipeline));
>
>     return 0;
> }
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel