Best way to indicate loss of live camera signal

[allanc]

I have an application that creates a pipeline of the form "rtspsrc location=rtsp://user:[hidden email]/chY/Z latency=0 retry=65535 timeout=0 tcp-timeout=0 drop-on-latency=True ! rtph264depay ! h264parse ! imxvpudec ! imxipuvideosink window-x-coord=0 window-y-coord=0 window-width=1024 window-height=768 sync=false". This is done with gst_parse_launch.

 

So, obviously, this is getting low latency streaming data from an IP camera.

 

However, I'm looking for a way to detect loss of signal from the camera and ensure it gets indicated somehow to the viewer. I also need to recover reasonably quickly and re-establish the stream if the camera comes back online.

 

=====

 

I actually have two issues here. The first has to do with how to easily switch between a live stream and a "signal lost" stream. The thoughts I've had about this come down to a few options:

 

1/ Have a text overlay on the stream which is usually invisible but, when the signal disappears, this is changed to something like "signal lost", or a big red "X" in the middle of the screen. This may be doable but it means we have to put pango into our embedded system to do text rendering. I'm also not sure of the process of finding and manipulating the textoverlay component when all I gave is the pipeline returned from gst_parse_launch().

 

2/ On loss of signal, shut down the current stream and simply run an equivalent "videotestsrc pattern=snow ! imxvpuvideosink ...". This is slightly problematic in that I no longer have the camera stream to detect whether it starts playing again. That means I'll have to poll it periodically (maybe to a fakesink) and, if it works, replace the test pattern with the proper stream.

 

I'd appreciate any guidance from others who have already done this sort of thing, or indeed ANY ideas on how best to approach this.

 

=====

 

The second issue has to do specifically with message capture, in that GStreamer doesn't appear to be calling my callbacks.

 

Keeping in mind this is a Qt application (no Gtk windowing stuff), I first call gst_init() from the main thread then start up a separate thread to do all the GStreamer stuff. Within that thread, I basically do the following:

 

GstElement *playBin = gst_parse_launch("<gstreamer pipeline>");

GstBus *bus = gst_pipeline_get_bus(GST_PIPELINE(playBin));

gst_bus_add_signal_watch(bus);

g_signal_connect(bus, "message::state-changed", (GCallback)Cb, NULL);

gst_element_set_state(playBin, GST_STATE_PLAYING);

 

GMainLoop *mainLoop = g_main_loop_new(NULL, FALSE);

g_main_loop_run(mainLoop);

 

The callback function simply outputs some debug messages at the moment:

 

static gboolean Cb(GstBus *bus, GstMessage *msg, gpointer data) {

    std::cout << "g_sig, bus = " << (void*)bus

              << ", msg = "      << (void*)msg

              << ", data = "     << (void*)data

              << std::endl;

    return TRUE;

}

 

Now, despite the fact I *think* I'm following the rules for message capture, nothing is output by the callback function so I'm assuming it's not being called. I've also tried it with the eos, error and element messages with similar results, and by using the more general gst_bus_add_watch() which I believe will fire for all messages.

 

I'd be grateful for any assistance as to why the callback is apparently being ignored.

 

Cheers.

 

Allan Chandler | Software Engineer

DTI Group Ltd | Transit Security & Surveillance

31 Affleck Road, Perth Airport, WA 6105, AU

P | F | [hidden email]

Visit our website www.dti.com.au

The information contained in this email is confidential. If you receive this email in error, please inform DTI Group Ltd via the above contact details. If you are not the intended recipient, you may not use or disclose the information contained in this email or attachments.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Tue, 2016-11-08 at 02:46 +0000, Allan Chandler wrote:
> [...]
> I'd appreciate any guidance from others who have already done this
> sort of thing, or indeed ANY ideas on how best to approach this.

How do you detect signal loss currently? What exactly happens here? The
RTSP connection shuts down? No more RTP packets for a long time?

> =====
>  
> The second issue has to do specifically with message capture, in that
> GStreamer doesn't appear to be calling my callbacks.
>  
> Keeping in mind this is a Qt application (no Gtk windowing stuff), I
> first call gst_init() from the main thread then start up a separate
> thread to do all the GStreamer stuff. Within that thread, I basically
> do the following:

You'll have to create a new GMainContext, let the GMainLoop use that
one and especially attach the GstBus watch to that very GMainContext.
Then it should work like that.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[allanc]

Thanks for that.

> How do you detect signal loss currently? What exactly happens here? The RTSP connection shuts down? No more RTP packets for a long time?

Well, I don't detect it. My understanding was that data loss would show up as an event of some description such as the state changing from playing to paused, or a buffering < X% message. I may well be wrong in that assumption, hence my question.

> You'll have to create a new GMainContext, let the GMainLoop use that one and especially attach the GstBus watch to that very GMainContext. Then it should work like that.

I didn't think it was actually REQUIRED to have a separate context if you're running in another thread. From what I'd read, you can set up a different context if you want to separate your threads with different message pumps but I only have the one of my threads running GLib/Gst. Hence, I don't actually need the separation from my point of view. The GStreamer stuff was put into a separate thread so both Qt and it could run their own infinite message pumps, but there's only one GLib pump.

On the off-chance doing gst_init in one thread then everything else in another was causing a problem, I moved the init into the same thread. But no luck.

However, you MAY be right since it appears that all the code setting up the callback is working, it's just that the callback isn't being called, suggesting there may be a disconnect somewhere.

... suitable pause while I check, so I don't look stupid to the entire mailing list :-) ...

And it appears you ARE actually right.  When I set up a new context within the thread thusly (before any other actions):
    m_context = g_main_context_new();
    g_main_context_push_thread_default(m_context);
    m_mainLoop = g_main_loop_new(m_context, FALSE);
then the messages from the callback start to magically appear.

Now that I can see the messages coming in, I can do some analysis as to what ones appear when the camera feed disappears.

Much appreciation, Sebastian,
Cheers,
Al.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Wed, 2016-11-09 at 02:38 +0000, Allan Chandler wrote:

> Thanks for that.
>
> > How do you detect signal loss currently? What exactly happens here?
> > The RTSP connection shuts down? No more RTP packets for a long
> > time?
>
> Well, I don't detect it. My understanding was that data loss would
> show up as an event of some description such as the state changing
> from playing to paused, or a buffering < X% message. I may well be
> wrong in that assumption, hence my question.

It depends on what happens there. That can happen, it can also happen
that it just gets no media for a while and then you don't get any
error.

Look at the watchdog element for detecting that case.

> > You'll have to create a new GMainContext, let the GMainLoop use
> > that one and especially attach the GstBus watch to that very
> > GMainContext. Then it should work like that.
>
> I didn't think it was actually REQUIRED to have a separate context if
> you're running in another thread. From what I'd read, you can set up
> a different context if you want to separate your threads with
> different message pumps but I only have the one of my threads running
> GLib/Gst. Hence, I don't actually need the separation from my point
> of view. The GStreamer stuff was put into a separate thread so both
> Qt and it could run their own infinite message pumps, but there's
> only one GLib pump.

Yes you can use the default main context, however that will cause
problems if you ever run with a Qt that is configured to use the GLib
main context for its event loop... as then you have Qt already running
a main loop on the default main context.

> [...]
> Now that I can see the messages coming in, I can do some analysis as
> to what ones appear when the camera feed disappears.

Great, good that it works now :)

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel