Synchronised video capture from multiple cameras with splitmuxsink

[john-mark]

Hi,

I'm developing an application to capture video (no audio) from multiple cameras simultaneously, encode them, and write them out to segmented files. I was hoping I could describe the design here and what I think it's achieving, so that  you good folks could confirm or refute my reasoning.

My requirement is that each file contain all video for a given (wall clock) time bucket for a given camera. Say each time bucket is 5s wide. So for 2 cameras started at 12:00:00 exactly, I'd want files 12:00:00-camera1, 12:00:00-camera2, 12:00:05-camera1, 12:00:05-camera2, for the first two buckets.

Monospace diagram of pipeline:
```
                                 -----------------------------
camera1src, converters, etc. --> |                           | --> splitmuxsink
                                 | multiqueue                |
camera2src, converters, etc. --> | max-size-buffers=1        | --> splitmuxsink
                              ^  | sync-by-running-time=true |      ^
    .                         |  |                           |      |   .
    . etc.                    |                                     |   . etc.
    .                         |                                     |   .
                              |        split-at-running-time        |
                          pad probe --------------------------------`
```

Key points and questions

• By putting everything in a single pipeline, I'm assuming we get the *start* of capture synchronised (obviously if the frames are syncrhonised or not will depend on hardware solutions)?
• I've read multiqueue with max-size-buffers is how you synchronise multiple video streams, is this correct? Won't this cause dropped frames if any camera runs any faster than any other? How can I synchronise and protect against dropping frames? Do I need queues  on each pipeline before the multiqueue?
• I don't really understand the documentation for the sync-by-running-time option. Is this what I want?
• The docs suggest to call split-at-running-time from a pad probe to prevent race conditions. I think that by doing this from one (any arbitrary one) of the src pads before the multiqueue should be safe, and I can look at the PTS of the buffers to decide when to assign the next split times

I'm hoping this will work properly. My alternative design that I've been considering is to replace the splitmuxsinks with swapping out everything downstream from the multiqueue with new encoder -> filesink elements. But I feel like the design above is simpler to implement?

Thanks for looking, any comments on any of this are very much appreciated

John-Mark
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel