Setting timestamps on filesrc pipeline

[bomba]

Hello,

I've got a bluetooth MP3 encoder, it sends data through serial port, say /dev/ttyUSB0.

I can successfully play "live" data with the following pipeline:
filesrc location=/dev/ttyUSB0 !  mpegaudioparse ! flump3dec ! autoaudiosink

the resulting playback it's somewhere near 1.5 seconds later than the live event, which should be normal
considering the capture, encoding and sending process before the serial communication.

Now, I want to combine this audio stream with some video coming from my laptop's webcam:

filesrc location=/dev/ttyUSB0 !  mpegaudioparse ! flump3dec ! autoaudiosink
v4l2src ! video/x-raw,width=320,height=240,framerate=30/1 ! videoconvert ! autovideosink

the audio branch won't play anything, unless I add "sync=false" to the audio sink element.
I assume this behaviour is tied to the missing/late audio timestamps.

To make this work, I have to calculate the exact audio encoding and transmitting time and correct timestamps near filesrc element.

How can I do that?

[Arjen Veenhuizen]

What you could do is set do-timestamp=true on both v4l2src and filesrc. Furthermore, i suggest you insert a queue before both sinks.

[bomba]

Arjen Veenhuizen wrote

set do-timestamp=true on both v4l2src and filesrc.

Putting timestamps on v4l2src makes perfect sense, it is a LIVE device, so timestamps would be correctly applied by the source element.

Putting do-timpestamps on filesrc instead doesn't make sense, because that source file is already 1.5 seconds late, I would mark as "now" something that happened "1.5 seconds ago".

Of course queues on sinks would help reproducing smoothly, but won't fix the issue.

[Arjen Veenhuizen]

AFAIK MP3 does not contain any timestamps (except from the option to embed a timecode in an extended ID3 tag). You could opt for muxing your MP3 stream in an MPEG2TS so that timestamps at added at the sending side. At the receiving side, you can use these timestamps to synchronize your webcam.

A more pragmatic approach would be to set render-delay on pulsesink to 1.5 seconds or add a queue and set its min-* and max-* properties to add 1.5 seconds delay.

[bomba]

I don't have any control on the encoding workflow. All I've got is a serial port, filled with encoded MP3 data.
Then, I've to adjust the buffer timestamps somehow.

I've managed getting near the result by adding a queue with a minimum threshold setting on the video pipeline.
Still very raw and unstable

[Nicolas Dufresne-5]

Le vendredi 16 juin 2017 à 03:39 -0700, bomba a écrit :

> Arjen Veenhuizen wrote
> > set do-timestamp=true on both v4l2src and filesrc.
>
> Putting timestamps on v4l2src makes perfect sense, it is a LIVE device, so
> timestamps would be correctly applied by the source element.
>
> Putting do-timpestamps on filesrc instead doesn't make sense, because that
> source file is already 1.5 seconds late, I would mark as "now" something
> that happened "1.5 seconds ago".
>
> Of course queues on sinks would help reproducing smoothly, but won't fix the
> issue.
>

Just to clarify, there is no valid use case for setting do-timestamp
either on v4l2src or filesrc. On v4l2src, setting do-timestamp will
make the timestamp worst then the one produced by the element (derived
from HW timestamp). On filesrc, as the stream segment is of type BYTES,
the timestamp would normally be ignored anyway since timestamp are
always in time. You'll find valid use case of do-timestamp when using
udpsrc.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le vendredi 16 juin 2017 à 08:47 -0700, bomba a écrit :
> I don't have any control on the encoding workflow. All I've got is a serial
> port, filled with encoded MP3 data.
> Then, I've to adjust the buffer timestamps somehow

Well, let the mpegaudioparse element do that for you. Pipeline will be
non-live, but I believe this is correct for your scenario. Your serial
port should be seen just like filesrc, and assumed to be fast enough.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel