gstreamer 1.14 h.264 timestamping

[dplane2720]

I inherited a project which requires the use of timestamps (machine rtc timecode, not sps or pps) to be encoded with each video frame. We use this video stream as part of a robotics platform and need these timestamps to be accurate to the time at which they are recorded - and not received by our main ROS server.

The previous solution involved a custom plugin which encoded the frameID and pc time and added it as a textbuffer into the gst_buffer_map as well as into a subtitle track.

However, the gstreamer pipeline splits the video:
Branch 1 - video is encoded as h.264 and sent via rtppay264 over the network.
Branch 2 - video is encoded as h.265, synced with the subtitle track, and saved locally as a .mp4

The problem I am having is I can not parse the timestamp from the buffer map, nor can I get the subtitle to stream over rtp.

Compounding this  problem is that this video stream is a critical system which can not have any downtime, but the video is being streamed from a system which desperately needs to be updated (Jetson Xavier running Ubuntu 18.04, with Gstreamer 1.14.x.).

So far, I have tried:
1. Utilizing GSCam, a gstreamer pluggin for ROS, but I still need the timecode for the ROS message timestamps.
1. duplicating the plugin with modifications (unable to find a suitable solution with 1.14 tools),
2. combining the subtitle track to a separate rtp stream (pipeline fails to properly combine elements with the timestamp plugin and spins out),
3. parsing the raw gst_buffer_map (data should be encoded as UTF-8, but the plugin writes the string to memory as raw data before pappending it to the buffer map)
4. duplicating the network setup and trying to use gstreamer 1.6.x API calls to create the proper timestamp

All of these attempts have failed, so I am hoping the developer forum can help! Any suggestions/ opinions/ ideas are welcome!

Here is the pipeline code:

#!/bin/bash
now=$(date +"%Y-%m-%d-%H-%M-%S")
echo "${now}"

export GST_PLUGIN_PATH=/usr/local/lib/aarch64-linux-gnu/gstreamer-1.0
export DISPLAY=:0
dest_ip=201.7.90.18
tablet_ip=201.7.90.149
port=5603
tablet_port=5703

gst-launch-1.0 v4l2src ! 'video/x-raw,format=YUY2,framerate=60/1,width=2880,height=1860' ! timestamp name=demux \
demux.textsrc ! queue ! "text/x-raw, format=utf8" ! mux.subtitle_0 \
demux.videosrc ! queue ! nvvidconv flip-method=0 ! 'video/x-raw(memory:NVMM),format=I420' ! tee name=s ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=2880, height=1860, format=(string)I420' ! nvv4l2h264enc maxperf-enable=true bitrate=1500000 insert-sps-pps=true insert-vui=false insert-aud=false ! 'video/x-h264,stream-format=(string)byte-stream' ! queue ! rtph264pay mtu=1400 ! udpsink host=${dest_ip} port=${port} auto-multicast=false sync=false async=false \
s. ! nvv4l2h265enc maxperf-enable=true bitrate=80000000 iframeinterval=1 insert-aud=false insert-vui=false insert-sps-pps=true ! 'video/x-h265,stream-format=(string)byte-stream' ! h265parse ! mux.video splitmuxsink name=mux max-size-time=120000000000 async-finalize=true async-handling=true muxer=qtmux muxer-properties="properties,reserved-max-duration=250000000000,reserved-moov-update-period=100000000" use-robust-muxing=true location=/mnt/logs/hdr-0/hdr-0-${now}-%04d.mp4 -e