Similar as udpsrc but to pass RTP packets directly to gstreamer

[bojan74]

My already working main application not using gstreamer receives RTP packets and do some work.

Now I need to implement additional functionality where above incoming RTP packets have to be saved also as mp4 file.
At least for "hello world" test I successfully implemented solution in the following way:
1. Main application (beside current work) send incoming RTP packets also to some port using udp socket
2. gstreamer application read those packets with udpsrc element
3. Then pass those packets further on pipeline and I get mp4 file
I have to emphasize that sending and receiving of RTP packets inside points 1 and 2 are local (not between different computers).

Does exist such gstreamer element that I could pass RTP packets directly to it without need to use udpsrc and socket sending in my main application?

I mean in the way that my gstreamer application would be written as DLL used by main application where I will just call something like writeRTPData(handler, rtpData, rtpDataLen) that will pass RTP packet to gstreamer element with same content as it is read currently with udpsrc.

So in this way I will not need to open two sockets (in my application and udpsrc) for each incoming stream just to pass that RTP data from my application to gstreamer part.

A few year ago I was working with mediastreamer2 and there I was able to register/pass own transport type to rtprecv filter to implement something similar as I was asking above.

[Tim Müller]

On Wed, 2017-01-18 at 10:17 -0800, bojan74 wrote:

Hi,

If this is all within the same application process, then you can use
the appsrc element to inject RTP/UDP packets into a GStreamer pipeline.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[bojan74]

Thank you very much!

It looks exactly what I was looking for.

[bojan74]

I will continue with this thread to not open new one.
As it was suggested to me I changed udpsrc with appsrc.

In both versions (using udpsrc or appsrc) I am using same input content, difference is that with:
- udpsrc version - I am sending content to some port and udpsrc read it and send further ... what ends with filesink where I get mp4 sile
- appsrc version - I am feeding pipeline directly with g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret) where I pack content in gst_buffer

In both cases I get mp4 file with similar size (+-1kB) so it looks that whole content comes to end. And I can play both files.

But with appsrc version everything is ok till about 55% of file but then playing of the mp4 file jump to the end. It looks like 45% of video is deleted or more probably have some problem with headers cause size of both files is similar and also length in seconds displayed in mp4 player.
In other words if video is 10s length, everything is ok till about 6s, after that playing jump directly to 10s (end).

First I though that maybe is problem cause I am not feeding pipeline enough fast but I didn't notice any change if I am calling g_signal_emit_by_name faster. I also didn't notice any change if I am feeding with more content, for example if length of video is 1 minute. Also in this case about first 35s will be ok, but then will jump till end.
I will understand that some queues could be full but then I guess they will be full at same absolute time no matter how long is input. Also if packets will be dropped then size of file at the end will probably be shorter?
At this case it would be easier to me to investigate where packets are lost, but in my situation it looks like some timing headers are not correct. But again what gets wrong after first 55% of file.

Any idea where is the problem, what to check to ...?
Thank you

[bojan74]

I must add additional info and corrections for what I wrote before.

Now I was testing will full h264 video that is input for my pipeline and I saw:
- both mp4 files (with udpsrc and with appsrc) contain full video, so nothing is lost
- udpsrc video length is 3:27 and it normally plays till end (time marker slider is going from start till end and also seeking is possible anywhere)
- appsrc video length is 2:15 but full video ends at 1:30. It is played 2-3 times faster then with udpsrc and so ends at 1:30. And I don't know from where gets 2:15 for length info. If I click between 1:30 and 2:15 in player marker jump to start.

To repeat again my whole pipeline is like this:
udpsrc/appsrc -> rtpjitterbuffer -> rtph264depay -> h264parse -> mp4mux -> filesink

appsrc caps are in both versions:
application/x-rtp, media=(string)video, clock-rate = (int)90000, encoding-name = (string)H264

and also other settings are mostly same.

But it looks that there are some problem with time. Currently I don't set anything with
          GST_BUFFER_PTS (buffer) =
          GST_BUFFER_DURATION (buffer) =

Is it necessary to use this in case of my pipeline?
If yes, what do I need to set here if I am feeding appsrc with RTP packets?
To use RTP timestamp fro GST_BUFFER_PST and to calculate duration from RTP header timestamp for GST_BUFFER_DURATION?


Additional info:
For test now I just add as each packet is 50ms despite it is not exactly like this but result is almost same as with udpsrc. Now I have length of video about 3:15 and marker move from start till end.

        GST_BUFFER_PTS(buffer) = timestamp ;
        GST_BUFFER_DURATION(buffer) = 50000000 ;
        timestamp += 50000000 ;

Instead of using 50ms should I decode RTP header timestamp and use that value?
Or what would be best practise?

[Tim Müller]

On Sun, 2017-01-22 at 06:04 -0800, bojan74 wrote:

Hi,

> But it looks that there are some problem with time. Currently I don't
> set
> anything with 
>   GST_BUFFER_PTS (buffer) = 
>   GST_BUFFER_DURATION (buffer) = 
>
> Is it necessary to use this in case of my pipeline?
> If yes, what do I need to set here if I am feeding appsrc with RTP
> packets? 
> To use RTP timestamp fro GST_BUFFER_PST and to calculate duration
> from RTP
> header timestamp for GST_BUFFER_DURATION?

You don't need to set GST_BUFFER_DURATION, but you probably want to
either set GST_BUFFER_PTS yourself (if you're feeding data from an
artificial source that doesn't operate in realtime), or set

  appsrc do-timestamp=true

Also make sure to set appsrc to format=time via

g_object_set (appsrc, "format", GST_FORMAT_TIME, NULL);

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mikl]

Hello, Tim

Sorry for jumping n between.

I have side question.

I did not found description of "do-timestamp" paeameter of appsrc

Can you give more details on it please?

Mikl

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[surprised]

Hi,Tim.

I saw a post you answered in 2017 about timestamp,and I have some similar
problem about timestamps.So can you help me?

I just want to achieve this:

At sender, First,I obtain YUV data via appsrc. I want to write an ID into
each frame of YUV(before encoding), and then I encode,and then I transmit
them into Local Area Network via rtph265pay and udpsink.

At receiver, I receive them via udpsrc and rtph265depay,then I decode and
get YUV data via appsink.

My main purpose is: At receiver,I can get the ID of each YUV data so that I
can have the knowledge that if there exists a phenomenon of lossing data
when transmitting by checking the continuous ID numbers.

Then I thought of  a method: Using timestamp maybe can achieve my purpose.So
I tried.

Now, I set "do-timestamp" as FALSE in appsrc, and I found that udpsrc also
has the "do-timestamp" property,so I also set "do-timestamp" as FALSE in
udpsrc. The rest of operations is as same as I set in the previous
posts(In appsrc, I set timestamp like this: GST_BUFFER_PTS(buffer)=100;
g_signal_emit_by_name(appsrc,"push_buffer",buffer,&ret);
In appsink,I get the timestamp like this: timestamp=GST_BUFFER_PTS(buffer);
).
Then I output the timestamp in appsink,but it doesn't equal to 100 which I
set in appsrc.

Is there something else I don't realize? Please help me find the problem.
Or if I set the timestamp in GstRtpBuffer(in this way,I can set timestamp as
frame ID and I can get this timestamp before decoding.),how can I do?





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel