I want to design a source element which can share video frames.

[Psychesnet Hsieh]

Hi Everyone,

I'm a newbie on developing gstreamer elements. We have a camera device and we can access the video frame via ioctl from SDK's function.

We would like to design a source element to share video frames to any services on the device, is it possible?

For now, we have a daemon(capture) which always does ioctl and access video frames from the driver. My plan are

1. Create a plugin/source element, every service can create source element to ask capture deliver video frames. Is it possible? How?

2. Create a shared memory map on capture and write a plugin/source element. Every service can create a source element to access a shared memory map to ask frames. 

    I think this is possible. Any existing examples?

3. Another better suggestion?

Please give me some advice. Thanks. Regards

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mandar Joshi]

Hi,

I've done something similar recently. Could you please go through this discussion https://lists.freedesktop.org/archives/gstreamer-devel/2020-July/075481.html Since this is recent, I should be able to answer any queries you have about the approach.

For your first question, this is my suggestion

So, create a source element based on GstPushSrc (like neonhttpsrc) and in the _create function, the following should work.

Map the buffer and write what you need to it. Make sure you set the SRC CAPS of the element to the correct ones so that a right element to decode/view the video data can be linked to the source element.

For your second question, I don't think I completely understand it. But maybe the answer to the first question will allow you to come up with the answer to the second one.

Feel free to ask any questions.

Regards

Mandar Joshi

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le vendredi 17 juillet 2020 à 08:34 +0800, Psychesnet Hsieh a écrit :

> Hi Everyone,
>
> I'm a newbie on developing gstreamer elements. We have a camera device and we
> can access the video frame via ioctl from SDK's function.
> We would like to design a source element to share video frames to any services
> on the device, is it possible?
> For now, we have a daemon(capture) which always does ioctl and access video
> frames from the driver. My plan are
> 1. Create a plugin/source element, every service can create source element to
> ask capture deliver video frames. Is it possible? How?
> 2. Create a shared memory map on capture and write a plugin/source element.
> Every service can create a source element to access a shared memory map to ask
> frames.
>     I think this is possible. Any existing examples?
> 3. Another better suggestion?

You also expose your camera through PipeWire (it already support V4L2 ioctl
interface). Then pipewiresrc in gstreamer (as provided by the PipeWire project).
Just an idea, I'm not certain I fully understand the goal here.

>
> Please give me some advice. Thanks. Regards
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Psychesnet Hsieh]

Hi Joshi,

Thanks. First of all, I think I need to describe my question more and we can fully address what I need. Thanks.

CAPTURE  -> mmap -> service 1

                             | ----> service 2

                             | ----> service 3

Above layout, it is our design for now. We have CAPTURE which always capture H264 video frames from DSP(via special ioctl, not v4l2).

There are a lot of server X can access H264 video frames at same time. The middle interface is mmap. It is working now.

Now, we would like to design a source element and it can satisfy following requirements.

1. All service X still can access H264 video frames at same time.

2. Gstreamer can work with this source element.

3. CAPTURE, for platform, we still need this part which capture H264 video frames via ioctl.

@Joshi,

According to your reply, I think I still need to keep my mmap and create a source element which can access mmap and delivery data through whole pipeline, right?

Thanks. Have a good day. ^^

Mandar Joshi <[hidden email]> 於 2020年7月17日 星期五寫道：

Hi,

I've done something similar recently. Could you please go through this discussion https://lists.freedesktop.org/archives/gstreamer-devel/2020-July/075481.html Since this is recent, I should be able to answer any queries you have about the approach.

For your first question, this is my suggestion

So, create a source element based on GstPushSrc (like neonhttpsrc) and in the _create function, the following should work.

Map the buffer and write what you need to it. Make sure you set the SRC CAPS of the element to the correct ones so that a right element to decode/view the video data can be linked to the source element.

For your second question, I don't think I completely understand it. But maybe the answer to the first question will allow you to come up with the answer to the second one.

Feel free to ask any questions.

Regards

Mandar Joshi

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le sam. 18 juill. 2020 10 h 00, Psychesnet Hsieh <[hidden email]> a écrit :

Hi Joshi,

Thanks. First of all, I think I need to describe my question more and we can fully address what I need. Thanks.

CAPTURE  -> mmap -> service 1

                             | ----> service 2

                             | ----> service 3

Above layout, it is our design for now. We have CAPTURE which always capture H264 video frames from DSP(via special ioctl, not v4l2).

There are a lot of server X can access H264 video frames at same time. The middle interface is mmap. It is working now.

Now, we would like to design a source element and it can satisfy following requirements.

1. All service X still can access H264 video frames at same time.

In this case, services are separate process ?

2. Gstreamer can work with this source element.

3. CAPTURE, for platform, we still need this part which capture H264 video frames via ioctl.

@Joshi,

According to your reply, I think I still need to keep my mmap and create a source element which can access mmap and delivery data through whole pipeline, right?

Thanks. Have a good day. ^^

Mandar Joshi <[hidden email]> 於 2020年7月17日 星期五寫道：

Hi,

I've done something similar recently. Could you please go through this discussion https://lists.freedesktop.org/archives/gstreamer-devel/2020-July/075481.html Since this is recent, I should be able to answer any queries you have about the approach.

For your first question, this is my suggestion

So, create a source element based on GstPushSrc (like neonhttpsrc) and in the _create function, the following should work.

Map the buffer and write what you need to it. Make sure you set the SRC CAPS of the element to the correct ones so that a right element to decode/view the video data can be linked to the source element.

For your second question, I don't think I completely understand it. But maybe the answer to the first question will allow you to come up with the answer to the second one.

Feel free to ask any questions.

Regards

Mandar Joshi

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Maersk-Moller-2]

Hi

You can do something like this:

     gst-launch-1.0 -v videotestsrc ! queue ! x264enc ! queue ! shmsink socket-path=/tmp/video-ctr wait-for-connection=0

The "videotestsrc ! queue ! x264enc" is here the stand-in for your capture device. And shmsink is your stand-in for mmap.

Then the clients would be something like this:

    gst-launch-1.0 -v shmsrc socket-path=/tmp/pmm ! identity silent=0 ! h264parse ! queue ! decodebin ! queue ! videoconvert ! autovideosink

The client is just an example. You need to consider shm size, security, optimization and a lot more.

Best regards

Peter Maersk-Moller

On Sat, Jul 18, 2020 at 2:55 PM Psychesnet Hsieh <[hidden email]> wrote:

Hi Joshi,

Thanks. First of all, I think I need to describe my question more and we can fully address what I need. Thanks.

CAPTURE  -> mmap -> service 1

                             | ----> service 2

                             | ----> service 3

Above layout, it is our design for now. We have CAPTURE which always capture H264 video frames from DSP(via special ioctl, not v4l2).

There are a lot of server X can access H264 video frames at same time. The middle interface is mmap. It is working now.

Now, we would like to design a source element and it can satisfy following requirements.

1. All service X still can access H264 video frames at same time.

2. Gstreamer can work with this source element.

3. CAPTURE, for platform, we still need this part which capture H264 video frames via ioctl.

@Joshi,

According to your reply, I think I still need to keep my mmap and create a source element which can access mmap and delivery data through whole pipeline, right?

Thanks. Have a good day. ^^

Mandar Joshi <[hidden email]> 於 2020年7月17日 星期五寫道：

Hi,

I've done something similar recently. Could you please go through this discussion https://lists.freedesktop.org/archives/gstreamer-devel/2020-July/075481.html Since this is recent, I should be able to answer any queries you have about the approach.

For your first question, this is my suggestion

So, create a source element based on GstPushSrc (like neonhttpsrc) and in the _create function, the following should work.

Map the buffer and write what you need to it. Make sure you set the SRC CAPS of the element to the correct ones so that a right element to decode/view the video data can be linked to the source element.

For your second question, I don't think I completely understand it. But maybe the answer to the first question will allow you to come up with the answer to the second one.

Feel free to ask any questions.

Regards

Mandar Joshi

Czar Softech

https://www.czarsoftech.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Psychesnet Hsieh]

Hi All,

Thanks. Looks like I need to implement a source element like shmsink/shmsrc.

@Nicolas,

Yes, services are separate process.

Peter Maersk-Moller <[hidden email]> 於 2020年7月18日 週六 下午11:57寫道：

Hi

You can do something like this:

     gst-launch-1.0 -v videotestsrc ! queue ! x264enc ! queue ! shmsink socket-path=/tmp/video-ctr wait-for-connection=0

The "videotestsrc ! queue ! x264enc" is here the stand-in for your capture device. And shmsink is your stand-in for mmap.

Then the clients would be something like this:

    gst-launch-1.0 -v shmsrc socket-path=/tmp/pmm ! identity silent=0 ! h264parse ! queue ! decodebin ! queue ! videoconvert ! autovideosink

The client is just an example. You need to consider shm size, security, optimization and a lot more.

Best regards

Peter Maersk-Moller

On Sat, Jul 18, 2020 at 2:55 PM Psychesnet Hsieh <[hidden email]> wrote:

Hi Joshi,

Thanks. First of all, I think I need to describe my question more and we can fully address what I need. Thanks.

CAPTURE  -> mmap -> service 1

                             | ----> service 2

                             | ----> service 3

Above layout, it is our design for now. We have CAPTURE which always capture H264 video frames from DSP(via special ioctl, not v4l2).

There are a lot of server X can access H264 video frames at same time. The middle interface is mmap. It is working now.

Now, we would like to design a source element and it can satisfy following requirements.

1. All service X still can access H264 video frames at same time.

2. Gstreamer can work with this source element.

3. CAPTURE, for platform, we still need this part which capture H264 video frames via ioctl.

@Joshi,

According to your reply, I think I still need to keep my mmap and create a source element which can access mmap and delivery data through whole pipeline, right?

Thanks. Have a good day. ^^

Mandar Joshi <[hidden email]> 於 2020年7月17日 星期五寫道：

Hi,

I've done something similar recently. Could you please go through this discussion https://lists.freedesktop.org/archives/gstreamer-devel/2020-July/075481.html Since this is recent, I should be able to answer any queries you have about the approach.

For your first question, this is my suggestion

So, create a source element based on GstPushSrc (like neonhttpsrc) and in the _create function, the following should work.

Map the buffer and write what you need to it. Make sure you set the SRC CAPS of the element to the correct ones so that a right element to decode/view the video data can be linked to the source element.

For your second question, I don't think I completely understand it. But maybe the answer to the first question will allow you to come up with the answer to the second one.

Feel free to ask any questions.

Regards

Mandar Joshi

Czar Softech

https://www.czarsoftech.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel