Hi all,
I have read that since the 1.6 version of gstreamer, frame tags are preserved by each plugin. My questions below are targeted at memory- and frame-size- independent tags, such as gps data, exposure, gain and so forth. - Is there a refcount accessible when defining a new metadata type, to avoid that one branch after a 'tee' destroys a metadata that's still used by the other branch ? - Is the preservation mechanism implemented outside of the plugins, or must it be implemented in each plugin ? In that case, is a generic preservation code available, that we could add e.g. to gstreamer-imx (imx hardware accelerated) plugins ? Thanks in advance Philippe -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Hi all,
I have read that since the 1.6 version of gstreamer, frame tags are preserved by each plugin. My questions below are targeted at memory- and frame-size- independent tags, such as gps data, exposure, gain that could vary from frame to frame, and so on. - Is the preservation mechanism implemented outside of the plugins, or must it be implemented in each plugin ? In that case, is a generic preservation code available, that we could add e.g. to gstreamer-imx (imx hardware accelerated) plugins ? - Is there a refcount accessible when defining the free function for a new metadata type, to avoid that one branch after a 'tee' destroys a metadata that's still used by the other branch ? Thanks in advance Philippe -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Hi all,
I have read that since the 1.6 version of gstreamer frame tags are preserved by each plugin. My questions below are targeted at the propagation by GstVideoEncoders of memory- and frame-size- independent tags, such as gps data, exposure, gain that could vary from frame to frame, and so on. I have the following pipeline : gst-launch-1.0 imxv4l2videosrc ! tee name=t ! queue ! \ imxvpuenc_mjpeg ! mprintmeta logname=1 ! fakesink i\ t. ! queue ! imxvpuenc_h264 ! mprintmeta logname=2 ! fakesink where imxv4l2videosrc has been modified to add custom metadata, and mprintmeta is debugging to show those metadata. I notice that the metadata are not present after the encoders - Is the propagation mechanism implemented outside of the plugins, or must it be implemented in each plugin ? In that case, is a generic propagation code available, that we could add e.g. to gstreamer-imx (imx hardware accelerated) encoder plugins ? - Is GstMetaFreeFunction only called after all instances are unused, or is there a refcount accessible inside the definition of GstMetaFreeFunction for a new metadata type, to avoid that one branch after a 'tee' destroys a metadata that's still used by the other branch ? Thanks in advance Philippe -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
I am not able to find the code implementing that specific encoder on the fly so I will try and work with whtever assumptions I can make about encoders in general and your post. You will need to make sure that your metadata implements a transformation function, the encoder implements a transformation function and that the tags of the metadata are correctly set for the specific transformation. Assuming that the imxvpuenc_mjpeg inherits the videoencoder class and does not override the transform_meta function, you will need to make sure that the meta tags include "video" and do not include "memory"Dimitrios On Thu, Jan 19, 2017 at 12:47 AM, Philippe De Muyter <[hidden email]> wrote: Hi all, _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
And i forgot to answer your specific questions, sorry about that :p 1) There is a default propagation mechanism implemented in every plugin. It usually goes by the name "transform_meta". You should not have to implement it assuming that elements inherit from the base classes. What you do need to implement is a transformation function within your metadata. On Thu, Jan 19, 2017 at 9:33 AM, Dimitrios Katsaros <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Dimitrios Katsaros
Hi Dimitrios,
thanks for your answer On Thu, Jan 19, 2017 at 09:33:23AM +0100, Dimitrios Katsaros wrote: > I am not able to find the code implementing that specific encoder on the > fly so I will try and work with whtever assumptions I can make about > encoders in general and your post. > > You will need to make sure that your metadata implements a transformation > function, the encoder implements a transformation function and that the > tags of the metadata are correctly set for the specific transformation. > Assuming that the imxvpuenc_mjpeg inherits the videoencoder class and does > not override the transform_meta function, you will need to make sure that > the meta tags include "video" and do *not* include "memory" The imxvpuenc_* encoders (https://github.com/Freescale/gstreamer-imx.git) inherit from GstVideoEncoderClass and do not provide their own transform_meta function. Our metadata provide a meta_transform function as last parameter to gst_meta_register, and I have now added a "video" tag. There is no "memory" tag. But our metadata meta_transform function is not called, while I can see that our meta_free function is well called. If I add a mprintmeta before the encoders, I can see our metadata being there, actually only one instance of our metadata shared by the two branches of our pipeline; but our metadata are still lost after the encoders. Philippe > > > Dimitrios > > On Thu, Jan 19, 2017 at 12:47 AM, Philippe De Muyter <[hidden email]> wrote: > > > Hi all, > > > > I have read that since the 1.6 version of gstreamer frame tags are > > preserved > > by each plugin. My questions below are targeted at the propagation by > > GstVideoEncoders of memory- and frame-size- independent tags, such as gps > > data, > > exposure, gain that could vary from frame to frame, and so on. > > > > I have the following pipeline : > > > > gst-launch-1.0 imxv4l2videosrc ! tee name=t ! queue ! \ > > imxvpuenc_mjpeg ! mprintmeta logname=1 ! fakesink i\ > > t. ! queue ! imxvpuenc_h264 ! mprintmeta logname=2 ! fakesink > > > > where imxv4l2videosrc has been modified to add custom metadata, and > > mprintmeta is debugging to show those metadata. I notice that the > > metadata are not present after the encoders > > > > - Is the propagation mechanism implemented outside of the plugins, or must > > it be implemented in each plugin ? In that case, is a generic propagation > > code > > available, that we could add e.g. to gstreamer-imx (imx hardware > > accelerated) > > encoder plugins ? > > > > - Is GstMetaFreeFunction only called after all instances are unused, or is > > there a refcount accessible inside the definition of GstMetaFreeFunction > > for a new metadata type, to avoid that one branch after a 'tee' destroys a > > metadata that's still used by the other branch ? > > > > Thanks in advance > > > > Philippe > > > > -- > > Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 > > Bruxelles gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Dimitrios Katsaros
On Thu, Jan 19, 2017 at 09:41:35AM +0100, Dimitrios Katsaros wrote:
> And i forgot to answer your specific questions, sorry about that :p > > 1) There is a default propagation mechanism implemented in every plugin. It > usually goes by the name "transform_meta". You should not have to implement > it assuming that elements inherit from the base classes. What you do need > to implement is a transformation function within your metadata. > > 2) metadata lifetime is linked with the buffer it is associated with. When > tee propagates a buffer it increases the buffers reference count for every > branch. So you should not have to worry about the reference counting for > the metadata. After the 'tee' I have only one instance of our metadata with a refcount set to 2. After the videoencoder, in the specific case where the metadata are invariant per frame, is that instance merely attached to the encoded buffer or must it be copied by the transformation function of our metadata ? Philippe > > DImitrios > > On Thu, Jan 19, 2017 at 9:33 AM, Dimitrios Katsaros <[hidden email]> > wrote: > > > I am not able to find the code implementing that specific encoder on the > > fly so I will try and work with whtever assumptions I can make about > > encoders in general and your post. > > > > You will need to make sure that your metadata implements a transformation > > function, the encoder implements a transformation function and that the > > tags of the metadata are correctly set for the specific transformation. > > Assuming that the imxvpuenc_mjpeg inherits the videoencoder class and does > > not override the transform_meta function, you will need to make sure that > > the meta tags include "video" and do *not* include "memory" > > > > > > Dimitrios > > > > On Thu, Jan 19, 2017 at 12:47 AM, Philippe De Muyter <[hidden email]> wrote: > > > >> Hi all, > >> > >> I have read that since the 1.6 version of gstreamer frame tags are > >> preserved > >> by each plugin. My questions below are targeted at the propagation by > >> GstVideoEncoders of memory- and frame-size- independent tags, such as gps > >> data, > >> exposure, gain that could vary from frame to frame, and so on. > >> > >> I have the following pipeline : > >> > >> gst-launch-1.0 imxv4l2videosrc ! tee name=t ! queue ! \ > >> imxvpuenc_mjpeg ! mprintmeta logname=1 ! fakesink i\ > >> t. ! queue ! imxvpuenc_h264 ! mprintmeta logname=2 ! fakesink > >> > >> where imxv4l2videosrc has been modified to add custom metadata, and > >> mprintmeta is debugging to show those metadata. I notice that the > >> metadata are not present after the encoders > >> > >> - Is the propagation mechanism implemented outside of the plugins, or must > >> it be implemented in each plugin ? In that case, is a generic propagation > >> code > >> available, that we could add e.g. to gstreamer-imx (imx hardware > >> accelerated) > >> encoder plugins ? > >> > >> - Is GstMetaFreeFunction only called after all instances are unused, or is > >> there a refcount accessible inside the definition of GstMetaFreeFunction > >> for a new metadata type, to avoid that one branch after a 'tee' destroys a > >> metadata that's still used by the other branch ? > >> > >> Thanks in advance > >> > >> Philippe > >> > >> -- > >> Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 > >> Bruxelles > >> _______________________________________________ > >> gstreamer-devel mailing list > >> [hidden email] > >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > >> > > > > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
If you have no tags in the gst_meta_api_type_register then the metadata should pass all the element checks. So adding the video tag and removing memory would only change something if you were already defining you own tags, like e.g. GPS. looking at this function: https://github.com/Freescale/gstreamer-imx/blob/master/src/vpu/encoder_base.c#L504 frames passed to finish frame are not the same as the input_frame param of the function. Your metadata is probably on the input frame and not being handled since the call to the meta transform happens in gst_video_encoder_finish_frame. I think you will have to implement the metadata handling for the finished frames of that element. On Thu, Jan 19, 2017 at 10:29 AM, Philippe De Muyter <[hidden email]> wrote: On Thu, Jan 19, 2017 at 09:41:35AM +0100, Dimitrios Katsaros wrote: _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Hi Dimitrios,
On Thu, Jan 19, 2017 at 12:47:43PM +0100, Dimitrios Katsaros wrote: > If you have no tags in the gst_meta_api_type_register then the metadata > should pass all the element checks. So adding the video tag and removing > memory would only change something if you were already defining you own > tags, like e.g. GPS. Well, I have now removed our own tag and the newly added "video" tag, calling thus gst_meta_api_type_register with an empty tag array, and now our meta_transform function (always copy) is called, and our metadata are propagated auto-magically to the encoded buffers :) > > looking at this function: > https://github.com/Freescale/gstreamer-imx/blob/master/src/vpu/encoder_base.c#L504 > frames passed to finish frame are not the same as the input_frame param of > the function. Your metadata is probably on the input frame and not being > handled since the call to the meta transform happens in > gst_video_encoder_finish_frame. I think you will have to implement the > metadata handling for the finished frames of that element. It seems thus that I won't need to implement metadata handling in encoder_base.c Thank you for your help Philippe > > On Thu, Jan 19, 2017 at 10:29 AM, Philippe De Muyter <[hidden email]> wrote: > > > On Thu, Jan 19, 2017 at 09:41:35AM +0100, Dimitrios Katsaros wrote: > > > And i forgot to answer your specific questions, sorry about that :p > > > > > > 1) There is a default propagation mechanism implemented in every plugin. > > It > > > usually goes by the name "transform_meta". You should not have to > > implement > > > it assuming that elements inherit from the base classes. What you do need > > > to implement is a transformation function within your metadata. > > > > > > 2) metadata lifetime is linked with the buffer it is associated with. > > When > > > tee propagates a buffer it increases the buffers reference count for > > every > > > branch. So you should not have to worry about the reference counting for > > > the metadata. > > > > After the 'tee' I have only one instance of our metadata with a refcount > > set > > to 2. After the videoencoder, in the specific case where the metadata are > > invariant per frame, is that instance merely attached to the encoded > > buffer or > > must it be copied by the transformation function of our metadata ? > > > > Philippe > > > > > > > > DImitrios > > > > > > On Thu, Jan 19, 2017 at 9:33 AM, Dimitrios Katsaros < > > [hidden email]> > > > wrote: > > > > > > > I am not able to find the code implementing that specific encoder on > > the > > > > fly so I will try and work with whtever assumptions I can make about > > > > encoders in general and your post. > > > > > > > > You will need to make sure that your metadata implements a > > transformation > > > > function, the encoder implements a transformation function and that the > > > > tags of the metadata are correctly set for the specific transformation. > > > > Assuming that the imxvpuenc_mjpeg inherits the videoencoder class and > > does > > > > not override the transform_meta function, you will need to make sure > > that > > > > the meta tags include "video" and do *not* include "memory" > > > > > > > > > > > > Dimitrios > > > > > > > > On Thu, Jan 19, 2017 at 12:47 AM, Philippe De Muyter <[hidden email]> > > wrote: > > > > > > > >> Hi all, > > > >> > > > >> I have read that since the 1.6 version of gstreamer frame tags are > > > >> preserved > > > >> by each plugin. My questions below are targeted at the propagation by > > > >> GstVideoEncoders of memory- and frame-size- independent tags, such as > > gps > > > >> data, > > > >> exposure, gain that could vary from frame to frame, and so on. > > > >> > > > >> I have the following pipeline : > > > >> > > > >> gst-launch-1.0 imxv4l2videosrc ! tee name=t ! queue ! \ > > > >> imxvpuenc_mjpeg ! mprintmeta logname=1 ! fakesink i\ > > > >> t. ! queue ! imxvpuenc_h264 ! mprintmeta logname=2 ! fakesink > > > >> > > > >> where imxv4l2videosrc has been modified to add custom metadata, and > > > >> mprintmeta is debugging to show those metadata. I notice that the > > > >> metadata are not present after the encoders > > > >> > > > >> - Is the propagation mechanism implemented outside of the plugins, or > > must > > > >> it be implemented in each plugin ? In that case, is a generic > > propagation > > > >> code > > > >> available, that we could add e.g. to gstreamer-imx (imx hardware > > > >> accelerated) > > > >> encoder plugins ? > > > >> > > > >> - Is GstMetaFreeFunction only called after all instances are unused, > > or is > > > >> there a refcount accessible inside the definition of > > GstMetaFreeFunction > > > >> for a new metadata type, to avoid that one branch after a 'tee' > > destroys a > > > >> metadata that's still used by the other branch ? > > > >> > > > >> Thanks in advance > > > >> > > > >> Philippe > > > >> > > > >> -- > > > >> Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 > > > >> Bruxelles > > > >> _______________________________________________ > > > >> gstreamer-devel mailing list > > > >> [hidden email] > > > >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > >> > > > > > > > > > > > > > _______________________________________________ > > > gstreamer-devel mailing list > > > [hidden email] > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > -- > > Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 > > Bruxelles > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel -- Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Thu, Jan 19, 2017 at 04:08:51PM +0100, Philippe De Muyter wrote:
> Hi Dimitrios, > > On Thu, Jan 19, 2017 at 12:47:43PM +0100, Dimitrios Katsaros wrote: > > If you have no tags in the gst_meta_api_type_register then the metadata > > should pass all the element checks. So adding the video tag and removing > > memory would only change something if you were already defining you own > > tags, like e.g. GPS. > > Well, I have now removed our own tag and the newly added "video" tag, > calling thus gst_meta_api_type_register with an empty tag array, > and now our meta_transform function (always copy) is called, and our > metadata are propagated auto-magically to the encoded buffers :) I also have rechecked with only the "video" tag, and that works also, because of the following lines in gst_video_decoder_transform_meta_default : if (!tags || (g_strv_length ((gchar **) tags) == 1 && gst_meta_api_type_has_tag (info->api, g_quark_from_string (GST_META_TAG_VIDEO_STR)))) return TRUE; where g_strv_length gives the number of entries in the tags array. So having no tag or only the GST_META_TAG_VIDEO_STR (= "video") is equivalent for gst_video_decoder_transform_meta_default Philippe _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Yea, the tags on the metadata tend to be an easy pitfall if you are implementing your own metadata. Good to hear you solved your problem :) DimitriosOn Fri, Jan 20, 2017 at 9:30 AM, Philippe De Muyter <[hidden email]> wrote: On Thu, Jan 19, 2017 at 04:08:51PM +0100, Philippe De Muyter wrote: _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Free forum by Nabble | Edit this page |