GstMeta GTypes - API vs implementation?

[Neil C Smith-2]

Hi All,

Recently I've been looking at one of the missing aspects of the higher
level Java bindings - mappings for GstMeta.  We've had two pull
requests in recent times for this, and neither have been suitable for
merging as is - I'm currently trying to find a way that uses elements
of both and fixes some key issues.

One concern is understanding the intended relationship between the
GTypes for API and implementation.  We currently don't expose GType at
all, but map everything to the Java type system, mirroring the GType
hierarchies where appropriate (eg. AppSink -> BaseSink -> Element ->
GstObject ...)  This is the aspect that's getting confusing!

Are there examples of GstMeta with a single API GType and multiple
implementation GTypes to look at?  If so, is there also anywhere the
difference in type is relevant in usage?

When you pass a GType in to gst_buffer_get_meta are there any
guarantees on what implementation types you get back, now or in
future?

At the moment we only support a 1-to-1 mapping between GTypes and Java
types.  I'm somewhat inclined to map our types to the API GType - eg.
VideoCropMeta to GstVideoCropMetaAPI.  That way we can have (Java)
type guarantees in a call to such as gst_buffer_get_meta.  Any
thoughts on where that might break down?

The section at https://gstreamer.freedesktop.org/documentation/additional/design/meta.html?gi-language=c#api-examples
seems to show passing in a GstMetaInfo rather than GType?  Our other
mapping approach might involve using an opaque MetaInfo wrapper in
place of API types.  But both those would seem to imply filtering by
implementation type.

Thanks in advance for any thoughts or pointers you might have here!

Best wishes,

Neil

--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mathieu Duponchelle]

Hey Neil,

The impl vs. API GType situation is very confusing indeed, in my understanding
that design was never used in practice, Wim could probably give more background
about that I reckon.

I know this doesn't directly answer your question, but we've recently merged a
custom meta API that should be enough for 99% of the GstMeta use cases:

<https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/609>.

I would also recommend you take a look at how GstMeta was wrapped in rust,
that might give you some more insight :)

Best,
Mathieu

On 10/6/20 7:03 PM, Neil C Smith wrote:

> Hi All,
>
> Recently I've been looking at one of the missing aspects of the higher
> level Java bindings - mappings for GstMeta.  We've had two pull
> requests in recent times for this, and neither have been suitable for
> merging as is - I'm currently trying to find a way that uses elements
> of both and fixes some key issues.
>
> One concern is understanding the intended relationship between the
> GTypes for API and implementation.  We currently don't expose GType at
> all, but map everything to the Java type system, mirroring the GType
> hierarchies where appropriate (eg. AppSink -> BaseSink -> Element ->
> GstObject ...)  This is the aspect that's getting confusing!
>
> Are there examples of GstMeta with a single API GType and multiple
> implementation GTypes to look at?  If so, is there also anywhere the
> difference in type is relevant in usage?
>
> When you pass a GType in to gst_buffer_get_meta are there any
> guarantees on what implementation types you get back, now or in
> future?
>
> At the moment we only support a 1-to-1 mapping between GTypes and Java
> types.  I'm somewhat inclined to map our types to the API GType - eg.
> VideoCropMeta to GstVideoCropMetaAPI.  That way we can have (Java)
> type guarantees in a call to such as gst_buffer_get_meta.  Any
> thoughts on where that might break down?
>
> The section at https://gstreamer.freedesktop.org/documentation/additional/design/meta.html?gi-language=c#api-examples
> seems to show passing in a GstMetaInfo rather than GType?  Our other
> mapping approach might involve using an opaque MetaInfo wrapper in
> place of API types.  But both those would seem to imply filtering by
> implementation type.
>
> Thanks in advance for any thoughts or pointers you might have here!
>
> Best wishes,
>
> Neil
>
> --
> Neil C Smith
> Codelerity Ltd.
> www.codelerity.com
>
> Codelerity Ltd. is a company registered in England and Wales
> Registered company number : 12063669
> Registered office address : Office 4 219 Kensington High Street,
> Kensington, London, England, W8 6BD
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Neil C Smith-2]

On Tue, 6 Oct 2020 at 18:38, Mathieu Duponchelle
<[hidden email]> wrote:
> The impl vs. API GType situation is very confusing indeed, in my understanding
> that design was never used in practice, Wim could probably give more background
> about that I reckon.

Yes, definitely confusing!  Thanks for getting back to me.

> I know this doesn't directly answer your question, but we've recently merged a
> custom meta API that should be enough for 99% of the GstMeta use cases:

Saw that in the docs - looks interesting, and something that we'll
look to add support for.

> I would also recommend you take a look at how GstMeta was wrapped in rust,
> that might give you some more insight :)

Yes, I often refer to Rust and C# bindings, and looking to get more
aligned - at some point given time/resources we'll be moving to
auto-generation, and new additions tend to be made with that in mind.

So, with regard to Rust and Meta, the code eg. here seems to imply
that the MetaAPI type is 1-to-1 mapped to the implementation type by
MetaAPI::GstType?  My Rust-fu is limited still!

https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/gstreamer/src/buffer.rs#L399
https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/gstreamer-video/src/video_meta.rs#L222

That's a sort of pattern we could work with, as long as it's
guaranteed to hold in all cases?

Thanks,

Neil


--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Wed, 2020-10-07 at 15:59 +0100, Neil C Smith wrote:

>
> > I would also recommend you take a look at how GstMeta was wrapped in rust,
> > that might give you some more insight :)
>
> Yes, I often refer to Rust and C# bindings, and looking to get more
> aligned - at some point given time/resources we'll be moving to
> auto-generation, and new additions tend to be made with that in mind.
>
> So, with regard to Rust and Meta, the code eg. here seems to imply
> that the MetaAPI type is 1-to-1 mapped to the implementation type by
> MetaAPI::GstType?  My Rust-fu is limited still!
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/gstreamer/src/buffer.rs#L399
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/gstreamer-video/src/video_meta.rs#L222
>
>
> That's a sort of pattern we could work with, as long as it's
> guaranteed to hold in all cases?

Maybe not. The current API in the Rust bindings is based on how the C
API is used in practice everywhere, not on how it could in theory be
used as that would complicate everything considerably and I don't see
it happening that someone ever actually makes use of that in the
future. If I'm wrong then it's not much of a problem and the API can be
updated :)

The reason for the split between two GTypes for the meta was (IIRC) to
be able to have some kind of inheritance. You could have one meta that
*also* implements another API, like a GstVideoMeta2 that is compatible
with GstVideoMeta. I don't see how this can be possible with the
current implementation in C and how it can be introduced at this point
without breaking a lot of code, so I'm not really worrying about that.

For all practical purposes, there are two GTypes for every meta, and
the reason for making that the API one in the Rust bindings is that
this is what e.g. gst_buffer_iterate_meta_filtered() works with.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Neil C Smith-2]

On Wed, 7 Oct 2020 at 21:28, Sebastian Dröge <[hidden email]> wrote:

>
> On Wed, 2020-10-07 at 15:59 +0100, Neil C Smith wrote:
> > That's a sort of pattern we could work with, as long as it's
> > guaranteed to hold in all cases?
>
> Maybe not. The current API in the Rust bindings is based on how the C
> API is used in practice everywhere, not on how it could in theory be
> used as that would complicate everything considerably and I don't see
> it happening that someone ever actually makes use of that in the
> future. If I'm wrong then it's not much of a problem and the API can be
> updated :)

Thanks Sebastian!  Yes, I think we're in the same position then.  I
assumed this was the case before initiating this thread, but wanted to
check I wasn't missing somewhere that theory had been put into
practice! :-)

> The reason for the split between two GTypes for the meta was (IIRC) to
> be able to have some kind of inheritance. You could have one meta that
> *also* implements another API, like a GstVideoMeta2 that is compatible
> with GstVideoMeta. I don't see how this can be possible with the
> current implementation in C and how it can be introduced at this point
> without breaking a lot of code, so I'm not really worrying about that.

Good to know.  I assumed that was the reason, but curious that it was
done (or had to be done) in a way that isn't introspectable via the
type system (as far as I can tell).

> For all practical purposes, there are two GTypes for every meta, and
> the reason for making that the API one in the Rust bindings is that
> this is what e.g. gst_buffer_iterate_meta_filtered() works with.

Yes, we'll do similar - live and die on the same sword! :-)

Many thanks for both of your inputs.

Best wishes,

Neil



--
Neil C Smith
Codelerity Ltd.
www.codelerity.com

Codelerity Ltd. is a company registered in England and Wales
Registered company number : 12063669
Registered office address : Office 4 219 Kensington High Street,
Kensington, London, England, W8 6BD
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel