Manually dropping video frame in element

[Ryan Brigden]

Hello!

I was directed to this mailing list in this issue [1] in the
gstreamer-rs repo on gitlab. My goal was to create a CustomMetaAPI in
Rust that is defined in a standalone crate, and then consumed in other
crates that run gst::Pipeline s and crates that define plugins. In my
example, the CustomMetaAPI is defined in a standalone crate and
consumed by both the crate creating the pipeline and writing the meta
to buffers using a pad probe AND a plugin that consumes the meta
downstream.

After some more digging to identify the cause of the problem I
mentioned in [1], I realized that the `gst_meta_api_type_register` was
being called twice:

1. In a crate that creates the pipeline and attaches my custom meta to
each buffer in a pad probe.
2. In a downstream plugin that consumes the meta from each buffer

I found this issue [2] on the gstreamer-rs gitlab repo to exactly
describe this double invocation of the meta registration (despite
having been declared as `Lazy`).

@slomo mentioned a possible solution (if I understand its implications
correctly) is to put the `imp` module in the official example [3] into
a separate cdylib and then copy over the wrapper code into each place
the custom meta is to be used -- in my case this would be in both the
crate invoking the pipeline and the crate with the plugin.

from the issue [2]
> For that you'll have to put the imp module into a cdylib and make those two functions that use
> the Lazy type public, and from everywhere else you would only use the non-imp part of the
> meta and dynamically link to that cdylib for accessing the functions.

So I'm not sure how to set up this solution correctly in my case.
Currently I am isolating the `imp` module in its own crate, building
it as a cdylib. Then in both the pipeline crate and the plugin crate I
am adding the wrapper code (everything outside the `imp` module in the
example[3]). Unfortunately I can't seem to get the linking to work in
this case.

Does anyone have a suggestion for a simpler way to accomplish a shared
CustomMetaAPI in Rust? It seems... not great to have to duplicate the
wrapper code or keep it in a separate crate as the ffi.

Thank you :)

- Ryan

[1]: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/303
[2]: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/261
[3]: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/examples/src/bin/custom_meta.rs
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Tue, 2021-01-19 at 16:32 -0800, Ryan Brigden wrote:
>
> I found this issue [2] on the gstreamer-rs gitlab repo to exactly
> describe this double invocation of the meta registration (despite
> having been declared as `Lazy`).

It's the standard static linking problem where the same code is
included twice in the resulting application.

The solution has to be to move this code into some kind of shared
library.

> @slomo mentioned a possible solution (if I understand its
> implications
> correctly) is to put the `imp` module in the official example [3]
> into
> a separate cdylib and then copy over the wrapper code into each place
> the custom meta is to be used -- in my case this would be in both the
> crate invoking the pipeline and the crate with the plugin.

You can also move the wrapper code into a separate crate instead of
copying it. The important part is that the meta implementation must be
in a shared library and only used through that.

It should also be possible to have cargo build the crate defining the
meta implementation into a shared Rust library, but the tooling for
that is rather suboptimal and you'll run into all kinds of interesting
problems unless you build that crate and all users of it as part of the
same cargo workspace.

> example[3]). Unfortunately I can't seem to get the linking to work in
> this case.

How does it fail and is your code or a testcase available somewhere?

> Does anyone have a suggestion for a simpler way to accomplish a
> shared CustomMetaAPI in Rust? It seems... not great to have to
> duplicate the wrapper code or keep it in a separate crate as the ffi.

Think of the wrapper code as bindings to your meta implementation that
is provided by a shared library with a C API :) The wrapper code would
live in its own bindings crate, just like gstreamer-rs currently. And
the implementation of the meta would be a separate project.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Ryan Brigden]

What you suggest is a nice solution given that the wrapper can be
shared from a single crate. We've put together an example that adapts
the CustomMetaAPI example in gstreamer-rs [1] to our current approach.

The master branch of the example repo reproduces the double meta api
initialization error [2].

Another branch [3] of this repo contains our attempt to resolve the
issue by separating the implementation (gst-meta-imp), which is built
as a dylib, and wrapper (gst-meta). The example in this branch also
includes a plugin that uses the custom meta.

In [3], the custom_meta example (in the gst-meta crate) runs fine, but
the test_pipeline fails with the error:
./target/release/test_pipeline: error while loading shared libraries:
libstd-e12de7683a34c500.so: cannot open shared object file: No such
file or directory

We're not sure what exactly requires a libstd shared object? Perhaps
we are taking the incorrect approach here?

Thanks again

[1]: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/examples/src/bin/custom_meta.rs
[2]: https://gitlab.freedesktop.org/alex4/gstmeta-test-case/-/tree/master
[3]: https://gitlab.freedesktop.org/alex4/gstmeta-test-case/-/tree/attempt-dynamic-linking
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Wed, 2021-01-20 at 16:35 -0800, Ryan Brigden wrote:
>
> The master branch of the example repo reproduces the double meta api
> initialization error [2].

Yeah that's exactly the case we talked about: it causes two copies of
the meta to be used and those conflict then.

> Another branch [3] of this repo contains our attempt to resolve the
> issue by separating the implementation (gst-meta-imp), which is built
> as a dylib, and wrapper (gst-meta). The example in this branch also
> includes a plugin that uses the custom meta.
>
> In [3], the custom_meta example (in the gst-meta crate) runs fine,
> but the test_pipeline fails with the error:
> ./target/release/test_pipeline: error while loading shared libraries:
> libstd-e12de7683a34c500.so: cannot open shared object file: No such
> file or directory
>
> We're not sure what exactly requires a libstd shared object?

The one from your currently used Rust toolchain. That's required
because the meta and its users have to use the same implementation of
the `String` type (etc) and that's defined in there.

If std was linked statically you would have three copies of that again
(one in the application, one in the plugin, one in the
libgst_meta_imp.so).

If you installed Rust via rustup you would find it at e.g.
  ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib

To get around that you would have to use only C types in the meta
implementation and compile it as a cdylib. You'd then have basically
the same situation as with the bindings for e.g. GstVideoMeta: a plain
C interface between shared libraries and Rust bindings for those.

> Perhaps we are taking the incorrect approach here?

It depends on your goals. What you do there would work. You can also do
it a bit simpler though. See the attached patch.

If you build libgst_meta_imp.so as a Rust shared library you must build
everything as part of the same cargo workspace or otherwise you'll
likely run into symbol mismatches unless you try really hard (and that
probably requires cargo changes).

The only way around that is using the C ABI between the shared
libraries. The above is all a result of Rust not having a stable ABI
and the tooling trying to prevent ABI mismatches as much as possible by
default.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel