Building GStreamer Plugins Base with NVMM <-> GL interop

[thaytan]

I'm building a kirkstone image, and trying build gstreamer1.0-plugins-base with the NVMM interop in the GL stack. Compiling requires depending on tegra-libraries-multimedia-utils and tegra-mmapi to pull in the nvbuf_utils header and lib.

Here's my gstreamer1.0-plugins-base_%.bbappend:

DEPENDS:append:tegra = " tegra-libraries-multimedia-utils tegra-mmapi"
PACKAGE_ARCH:tegra = "${TEGRA_PKGARCH}"

The first issue was that causing a circular build dep on tegra-libraries-multimedia-utils, but I saw that was solved in master last year via #1243. I backported that, and the gstreamer1.0-plugins-base build successfully detects and tries to build the NVMM support. The problem is that it cause a gobject-introspection failure, with the nvidia library trying to initialise things that aren't available in qemu (I think):

FAILED: gst-libs/gst/gl/GstGL-1.0.gir 
.. giant g-ir-scanner-wrapper invocation
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
If the above error message is about missing .so libraries, then setting up GIR_EXTRA_LIBS_PATH in the recipe should help.
(typically like this: GIR_EXTRA_LIBS_PATH="${B}/something/.libs" )

I'm not sure how to work around that, or why it doesn't affect some of the other nvidia GStreamer packages that build against tegra-libraries-multimedia-utils

Any suggestions appreciated.

[madisongh]

Unless you really need the gobject introspection files, you could work around the problem by disabling their generation for that recipe.

To fix the problem, though, you need to track down where in the gstreamer code it's calling on the nvbuf APIs, and see if you can avoid doing that for the code paths used during GI generation... assuming that's possible. Otherwise, you might have to defer the generation to on-target, since it's unlikely the NVIDIA libraries could ever be made to work in the usermode QEMU used during the build.

[thaytan]

Thanks! For this build I don't think gir will be important, but I wasn't sure how to disable it for one recipe, or if there'd be fallout elsewhere.

The initialisation / use of the Nvidia libraries has to come from introspecting glupload and gldownload. I'm not sure if it can be avoided, but will look into it

[thaytan]

I think simply linking libnvbuf_utils is the problem here, and that the failure is caused by calling methods in the library init section, or one of the other nvidia libs it pulls in. That makes it hard to avoid, short of preloading a dummy stub library. Disabling gir for libgstgl will be easier.

[thaytan]

That turned out to be the easiest option:

.../gstreamer1.0-plugins-base_%.bbappend:
@@ -1,7 +1,13 @@
+DEPENDS:append:tegra = " tegra-libraries-multimedia-utils tegra-mmapi"
+PACKAGE_ARCH:tegra = "${TEGRA_PKGARCH}"
+
+# Disble gobject introspection. It doesn't work with tegra libs linked in
+GIR_MESON_OPTION = ""

If it's the nvidia library doing things in a constructor, it's hard to avoid, so I'm not sure how to get a build that could have NVMM integration by default. It would either need to use a dummy nvbuf_utils library for gobject-introspection, or use a build without the library linked just to extract the gir info... or change the GStreamer GL lib to dlopen the library only when it actually needs it.

[madisongh]

I'm not surprised. You probably want to add the :tegra modifier to that setting, too, but at least you have something that gets it building.

Note, btw, that as of L4T R35.4.1, the nvbuf_utils library is gone; NVIDIA deprecated it a while back. I expect the nvbufsurface libraries that replaced it probably behave the same way, though.