Add KHR_mesh_variants

[edzis]

This has been discussed in 3D Commerce Configurability TSG for a couple of months in relation to #1681 with general consensus that the core functionality of toggling visibility of meshes at runtime is needed. This PR attempts to communicate the needs and suggests one approach to cover not just the core needs but also more complex scenarios.

It is published at a raw state with hopes to gather feedback from a wider audience and also to ensure this extension does not require any changes to #1681 for both to be used together. Feedback is wanted.

A lot of implementation details are not yet clear. Help is wanted.

[abwood]

We had to solve this problem (about 25years ago!) for our desktop software, STK. To make our solution compatible with glTF we added a vendor extension, AGI_articulations.

Just to give a real world example of how we needed to control visibility of nodes in a model, consider a model of a launch vehicle. If this model were just taken as-is and loaded into an application, we would see flames shooting out of the boosters. When this model is loaded into our application, it will initially be placed on a launch pad and for that reason we want those flames to be hidden by default.

Here's a little comparison of these two models in VSCode. On the right, I am viewing this model in Cesium, which is aware of the AGI_articulations extension and honors the default value of the scale articulation on the SRBFlames.

Internally, an articulation is the definition of a constraint and default values. For the flames, this model defines a scale constraint that ranges from 0-1 with a default value of 0 (hidden). Articulations is a bit broader than what is proposed here, as an articulation also covers translation and rotation constraints. In our domain this information is used to say "this node that represents the panning of radar dish, can rotate about the y-axis between 30-130 degrees". We then use that constraint to limit transformations at runtime (e.g., the radar dish that is tracking an object cannot track beyond 130 horizontally).

Obviously our vendor extension was not designed for 3D commerce, but maybe there are ideas here that can be applicable.

[edzis]

@abwood Thanks for sharing this!
I think articulations are a bit too verbose, unintuitive and error-prone for showing/hiding nodes - you need to deal with the extra depth of stages and breadth of min/max values, extra mapping from tags, ability to set scale in between the expected values.
But I would like to adopt the approach of an external file and explore the way how you are applying the same transformations to multiple nodes.
And there are other conversations within the Configuration TSG about transformations where the articulations extension could be very relevant.

[donmccurdy]

Which structure should host the extension - mesh, node or primitive?

In general, a glTF mesh will only have one primitive. There are exceptions to that, but many tools simply treat additional primitives as additional meshes. I would try to avoid associating too much meaning with distinct primitives in the same mesh. In many viewers, glTF files containing (a) a mesh with two primitives or (b) two meshes, would be indistinguishable after loading into the engine.

Not sure whether that actually answers the question, but it may be relevant. 🙂

How to hide some meshes by default, also in viewers that do not support this extension?

Perhaps something like KHR_material_variants, but at the node level:

"nodes": [{
  "name": "convertible",
  "extensions": {
    "KHR_mesh_variants": {
      "mapping": [
        {
          "tags": ["roof_detached"]
        },
        {
          "tags": ["roof_down"],
          "mesh": 0
        },
        {
          "tags": ["roof_up"],
          "mesh": 1
        }
      ]
    }
  }
}]

In this case the node has no visible mesh by default, or when the roof_detached tag is active. The roof_down and roof_up tags display different meshes.

EDIT: Unfortunately using distinct mesh indices does complicate the material variants behavior, as presumably different meshes could already have different materials. That seems messy...

[emackey]

In general, a glTF mesh will only have one primitive.

Don knows this already, but I feel the need to clarify for anyone else reading. One notable exception is that a single primitive may use only one glTF material.

So, a mesh that uses multiple glTF materials will typically have one primitive per material. (And in a rasterizer, each primitive then roughly corresponds to one draw call, and each glTF material corresponds to one shader program associated with one stack of textures and settings).

[emackey]

Which structure should host the extension - mesh, node or primitive?

I'd strongly vote for node over the others. A mesh is allowed to be referenced by multiple nodes, and sometimes you want to turn some off without others.

For example, if you were selling different Atlas rocket configurations, you would want the ability to configure out some of the SRBs (the smaller tubes near the base) but not all of them. Multiple nodes there might reference the same SRB mesh. So you'd need the configuration on the node.

Likewise, position, rotation, scale, animations, and articulations all target the node level, not meshes or primitives.

[rsahlin]

We already have a PR that adresses this issue - on a node level:
#1760

Please look into that and if that solves your usecase @edzis then please close this PR.
Thanks

(My comment on #1760 is that we should do it the same way as geometry visibility is toggled in USD)

[emackey]

@rsahlin I'm confused, didn't you advocate for this in #1681 (comment)?

Looks like #1760 doesn't use the "variants" system that #1681 calls for.

[rsahlin]

Well I guess that if the purpose of this extension is to allow toggling the visibility of geometry - then I think we should use the definition in #1760 - I think the behavior is clean and mimics what is used in for instance USD.
So, no - it does not follow the tags/variants system.
Personally I do not see a problem with that since the declaration is on Node level.
It defines a new state used while traversing the tree.

Hope that clarifies your question @donmccurdy ?

[edzis]

@rsahlin @emackey @donmccurdy Please see #1760 (comment)
but the essence of that is:

But I believe that to make this relevant for Configurability this extension needs to have a dynamic nature beyond just specifying the visibility value at authoring time.

[Drigax]

Can we clarify why we're choosing to use this variants system over recommending node visibility toggling via animations?

[donmccurdy]

The animation spec does not currently support toggling visibility.

[Drigax]

Ok, my mistake. I thought we did support visibility. Suppose we instead implement node visibility animations; is there an inherent advantage of implementing this system instead of using animation?

EDIT:
The reason I ask this, is I can forsee a future ask for animating the current active variant/variants as well...
These both seem like methods of dynamically changing our scene, maybe we want to rethink our implementation?

[edzis]

@Drigax From my perspective this is meant to interact with the same tags that are used on #1681 without any extra logic from the viewer. Using animations would require more mapping data and logic on runtime.

[aaronfranke]

This extension seems like it can be replaced with a combination of KHR_node_visibility and KHR_animation_pointer using animations to alter node visibility based on which animation is played, or a combination of KHR_node_visibility and KHR_interactivity using interactivity graphs to alter visibility depending on arbitrary conditions.