Very poor VP2 performances with point instancers and "Depth Peeling" or "Alpha Cut"

[jufrantz]

Describe the bug

We notice huge VP2 performance loss on scenes containing large point instancers when switching to Depth Peeling or Alpha Cut transparency.

The issue does not appear when world consolidation is enabled, but we need to disable it in some cases.

Steps to reproduce

1. Create a mayaUsd proxyShape with 100K point instances.

import random
import pxr.UsdGeom
import maya.cmds
import mayaUsd.ufe
import mayaUsd_createStageWithNewLayer
maya.cmds.loadPlugin("mayaUsdPlugin", quiet=True)

proxyShape = mayaUsd_createStageWithNewLayer.createStageWithNewLayer()
shapeStage = mayaUsd.ufe.getStage(proxyShape)
shapeStage.GetRootLayer().ImportFromString('''#sdf 1
(
    defaultPrim="PI"
)
def PointInstancer "PI"
{
    rel prototypes = [</PI/Protos/Sphere>]
    def Scope "Protos" 
    { 
        def Sphere "Sphere" {}
    }
}
''')

instancer = pxr.UsdGeom.PointInstancer(shapeStage.GetDefaultPrim())
instCount = 100000
instBounds = (-1000.0, 1000.0)
instancer.CreateProtoIndicesAttr().Set([0] * instCount)
instancer.CreatePositionsAttr().Set([ 
    (random.uniform(*instBounds), 0.0, random.uniform(*instBounds)) 
    for i in range(instCount)
])

2. Disable VP2 consolidation. I beleive it is hiding the issue.

maya.cmds.setAttr("hardwareRenderingGlobals.consolidateWorld", 0)

3. Draw the scene with default Object Sorting transparency.

maya.cmds.setAttr("hardwareRenderingGlobals.transparencyAlgorithm", 1)
maya.cmds.ogs(traceRenderPipeline=True)
maya.cmds.refresh()
maya.cmds.ogs(traceRenderPipeline=False)
# -- VP2 Trace[modelPanel4][3d Beauty Pass]
# Count for list: [Opaque] = 1

4. Check VP2 performances:

• traceRenderPipeline shows a unique (instanced) opaque item.
• maya profiler shows
  • VP2UpdateScene ≈ 300us
  • VP2Draw3dBeautyPass ≈ 400us
  • VP2ClearRenderLists ≈ 2us

5. Draw the scene with Depth Peeling transparency (or ALpha Cut)

maya.cmds.setAttr("hardwareRenderingGlobals.transparencyAlgorithm", 3)
maya.cmds.ogs(traceRenderPipeline=True)
maya.cmds.refresh()
maya.cmds.ogs(traceRenderPipeline=False)
# -- VP2 Trace[modelPanel4][3d Beauty Pass]
# Count for list: [Opaque] = 100000
# Count for list: [Opaque] = 100000

6. Check VP2 performances:

• traceRenderPipeline shows 100000 (non-instanced) opaque items.
• maya profiler shows
  • VP2UpdateScene ≈ 120ms
  • VP2Draw3dBeautyPass ≈ 500ms
  • VP2ClearRenderLists ≈ 40ms

Specs (if applicable):

• CentOS 7.8
• Maya 2023.3
• maya-usd 0.25
• USD 22.11