Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve sort order #5341

Merged
merged 19 commits into from
Aug 11, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions docs/components/renderer.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ It also configures presentation attributes when entering WebVR/WebXR.
| colorManagement | Whether to use a color-managed linear workflow. | true |
| highRefreshRate | Increases frame rate from the default (for browsers that support control of frame rate). | false |
| foveationLevel | Amount of foveation used in VR to improve perf, from 0 (min) to 1 (max). | 1 |
| sortObjects | Whether to sort objects before rendering. | false |
| sortTransparentObjects | Whether to sort transparent objects (far to near) before rendering | false |
| physicallyCorrectLights | Whether to use physically-correct light attenuation. | false |
| maxCanvasWidth | Maximum canvas width. Uses the size multiplied by device pixel ratio. Does not limit canvas width if set to -1. | 1920 |
| maxCanvasHeight | Maximum canvas height. Behaves the same as maxCanvasWidth. | 1920 |
Expand All @@ -43,7 +43,7 @@ It also configures presentation attributes when entering WebVR/WebXR.
| exposure | When any toneMapping other than "no" is used this can be used to make the overall scene brighter or darker | 1 |
| anisotropy | Default anisotropic filtering sample rate to use for textures | 1 |

> **NOTE:** Once the scene is initialized, none of these properties may no longer be changed apart from "exposure" and "toneMapping" which can be set dynamically.
> **NOTE:** Once the scene is initialized, none of these properties may no longer be changed apart from "exposure", "toneMapping", and "sortTransparentObjects" which can be set dynamically.

### antialias

Expand Down Expand Up @@ -82,14 +82,22 @@ Controls the amount of foveation which renders fewer pixels near the edges of th
when in stereo rendering mode on certain systems. The value should be in the range of 0 to 1, where
0 is the minimum and 1 the maximum amount of foveation. This is currently supported by the Oculus Browser.

### sortObjects


### sortTransparentObjects

[sorting]: ../introduction/faq.md#what-order-does-a-frame-render-objects-in

Sorting is used to attempt to properly render objects that have some degree of transparency.
Due to various limitations, proper transparency often requires some amount of careful setup.
By default, objects are not sorted, and the order of elements in the DOM determines order of
By default, transparent objects are not sorted, and the order of elements in the DOM determines order of
rendering. Re-ordering DOM elements provides one way of forcing a consistent behavior, whereas
use of `renderer="sortObjects: true"` may cause unwanted changes as the camera moves.

Some more background on how A-Frame sorts objects for rendering can be found [here][sorting]



### physicallyCorrectLights

By default, point and spot lights attenuate (or, appear dimmer as they become farther away)
Expand Down
18 changes: 18 additions & 0 deletions docs/introduction/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,3 +343,21 @@ Phones with Adreno 300 series GPUs are notoriously problematic. Set [renderer pr
## Can I use A-Frame offline or self hosted?

Using A-Frame online sometimes is not possible or inconvenient, like for instance when traveling or during public events with poor Internet connectivity. A-Frame is mostly self-contained so including the build (aframe.min.js) in your project will be sufficient in many cases. Some specific parts are lazy loaded and only fetched when used. This is for example the case of the fonts for the text component and the 3D models for controllers. In order to make an A-Frame build work either offline or without relying on A-Frame hosting infrastructure (typically cdn.aframe.io), you can monitor network requests on your browser console. This will show precisely what assets are being loaded and thus as required for your specific experience. Fonts can be found via FONT_BASE_URL in the whereas controllers via MODEL_URLS. Both can be modified in the source and included in your own [custom build](https://github.com/aframevr/aframe#generating-builds)

## What order does A-Frame render objects in?

[sortTransparentObjects]: ../components/renderer.md#sorttransparentobjects

In many cases, the order in which objects is rendered doesn't matter much - most scenes will look the same whatever order the objects are rendered in - but there are a few cases where sorting is important:

- for transparent objects, it's typically better to render objects furthest to nearest (although some cases are more complex and require [more sophisticated approaches](https://threejs.org/manual/?q=transp[#en/transparency)). However, when the camera and/or objects are moving, this can result in undesirable visual effects when objects switch in terms of their relative distance from the camera
- for performance reasons, it's typically desirable to render objects nearest to furthest, so that GPU doesn't spend time drawing pixels that end up being drawn over.
- for AR "hider material" used to hide parts of the scene to create the appearance of occlusion by real-world objects, it's typically necessary to render these before the rest of the scene.

By default, A-Frame sorts objects as follows:

- for all objects, if [`renderOrder`](https://threejs.org/docs/index.html?q=object3D#api/en/core/Object3D.renderOrder) is set on the Object3D or a Group that it is a member of, the specified order will be respected
- otherwise, opaque objects are rendered furthest to nearest, and transparent objects are rendered in the order they appear in the THREE.js Object tree (in most cases, this is the same as the order they appear in the DOM)

The `renderer` system has a [`sortTransparentObjects`][sortTransparentObjects] property, which can be used to render transparent objects furthest to nearest, rather than in DOM order.

2 changes: 1 addition & 1 deletion src/core/scene/a-scene.js
Original file line number Diff line number Diff line change
Expand Up @@ -622,7 +622,7 @@ class AScene extends AEntity {

renderer = this.renderer = new THREE.WebGLRenderer(rendererConfig);
renderer.setPixelRatio(window.devicePixelRatio);
renderer.sortObjects = false;

if (this.camera) { renderer.xr.setPoseTarget(this.camera.el.object3D); }
this.addEventListener('camera-set-active', function () {
renderer.xr.setPoseTarget(self.camera.el.object3D);
Expand Down
56 changes: 56 additions & 0 deletions src/core/scene/sortFunctions.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Custom A-Frame sort functions.
Copy link
Contributor Author

@diarmidmackenzie diarmidmackenzie Jul 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think putting these into a new module is right. Not sure whether this is the right place in the directory structure, or if there's a more suitable place.

I didn't think these were utils as they aren't part of the exposed A-Frame API. Couldn't see another obvious directory to put them in, so put them here.

Copy link
Member

@dmarcos dmarcos Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put them in the renderer file (define at the bottom) where they are used. see setupCanvas function in a-scene https://github.com/aframevr/aframe/blob/master/src/core/scene/a-scene.js#L863 No need to prepend aframe to the function name

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can think of modularizing later if the functions turn to be useful somewhere else (like those in utils)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about this? They are used in both a-scene, and renderer. Defining them in renderer means that a core module (a-scene) will have a dependency on a systems module (renderer).

That seems like the wrong direction for a dependency to me.

It might be preferable to define them in a-scene?

Agree with removing aframe from the function names, and have made that change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry missed they're used in a-scene as well. Do we need them in both places? Can we consolidate in just one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the functions used in a-scene. I might be missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my mistake. They were used in a-scene until I did some refactoring proposed by @mrxz . I'll fix as you suggest now.

// Variations of Three.js default sort orders here:
// https://github.com/mrdoob/three.js/blob/ebbaecf9acacf259ea9abdcba7b6fb25cfcea2ab/src/renderers/webgl/WebGLRenderLists.js#L1
// See: https://github.com/aframevr/aframe/issues/5332

// Default sort for opaque objects:
// - respect groupOrder & renderOrder settings
// - sort front-to-back by z-depth from camera (this should minimize overdraw)
// - otherwise leave objects in default order (object tree order)

function sortOpaqueDefault (a, b) {
Copy link
Member

@dmarcos dmarcos Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function names are a bit cryptic. sortOpaqueObjectsFirst maybe? Default has not meaning without more context

if (a.groupOrder !== b.groupOrder) {
return a.groupOrder - b.groupOrder;
} else if (a.renderOrder !== b.renderOrder) {
return a.renderOrder - b.renderOrder;
} else if (a.z !== b.z) {
return a.z - b.z;
} else {
return 0;
}
}

// Default sort for transparent objects:
// - respect groupOrder & renderOrder settings
// - otherwise leave objects in default order (object tree order)
function sortTransparentDefault (a, b) {
if (a.groupOrder !== b.groupOrder) {
return a.groupOrder - b.groupOrder;
} else if (a.renderOrder !== b.renderOrder) {
return a.renderOrder - b.renderOrder;
} else {
return 0;
}
}

// Spatial sort for transparent objects:
// - respect groupOrder & renderOrder settings
// - sort back-to-front by z-depth from camera
// - otherwise leave objects in default order (object tree order)
function sortTransparentSpatial (a, b) {
if (a.groupOrder !== b.groupOrder) {
return a.groupOrder - b.groupOrder;
} else if (a.renderOrder !== b.renderOrder) {
return a.renderOrder - b.renderOrder;
} else if (a.z !== b.z) {
return b.z - a.z;
} else {
return 0;
}
}

module.exports = {
sortOpaqueDefault: sortOpaqueDefault,
sortTransparentDefault: sortTransparentDefault,
sortTransparentSpatial: sortTransparentSpatial
};
18 changes: 16 additions & 2 deletions src/systems/renderer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
var registerSystem = require('../core/system').registerSystem;
var utils = require('../utils/');
var THREE = require('../lib/three');
var sortFunctions = require('../core/scene/sortFunctions');

var debug = utils.debug;
var warn = debug('components:renderer:warn');
Expand All @@ -20,7 +21,7 @@ module.exports.System = registerSystem('renderer', {
toneMapping: {default: 'no', oneOf: ['no', 'ACESFilmic', 'linear', 'reinhard', 'cineon']},
precision: {default: 'high', oneOf: ['high', 'medium', 'low']},
anisotropy: {default: 1},
sortObjects: {default: false},
sortTransparentObjects: {default: false},
colorManagement: {default: true},
alpha: {default: true},
foveationLevel: {default: 1}
Expand All @@ -32,7 +33,7 @@ module.exports.System = registerSystem('renderer', {
var toneMappingName = this.data.toneMapping.charAt(0).toUpperCase() + this.data.toneMapping.slice(1);
// This is the rendering engine, such as THREE.js so copy over any persistent properties from the rendering system.
var renderer = sceneEl.renderer;
renderer.sortObjects = data.sortObjects;

renderer.useLegacyLights = !data.physicallyCorrectLights;
renderer.toneMapping = THREE[toneMappingName + 'ToneMapping'];
THREE.Texture.DEFAULT_ANISOTROPY = data.anisotropy;
Expand All @@ -47,6 +48,10 @@ module.exports.System = registerSystem('renderer', {
if (sceneEl.hasAttribute('logarithmicDepthBuffer')) {
warn('Component `logarithmicDepthBuffer` is deprecated. Use `renderer="logarithmicDepthBuffer: true"` instead.');
}

// These properties are always the same, regardless of rendered oonfiguration
renderer.sortObjects = true;
renderer.setOpaqueSort(sortFunctions.sortOpaqueDefault);
},

update: function () {
Expand All @@ -57,6 +62,15 @@ module.exports.System = registerSystem('renderer', {
renderer.toneMapping = THREE[toneMappingName + 'ToneMapping'];
renderer.toneMappingExposure = data.exposure;
renderer.xr.setFoveation(data.foveationLevel);

if (data.sortObjects) {
warn('`sortObjects` property is deprecated. Use `renderer="sortTransparentObjects: true"` instead.');
}
if (data.sortTransparentObjects) {
renderer.setTransparentSort(sortFunctions.sortTransparentSpatial);
} else {
renderer.setTransparentSort(sortFunctions.sortTransparentDefault);
}
},

applyColorCorrection: function (texture) {
Expand Down
6 changes: 4 additions & 2 deletions tests/__init.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,12 @@ setup(function () {
},
dispose: function () {},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reordered this list to be alphabetic

getContext: function () { return undefined; },
render: function () {},
setAnimationLoop: function () {},
setSize: function () {},
setOpaqueSort: function () {},
setPixelRatio: function () {},
render: function () {},
setSize: function () {},
setTransparentSort: function () {},
shadowMap: {enabled: false}
};
});
Expand Down
80 changes: 80 additions & 0 deletions tests/core/sortFunctions.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/* global assert, process, suite, test, setup */

var {sortOpaqueDefault, sortTransparentDefault, sortTransparentSpatial} = require('core/scene/sortFunctions');

suite('sortFunctions', function () {
var objects;
var objectsRenderOrder;
var objectsGroupOrder;

setup(function () {
objects = [
{ name: 'a', renderOrder: 0, z: 1 },
{ name: 'b', renderOrder: 0, z: 3 },
{ name: 'c', renderOrder: 0, z: 2 }
];

objectsRenderOrder = [
{ name: 'a', renderOrder: 1, z: 1 },
{ name: 'b', renderOrder: 0, z: 3 },
{ name: 'c', renderOrder: -1, z: 2 }
];

objectsGroupOrder = [
{ name: 'a', groupOrder: 0, renderOrder: 1, z: 1 },
{ name: 'b', groupOrder: 0, renderOrder: 0, z: 3 },
{ name: 'c', groupOrder: 1, renderOrder: -1, z: 2 }
];
});

function checkOrder (objects, array) {
array.forEach((item, index) => {
assert.equal(objects[index].name, item);
});
}

test('Opaque sort sorts front-to-back', function () {
objects.sort(sortOpaqueDefault);
checkOrder(objects, ['a', 'c', 'b']);
});

test('Opaque sort respects renderOrder', function () {
objectsRenderOrder.sort(sortOpaqueDefault);
checkOrder(objectsRenderOrder, ['c', 'b', 'a']);
});

test('Opaque sort respects groupOrder, then renderOrder', function () {
objectsGroupOrder.sort(sortOpaqueDefault);
checkOrder(objectsGroupOrder, ['b', 'a', 'c']);
});

test('Transparent default sort sorts in DOM order', function () {
objects.sort(sortTransparentDefault);
checkOrder(objects, ['a', 'b', 'c']);
});

test('Transparent default sort respects renderOrder', function () {
objectsRenderOrder.sort(sortTransparentDefault);
checkOrder(objectsRenderOrder, ['c', 'b', 'a']);
});

test('Transparent default sort respects groupOrder, then renderOrder', function () {
objectsGroupOrder.sort(sortTransparentDefault);
checkOrder(objectsGroupOrder, ['b', 'a', 'c']);
});

test('Transparent spatial sort sorts back-to-front', function () {
objects.sort(sortTransparentSpatial);
checkOrder(objects, ['b', 'c', 'a']);
});

test('Transparent spatial sort respects renderOrder', function () {
objectsRenderOrder.sort(sortTransparentSpatial);
checkOrder(objectsRenderOrder, ['c', 'b', 'a']);
});

test('Transparent spatial sort respects groupOrder, then renderOrder', function () {
objectsGroupOrder.sort(sortTransparentSpatial);
checkOrder(objectsGroupOrder, ['b', 'a', 'c']);
});
});
32 changes: 27 additions & 5 deletions tests/systems/renderer.test.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
/* global assert, suite, test, setup, teardown, THREE */
var {sortOpaqueDefault,
sortTransparentDefault,
sortTransparentSpatial} = require('core/scene/sortFunctions');

suite('renderer', function () {
function createScene () {
Expand All @@ -9,22 +12,26 @@ suite('renderer', function () {

test('default initialization', function (done) {
var sceneEl = createScene();
var sortFunction;
sceneEl.renderer.setOpaqueSort = function (s) { sortFunction = s; };
sceneEl.addEventListener('loaded', function () {
// Verify the properties which are part of the renderer system schema.
var rendererSystem = sceneEl.getAttribute('renderer');
assert.strictEqual(rendererSystem.foveationLevel, 1);
assert.strictEqual(rendererSystem.highRefreshRate, false);
assert.strictEqual(rendererSystem.physicallyCorrectLights, false);
assert.strictEqual(rendererSystem.sortObjects, false);
assert.strictEqual(rendererSystem.sortTransparentObjects, false);
assert.strictEqual(rendererSystem.colorManagement, true);
assert.strictEqual(rendererSystem.anisotropy, 1);

// Verify properties that are transferred from the renderer system to the rendering engine.
var renderingEngine = sceneEl.renderer;
assert.strictEqual(renderingEngine.outputColorSpace, THREE.SRGBColorSpace);
assert.notOk(renderingEngine.sortObjects);
assert.ok(renderingEngine.sortObjects);
assert.strictEqual(sortFunction, sortOpaqueDefault);
assert.strictEqual(renderingEngine.useLegacyLights, true);
assert.strictEqual(THREE.Texture.DEFAULT_ANISOTROPY, 1);

done();
});
document.body.appendChild(sceneEl);
Expand All @@ -41,11 +48,26 @@ suite('renderer', function () {
document.body.appendChild(sceneEl);
});

test('change renderer sortObjects', function (done) {
test('change renderer sortTransparentObjects', function (done) {
var sceneEl = createScene();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that Unit Tests mock out the setupRenderer function means that I was limited in how much of the code changes I could Unit Test in a straightforward way.

Should we be looking for a way to improve UT coverage of the setupRenderer function?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the configuration of the renderer is a bit split between both setupRenderer and the init function of the renderer system. It might be easier to move the setting of sortObjects and sorting functions into the init function as well. That way the tests here can:

  • Verify that the default values are set properly (currently the test 'default initialization' asserts that sortObjects defaults to false and doesn't fail)
  • Verify that changing the sortTransparentObjects result in the correct sorting functions being set

This unit test now also tests that the sorting behaviour is correct, but I would split that off into a new sortFunctions.test.js that specifically tests them and let this unit test here cover only the responsibility of the renderer system of setting the right sort functions.

Finding a way to improve test coverage of setupRenderer would still be nice, but I think with the above the need for it in the scope of this change is mostly alleviated

Copy link
Contributor Author

@diarmidmackenzie diarmidmackenzie Jul 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed some changes to address these comments...

It might be easier to move the setting of sortObjects and sorting functions into the init function as well.

The renderer init function will only be called if renderer system is initialized. It does appear that's always the case today (even if the renderer property is not explicitly specified on the scene), but I'm not sure it's good to bake that assumption into core code. So I'd argue these properties should be set by the AScene class, and setupRenderer() seems like the right place to do it.

With the updated code, renderer.sortObjects is always true, and the renderer component has no bearing on it at all, so I'd argue it is cleaner to not have the renderer code touch renderer.sortObjects.

Sorting functions are set in the renderer update function, as well as in setupRenderer(). No reason to move them to init, which just restricts them from being changed later.

currently the test 'default initialization' asserts that sortObjects defaults to false and doesn't fail

Removed that check from the test. renderer.sortObjects no longer has anything to do with the renderer component, so that check doesn't belong here any more.

This unit test now also tests that the sorting behaviour is correct, but I would split that off into a new sortFunctions.test.js that specifically tests them and let this unit test here cover only the responsibility of the renderer system of setting the right sort functions.

That's fair. Since the since the sort functions are internal to A-frame, I hadn't twigged that I can just import them, and then compare the functions directly. Have now done that and it works fine.

A full UTs of the sort functions is also a sensible idea - I have added.

Copy link
Contributor

@mrxz mrxz Jul 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The renderer init function will only be called if renderer system is initialized. It does appear that's the case today, but I'm not sure it's good to bake that assumption into core code. So I'd argue these properties should be set by the AScene class, and setupRenderer() seems like the right place to do it.

I kind of see it the other way around. It's the AScene class that is effectively overstepping its boundaries by depending on both the camera and renderer systems. Conceptually in an ECS if the renderer system would be left out, you'd be left with a headless instance. From a pragmatic and legacy perspective I can see how this situation came to be and it's now unfeasible to every change (lots of code depends on the scene holding the renderer and camera).

That does leave us in the situation where renderer initialization is effectively split in two parts AScene.setupRenderer and renderer.init. The logical split I see at the moment is that setupRenderer only sets the properties needed at WebGLRenderer construction time and renderer.init sets anything that can change 'on the fly'. The object sorting is the only odd one out (even before this PR).

That being said, the actual sortObjects property can be left default (= true) so neither needs to assign anything to this. Unless we explicitly want to assert this default (and potentially shield us against three.js ever changing that default). That only leaves the sorting functions which, given the current state of the code, feel more at home in init to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I completely agree, as most of the renderer settings can't be changed on the fly (according to the docs, at least).

https://aframe.io/docs/1.4.0/components/renderer.html

But happy this is the right direction of travel, so I made the changes you suggested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you're right, seems I overlooked the few that are set by renderer but not changeable on the fly. The docs and the schema do paint a clear picture that all these settings do belong to the renderer system, yet a good chunk of them are handled (parsed even) by the AScene, which in an ideal setup wouldn't have been the case, but alas.

Anyway, the PR looks good. Just two more ES6 destructuring assignments in renderer.test.js and sortFunctions.test.js and then it's all good to go IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding was that the ES5 restriction only applies to /src.

Plenty of existing code under /test uses let, const, arrow functions etc.

See e.g. animation.test.js

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, then everything is good to go IMO :-)

var sortFunction;
sceneEl.renderer.setTransparentSort = function (s) { sortFunction = s; };
sceneEl.setAttribute('renderer', 'sortTransparentObjects: true;');
sceneEl.addEventListener('loaded', function () {
assert.strictEqual(sortFunction, sortTransparentSpatial);
done();
});
document.body.appendChild(sceneEl);
});

test('default renderer sortTransparentObjects', function (done) {
var sceneEl = createScene();
sceneEl.setAttribute('renderer', 'sortObjects: true;');

var sortFunction;
sceneEl.renderer.setTransparentSort = function (s) { sortFunction = s; };
sceneEl.addEventListener('loaded', function () {
assert.ok(sceneEl.renderer.sortObjects);
assert.strictEqual(sortFunction, sortTransparentDefault);
done();
});
document.body.appendChild(sceneEl);
Expand Down
Loading