GLTF edits without loading external images

[benferns]

Is there any way to open a GLTF (+ bin, if buffer-related editing occurs) without requiring the images to be present at that moment? I have some very large files being processed, and to speed things up I'd like to copy only the resources I need from s3 to the local node environment.

[donmccurdy]

There isn't currently an I/O function for that, or an internal representation of a remote texture. What you could do instead would be to manually create a JSONDocument with placeholders for the textures, then parse it with io.readJSON:

import fs from 'fs/promises';
import { NodeIO } from '@gltf-transform/core';

const io = new NodeIO();

const json = await fs.readFile('path/to/file.gltf');
const resources = {};

// Load buffers (.bin).
for (const buffer of json.buffers) {
  resources[buffer.uri] = await fs.readFile(buffer.uri);
}

// Add placeholder images.
let i = 0;
for (const image of json.images) {
  resources[image.uri] = new Uint8Array([ i++ ]).buffer;
}

const document = io.readJSON({json, resources});

Loading the .bin probably isn't avoidable, though.

[benferns]

Great thanks - this is super helpful to save me digging around to find out whats required for loading in which contexts etc. The bin part is understandable given how the byte-structure flows into all the other elements.

[benferns]

Hi - I just wanted to check if some odd behaviour I'm seeing with this approach is expected or not. I've made a simple test case without any images or other handling of the GLTF (splitting, dedup-ing etc), and I find any data referenced by animation accessors becomes invalid (specifically the sampler seems to read in chunks of other original accessors - e.g. some position values are in the output rotation keyframes with off-by-one step errors etc).

I've uploaded the test case, where you can switch the commenting on line 21/22 to see it occur.

At first I thought it was due to me splitting bins and and the array order not being preserved, but this is with a single item in the buffer array. Is it possible the order of bufferViews is changing?

animation-buffer-issue.zip

via NodeIO

via WebJSON & resource mapping

[donmccurdy]

Ah sorry, my example code above was mixing code from v0.x–v1.x and v2.x of this library, and I think that mislead you.

v0.x–1.x (stable release)

- const { Document, Scene, WebIO } = require("@gltf-transform/core");
+ const { Document, Scene, WebIO, BufferUtils } = require("@gltf-transform/core");
const { NodeIO } = require("@gltf-transform/core");
const fs = require("fs");
const io = new NodeIO();

const resources = {};
const json = JSON.parse(fs.readFileSync("test.gltf", "utf8"));

for (const buffer of json.buffers) {
  console.log("loading buffer: " + buffer.uri);
-  resources[buffer.uri] = fs.readFileSync(buffer.uri);
+  resources[buffer.uri] = BufferUtils.trim(fs.readFileSync(buffer.uri));
}

v2.x (upcoming release)

const { Document, Scene, WebIO } = require("@gltf-transform/core");
const { NodeIO } = require("@gltf-transform/core");
const fs = require("fs");
const io = new NodeIO();

const resources = {};
const json = JSON.parse(fs.readFileSync("test.gltf", "utf8"));

for (const buffer of json.buffers) {
  console.log("loading buffer: " + buffer.uri);
  resources[buffer.uri] = fs.readFileSync(buffer.uri);
}
let i = 0;
if (json.images && json.images > 0) {
  for (const image of json.images) {
-     resources[image.uri] = new Uint8Array([i++]).buffer;
+     resources[image.uri] = new Uint8Array([i++]);
  }
}

//switch between loading methods to test issue
- const document = io.readJSON({ json, resources });
+ const document = await io.readJSON({ json, resources });
- // const document = io.read("test.gltf");
+ // const document = await io.read("test.gltf");

If you'd rather switch to the v2.x syntax sooner you can test that with:

npm install @gltf-transform/core@next

[benferns]

Just had a chance to try the updated syntax, and everything's as expected now, thanks!