Implement sprite sheet animation

[ThatNintendoNerd]

Currently, all the viewport displays of sprite sheet animations is rescaling the UVs depending on the state of CustomBoolean9, but attempting to play an animation containing manual sprite sheet animation or automatic sprite sheet animation from materials results in no relevant playback. No other renderer for Smash Ultimate models has implemented sprite sheet animation to my knowledge, so this is something I would like to see finally implemented into a tool.

I would be interested in implementing this if I could get an overview of the areas that need work done to them. I completely understand the material parameters for sprite sheet animations and how UV scale and offsets are calculated. After going over some of the animation and rendering code in both SSBH Editor and ssbh_wgpu, I get the impression that most of the code to handle the playback will be by ssbh_wgpu, but I could be wrong.

• sprite sheet material animations
• sprite sheet animations using current frame value

[ScanMountGoat]

Have you figured out the exact math for updating the UV coordinates? You need to convert a 1D sprite sheet index value to a 2D UV coordinate change. This would also need math to differentiate between when to use the elapsed time and when to use the baked keyframe values.

This would need to be handled entirely within ssbh_wgpu. Please keep all discussion on the ssbh_editor repo for now. Implementing this would involve changes to the uniform buffer definitions to allow accessing the current frame from the shaders. The current frame value also needs to be editable from the main SsbhRenderer type in ssbh_wgpu. The final step is to edit the vertex shader to handle the sprite sheets.

I don't think the renderer and uniform buffer changes are beginner friendly. Let me know if you want to try implementing the vertex shader changes in src/shaders/model.wgsl for ssbh_wgpu. You should be able to test on manually animated spritesheet animations right now with just shader edits. You would run the code from the ssbh_wgpu repo as cargo run -p ssbh_wgpu_viewer <path to the model folder> <animation file path>. Hit space to play/pause.

[ThatNintendoNerd]

Have you figured out the exact math for updating the UV coordinates? You need to convert a 1D sprite sheet index value to a 2D UV coordinate change. This would also need math to differentiate between when to use the elapsed time and when to use the baked keyframe values.

With automatic sprite sheet animation (CustomBoolean9 == false), the logic is as follows:

spriteIndex = (framesPerSprite / floor(currentFrame)) % spriteCount;
u_offset = (1.0 / columnCount) * (spriteIndex % columnCount);
v_offset = (1.0 / rowCount) * (spriteIndex / columnCount);

The value of currentFrame could be the current baked animation frame index or the elapsed time, since performing a floor on the variable would essentially act like a step to the next frame so no interpolation takes place.

However, if CustomBoolean9 is true—which allows us to select a sprite index—the logic is almost identical:

spriteIndex %= spriteCount;
u_offset = (1.0 / columnCount) * (spriteIndex % columnCount);
v_offset = (1.0 / rowCount) * (spriteIndex / columnCount);

I don't think the renderer and uniform buffer changes are beginner friendly. Let me know if you want to try implementing the vertex shader changes in src/shaders/model.wgsl for ssbh_wgpu. You should be able to test on manually animated spritesheet animations right now with just shader edits.

For now, I think I will just stick to trying to implement sprite sheet animations handled in animation files rather than materials so I only have to deal with modifying the shader code. Do you think it would be worth making a new issue to allow for automatic animation from materials and rename this issue to fit the current goal of getting it to work from just animation files, or am I misunderstanding the middle paragraph?

[ScanMountGoat]

That logic looks right. The shaders are written in WGSL, which has a similar syntax to Rust or Javascript. The WGSL spec is still a WIP, so let me know if you run into any weird errors. For x % y for floating point numbers, I'm pretty sure you'll have to write your own function.

fn float_remainder(x: f32, y: f32) -> f32 {
    return x - y * floor(x / y);
}

You can find the related shader code here:
https://github.com/ScanMountGoat/ssbh_wgpu/blob/d3fa3133e28b80796967103ea1f6b2e6be9197fb/ssbh_wgpu/src/shader/model.wgsl#L652-L654

It's fine to keep both automatic and manual animation of sprite sheets in this issue. Let me know if you run into any issues, get stuck, etc. WGSL and WGPU generally have pretty good error messages, but they can still be a little hard to debug compared to Rust.

[ThatNintendoNerd]

Manual sprite sheet animation has been implemented into the shader code. The trampoline utilizes manual sprite sheet animation, while the rest of the enemies in the background use automatic sprite sheet animation, so they don't animate yet.

Expand for GIF

I found out that I made a couple of errors in my logic, the first being that the sprite index must be subtracted by 1 if CustomBoolean9 is enabled since it's not zero-indexed. In addition, the spriteIndex / columnCount calculation had to be floored since floating-point division doesn't result in an integer (type casting might work too, but I'm not sure if that'd be more efficient). Somewhat related, but the modulo operator does appear work as intended with floating-point numbers.

One thing I am noticing is that the sprite sheet animation appears to lag behind the transform animation by one frame. I'm not sure if this is a problem with my implementation or a problem with ssbh_wgpu. Am I able to log values to the console from shader code or is this not possible?

[ScanMountGoat]

Nice work! I wouldn't worry too much about making the code efficient. A lot of systems will be CPU bound rather than GPU bound anyway since the viewport is relatively small. The measured bottlenecks aren't always in the places you might think. Most of the slowdowns are in texture loads, but that would require making lots of specialized shaders that each load fewer textures.

One thing I am noticing is that the sprite sheet animation appears to lag behind the transform animation by one frame. I'm not sure if this is a problem with my implementation or a problem with ssbh_wgpu. Am I able to log values to the console from shader code or is this not possible?

The transforms and material parameters all get updated every frame in ssbh_wgpu while animating. The renderer uses interpolation on vectors and transforms to avoid linking the framerate to the animation playback speed. This may cause some discrepencies due to how the time from the previous frame is measured. You may also be off by one somewhere in your shader code. Do you have a specific frame of an animation that doesn't match up with in game?

[ScanMountGoat]

One other trick you can try for debugging is to hardcode the current frame in the main.rs file of ssbh_wgpu_viewer to rule out any animation frame interpolation issues.

[ThatNintendoNerd]

It seems the bug lies with how ssbh_wgpu_viewer renders both material and visibility animations. I tweaked the program to allow me to advance in the animation by a single frame and was able to understand the situation better:

• Opening a model in the program renders the model in its rest pose with default visibility and material data not yet applied from the animation file.
• Progressing one frame into the animation advances the transform tracks by one frame but does not advance the visibility or material tracks yet. The model's visibility and material components are initialized with those from the first animation frame index by this point.
• Progressing one more frame advances all the tracks, resulting in the visibility and material tracks to lag behind the transform track by one frame. The transform tracks are on frame index 2 by this point, while the material and visibility tracks are on frame index 1.

However, this problem does not occur in SSBH Editor, even with the tweaked shader code. The manual sprite sheet animation renders at the correct time, too. I cannot currently explain why this is happening.

[ScanMountGoat]

@ThatNintendoNerd I can merge what you have so far if you make a fork and submit a pull request. If something doesn't work or you at least think it doesn't work, you can just leave a TODO comment in the code. There may be some differences in how frames are set in ssbh_wgpu vs ssbh_editor. The animation code is identical of course. Visibility tracks don't interpolate, so there may also be a difference in which frame index gets used. It's not directly related to this issue, so it doesn't need to be a blocker for merging the changes you have so far.

[ThatNintendoNerd]

I've been meaning to get around to pushing my current work, I've just been busy and preoccupied with other things. I can take care of that sometime later. Through my brief testing, what I've implemented appears to work correctly, and the aforementioned issues only affect ssbh_wgpu which wasn't the reason I held off on pushing my work.