How to reduce jitter in rendered results

[ak01user]

Hi!

Amazing work!

Is there any smoothing operation?

[filby89]

Hi !
In SMIRK we did not consider any kind of temporal adjustments or smoothing operations - the method was built to be applied in separate images.

[akk-123]

@filby89 if we want use smirk in video, is there any way to improve the smoothness of the video?

[georgeretsi]

Hi there, there are two ways to tackle this: you can use a post-processing temporal filter that smooths the output of consecutive frames, or you can retrain the smirk pipeline with a temporal convolution using video data as done in our previous work Spectre (https://github.com/filby89/spectre).

Note that if you want to remove jittering in the image reconstruction space, a re-training for temporal consistency is needed, along with selecting a specific set of "anchor" face pixels during inference. This is something we plan to explore and resolve in the future.