Running Skottie animations inside of a skia canvas, mask, shader, etc.

[merveillevaneck]

I have been playing around with animations using skottie and skia separately. In skia, I managed to run a shader over a video to turn black pixels transparent, which is useful if you cant load webm vp9 files in react native. However, I was wondering if it is possible to get a Skottie player running inside of a Skia canvas the same way other elements do.

Since Skottie is actually just lottie with some Skia under the hood, it doesnt seem like it would be outright impossible? I don't know enough to even know where to begin researching this. but lets say you could:

• Run a lottie.json file inside a skottie view
• put the skottie view inside the canvas
• mask the skottie view so that it only displays within a skia Path as boundaries

That would open up the stage for using Skottie's solid transparency and playback support without the need to use SkSL and useVideo to mutate video frames in real time (this would also improve perf a lot for use cases like this)

(P.S. most of this is coming from me needing to embed animations in a Skia mask, without being able to load webm vp9 easily. so using mp4 to achieve my "transparent video" feature). I'm not happy with the outcome though as I lose out on a lot of vital pixel data because of using a luminance based custom shader.)

[wcandillon]

We would like to seriously consider this (cc @fdecampredon who has also expressed interest for this and @hannojg the maintainer of react-native-skottie). To do so, we would need to know the impact of adding skottie to the bundle size.
Instructions on setting up Skia for development: https://github.com/shopify/react-native-skia?tab=readme-ov-file#library-development
Please add Skottie here for Android: https://github.com/Shopify/react-native-skia/blob/main/package/android/CMakeLists.txt#L13
And here for iOS: https://github.com/Shopify/react-native-skia/blob/main/package/react-native-skia.podspec#L38

From there, can you report back the increase on the bundle size based on: https://shopify.github.io/react-native-skia/docs/getting-started/bundle-size

Please note that the iOS figure in the documentation might not be up to date anymore. I am excited for this.

[merveillevaneck]

Hi! I’m really happy to hear that this is something that would be considered valuable. No idea if this is useful at all, but after some digging I did find something interesting: The skottie API export allows you to create an instance of a skottie animation, which has a render function. This render function can accept a canvas instance. The only blocker I ran into was getting a reference to the SkCanvas from skia. While I can access the SkiaViewRef, I am not able to gain access to the SkCanvas instance itself. Is there a way to do this? If so, running skottie inside an existing canvas could be just a few methods away. I have no idea if my analysis is correct or not. What is the main obstacle to making the skottie and skia instances share a canvas? P.S. I’m happy to help with any of this as i love what skia can do, but don’t have much experience with native module development. So I may need some help understanding what to do 😂

…

On Tue, 09 Jul 2024 at 13:01, William Candillon ***@***.***> wrote: We would like to seriously consider this (cc @fdecampredon <https://github.com/fdecampredon> who has also expressed interest for this and @hannojg <https://github.com/hannojg> the maintainer of react-native-skottie). To do so, we would need to know the impact of adding skottie to the bundle size. Instructions on setting up Skia for development: https://github.com/shopify/react-native-skia?tab=readme-ov-file#library-development Please add Skottie here for Android: https://github.com/Shopify/react-native-skia/blob/main/package/android/CMakeLists.txt#L13 And here for iOS: https://github.com/Shopify/react-native-skia/blob/main/package/react-native-skia.podspec#L38 From there, can you report back the increase on the bundle size based on: https://shopify.github.io/react-native-skia/docs/getting-started/bundle-size Please note that the iOS figure in the documentation might not be up to date anymore. I am excited for this. — Reply to this email directly, view it on GitHub <#2523 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD43LPDWRKDJ2R5QCVMK2DTZLO7HXAVCNFSM6AAAAABKQXRGFSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TSOJXGMYTG> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[wcandillon]

this is how you can get a canvas:

  const surface = Skia.Surface.MakeOffscreen(200, 200)!;
  const canvas = surface.getCanvas();

But I don't think this will help you. I would like to get a sense of how much it would increase the library size before moving in providing the Skottie API directly.

[merveillevaneck]

I see exactly what you mean. Even if I drew the skottie animation on an offscreen canvas I couldnt really show the animation ona SkiaView unless I used snapshots as frames and ran them through an Image component on the UI thread. Even then, I doubt it would actually work as intended.

Is the plan then to include a Skottie specific element in the RN Skia exports? Thats the smallest change I can think of that would make this feasible, since Skottie is already drive-able with reanimated state.

Let me know if there is anything I can do to help 🙌 I love this package and wouldn't mind contributing some form of value, no matter how small.

[wcandillon]

If you can provide us information on the size increase by adding Skottie and the size increase is reasonable (and I suspect it might be), then we will ship it part of this library. In the first answer, I provided pointers on how to do it and the kind of data we are looking at. Please let me know if you have any questions.

[merveillevaneck]

Thanks @wcandillon I'll be giving this a go this weekend. Will circle back to you if I have questions :) Thanks for the instructions!