Cannot play animations from Mixamo

[george-lim]

Hey there Asny! First of all, just wanted to point out that I love this project. It's definitely the most WASM-friendly renderer that I have worked with in Rust.

I'm having trouble getting any animation from mixamo playing with three-d.

The usual flow I use to get animations working is:

1. Go to Mixamo, pick any animation and export the the animation with skin, 30 FPS, and no keyframe reduction. The exported format is an FBX binary.
2. Go to blender and re-export the FBX binary into GLTF 2.0 as a binary.

I have this animation loaded and playing properly in Bevy. Attached is a zip that contains the animation files, in case you want to see the input files that I'm working with. If you prefer not to open the zip file for security purposes, you can also repeat the steps above and probably get the same result.

Idle-animation.zip

While I don't want this to turn into a debugging session, I also really need to get animations working. I tried looking into your animations example in three-d, and noticed that you used COLLADA2GLTF. I tried doing the same, having mixamo export to a Collada format and then using the tool, but I get the same result.

I also verified that I am following the instructions for importing the asset, choosing the animation, and animating it on screen with a time, etc.

I would be very grateful if you could help me figure out a process on how to get these Mixamo animations working with the renderer, but if that's a lot of work, perhaps you could give me some step on how you created the BoxAnimation animation for the animation example so that I could try to see what steps I'm missing.

I have tried a bunch of different configuration options across multiple days, and nothing worked for me.

[george-lim]

Hmm after struggling with this for a few more days, I think the problem might be that this library assumes the animation must be tied to the mesh directly. I'm still new to the world of 3D models, but the Mixamo FBX files look like the animation is attached to an Armature which is the parent of the meshes.

This is the structure of the BoxAnimation file in your examples:

And this is what Mixamo generates:

I tried exporting the animation as NLA tracks but it doesn't get recognized at all 🤔.

Do I seem to be on the right track here? I'm trying to look into the bevy source code too because they have this animation working, looks like they are also using the gltf crate under the hood link

[asny]

Thank you for the kind words 🙏 And sorry for the crazy late reply, I've been very busy. This is also not an easy question to answer so I wanted to spend some time debugging this before I did answer you, but it seems like I never get the time, so I thought it was about time I gave you some answer at least.

Basically, animation is a fairly recent addition to three-d and it is by no means complete. As the very first version, I focused on simple animations from the gltf example files, I haven't tested on anything else. There's also no support for skinning or morphing yet. I hadn't worked with animated meshes when I did this implementation, so I was not aware of the structure of different formats (there's definitely a lot of different ones). That's why I just wanted to start with support for the simple animations in the gltf example library and then take it from there. Since then, I changed jobs and I'm actually working with animations now, so I have a much better understanding. Unfortunately, I also have much less time.

I definitely think you're on the right track, the box animation and similar gltf example animations are probably very simple. So it's probably because I assumed a wrong structure from those files. I do seem to remember that this was working for animations of the parent node though.

I'm not sure when I get the time to look at it, but when I do, I thank you for the example file, that's going to help me a lot 👍 If you have time and energy to try to solve this, I'll be more than happy to review a PR or guide you in the right direction if this requires architectural changes.