Professionals in the animation industry know that AI character animation has always had an unavoidable problem. In the past, the vast majority of action transfer schemes required extracting poses, bones, and keypoints first. Once the character's angles were complex and micro expressions were rich, it was easy to experience identity drift and action distortion, and it was even more difficult to do well with multiple characters in the same frame.
The release of Ali Wanxiang WanAnimate-2 directly changed the problem-solving approach, completely removing the intermediate skeleton extraction step and creating end-to-end character animation generation, with complete open-source weights, code, and technical documentation.
According to a tweet released by Alibaba Wan on the official account of "X", it is announced that the new generation of character animation framework Wan-Animate-2 is officially open source to the public. In addition to the basic version, a lightweight variant Wan-Animate-2-Lite will be launched simultaneously, targeting the real-time streaming generation needs of digital live streaming and virtual interactive scenes. Developers can obtain the complete resource package on ModelScope, HuggingFace, and GitHub.
Why abandon bone extraction? The practical shortcomings of traditional solutions
Many people have been exposed to the first generation of character animation models, and the basic working process is: reference image+driving video → motion extractor parsing skeleton posture → action redrawing to the target character. The shortcomings of this plan are very practical:
Serious loss of details: Bone extraction itself filters out a large amount of information, and micro expressions such as blinking and changes in the corners of the mouth are easily overlooked
Poor adaptation to non-human objects: non-standard human structures such as cartoon characters, robots, animals, etc., and bone adaptation often encounters obstacles
Difficulty in having multiple characters in the same frame: Multi person scenes require splitting and rendering multiple times, which is inefficient and difficult to ensure consistency
Wan-Antimate-2 core breakthrough: end-to-end generation, two intuitive benefits
The biggest change of Wan-Animate-2 is to completely abandon the explicit pose skeleton extraction process and directly input the potential representations of the driving video into the redesigned diffusion Transformer (DiT), allowing the model to learn motion patterns from the original video on its own, without relying on intermediate pose modules for translation and transformation. This brings two intuitive benefits:
High fidelity character restoration
Humans, cartoon characters, robots, and animals can all transfer movements, with more complete retention of micro expressions and subtle hand movements, significantly improving character identity consistency.
Natural support for multi character animation
A single round of generation can handle multiple subjects within the image, with each character retaining independent identities and actions, without the need to split and render multiple times.
Bottom layer architecture: supported by three core technologies
| Technical module | core mechanism | problems solved |
|---|---|---|
| Dual branch DiT architecture | Fixed at time t=0, the reference branch outputs motion related Key Value features to the denoising generation branch | Avoid continuous attenuation of motion information during noise reduction process |
| Time aligned RoPE | The reference token and the target token share the same set of time streams | When the resolution before and after is inconsistent, the temporal correspondence between frames remains stable |
| Sparse reference attention | Each target frame token only establishes attention calculation with reference tokens aligned in time sequence | Significantly reduce memory usage and computational overhead without compromising image quality |
Highlight feature: Text control camera angle
In the past, when transferring actions, the output camera angle was tightly bound to the driving video. If you want to switch to top-down, surround, or side shooting, you have to prepare a new driving material.
Wan-Animate-2 trained the perspective LoRA and completed the training based on approximately 50K Unreal Engine multi perspective samples, transforming the position control into a text task. Developers can directly input prompt words such as "top-down perspective" and "slow surround camera" to decouple the camera perspective of the generated image from the original driving video. The same set of action materials can output multiple shot versions, which is highly practical for animation storyboarding and short video creation.
Wan-Antimate-2-Lite: Targeting Real time Interactive Scenarios
If the basic version tends to produce high-quality offline animations, Wan-Animate-2-Lite is specifically designed for online interactive scenes:
After model distillation and inference optimization, the inference delay is reduced to the available threshold for real-time streaming character animation
Capable of adapting to latency sensitive services such as live streaming of digital humans and real-time virtual interaction
No need to wait for the complete video rendering to be completed, achieve simultaneous inference and output of frame images
The design concept of Wan-Animate-2 is to minimize the use of manually defined intermediate representations, allowing the base model to directly learn motion, expression, and shot information from the original video, reducing information loss caused by upstream preprocessing.
Objective limitations and industry value
Of course, we also need to objectively see the limitations:
The real-time Lite version still has certain requirements for GPU hardware
Complex multi character extreme scenes, occasionally with minor body flaws that still need to be fine tuned and repaired later
We cannot deny that Wan-Animate-2 has provided a complete and practical end-to-end technology baseline for the domestic open source character animation field. High fidelity identity preservation, multi character frame synchronization, text decoupled camera operation, real-time streaming lightweight version, several capabilities are concentrated in the same open source project. Whether it is independent creators creating secondary animation, or enterprises building digital humans and virtual live streaming pipelines, they can directly develop based on open source weights.
