Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts  

Yue Ma1* Yingqing He1* Hongfa Wang2,3* Andong Wang2 Chenyang Qi1
Chengfei Cai2 Xiu Li3 Zhifeng Li2 Heung-Yeung Shum1,3 Wei Liu2✝ Qifeng Chen1✝
*Equal Contribution. Corresponding Author.
1HKUST 2Tencent, Hunyuan 3Tsinghua University

[Paper]     [Github]     [BibTeX]


Click to play result from Follow-Your-Click!

User Click

Case 1 Image

Output

User Click

Case 2 Image

Output

User Click

Case 5 Image

Output

"Tune the head"

"Flap the wings"

"Storm"

Case 1 Image
Case 2 Image
Case 5 Image

"Smile"

"Sad"

"Launch"

Case 1 Image
Case 2 Image
Case 5 Image

"Drift"

"Dancing"

"Drive back and forward"

Gallery

Here we demonstrate more animation results via our framework.

Click to play result from Follow-Your-Click!

"Fire"

"Tune the body"

"Launch down"

"Driving forward"

"Flow"

"Tune the body"

"Shake the body"

"Dancing"

"Running"

"Driving"

"Dancing"

"Running"

"Angry"

"Go forward"

"Running"

"Flying"

"Tune the body"

"driving back"

"Dancing"

"Dancing"

"Shake the body"

"Dancing "

"Ear wiggle"

"Angry"

"Drift"

"Blink"

"Shake the body"

"Sad"

"Astonish"

"Blink"

"Driving back"

"Storm"

"Happy"

"Blink"

"Raise head"

"Swimming"

"Nod"

"Dancing"

"Blink"

"Blink"

"Close the month"

"Shake head"

"The wind is blowing"

"Drift"

"Walking"

Comparisons

Here we demonstrate the animations generated by different methods.
We qualitatively compare our approach with the most recent open-sourced state-of-the-art animation methods, including Animate anything, SVD, Dynamicrafter and I2VGen-XL. We also compare our approach with commercial tools such as Gen-2, Genmo, and Pika Labs.

Click to play the following animations!


Motion Strength Control

Here we demonstrate the comparisons between our optical flow motion magnitude control (OFM) and FPS-based motion magnitude control (FPS).

Click to play the following animations!


Ablation Study

Here we demonstrate the qualitative results of ablation the constructed short prompt dataset (D) and motion-augmented module (M). The motion prompt is “running”.

Click to play the following animations!

Limitation

Our approach is limited in generating large and complex human motions, as shown in the video.This maybe due to the complexity of the action and the rareness of related training samples.

Click to play the following animations!

Project page template is borrowed from DreamBooth.