Project page · 2026

LiveAnimate Stable Long-Form Streaming Human Animation in Real-Time

Yuxuan Zhang^1,2 Haozhong Xiong² Yubo Huang² Jiayi Song² Jinpeng Yu² Haofan Wang³ Jiaming Liu² Ruihua Huang² Liwei Wang¹

¹ The Chinese University of Hong Kong ² Qwen Applications Business Group of Alibaba ³ Liblib AI

LiveAnimate turns a reference image and streaming pose controls into stable animation over long durations. — **LiveAnimate enables real-time, stable long-form streaming human animation.** Given a reference image and a stream of body-pose and facial controls, LiveAnimate generates identity-consistent animation causally, one temporal block at a time. Pose-Retrieval Sink Attention recalls pose-relevant historical context when similar poses recur, preserving appearance over extended streams. With three-step sampling, LiveAnimate runs at ~20 FPS on two NVIDIA H100 GPUs while maintaining stable long-form generation.

Abstract

Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip. We present LiveAnimate, to our knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer. A two-stage training pipeline turns a pretrained bidirectional DiT into a block-causal three-step generator, and Pose-Retrieval Sink Attention keeps appearance consistent over arbitrarily long streams with bounded memory. The system streams at 19.63 FPS on two H100 GPUs, and on a three-minute benchmark its perceptual quality and identity are nearly constant from the first 30 seconds to the final minute (IQA 4.047 vs. 4.026).

Motivation

Virtual try-on, digital humans, and telepresence all want the same thing: a reference person animated by a live pose stream, with appearance, pose accuracy, and temporal coherence intact. Video diffusion models do this well for short clips, but every prior full-body system is offline — fixed-length outputs, minutes to hours per clip, and no mechanism for an open-ended pose stream. Closing the gap means solving three coupled problems at once:

Billion-scale latency

A 14B-parameter DiT must meet an interactive per-block latency budget.

Bidirectional → causal

The pretrained bidirectional model must become a causal generator without losing quality.

Long-horizon drift

Identity and appearance must survive arbitrarily long rollouts, not just short clips.

Method

LiveAnimate adapts a pretrained bidirectional video DiT into a block-causal streaming generator, distills it to three denoising steps per block, and then keeps long streams stable with a bounded, pose-aware KV cache — so memory and per-block latency stay constant no matter how long the stream runs.

LiveAnimate pipeline: reference-anchored teacher-forcing adaptation, block-wise self-forcing distillation, and pose-retrieval sink attention at inference. — **Overview of LiveAnimate.** Given a reference image and streaming pose signals, our system generates video blocks autoregressively. Each block undergoes 3-step denoising followed by a clean KV update. PR-Sink augments a three-block rolling window with the first generated block and a pose-matched historical block selected from a compact memory bank. Ulysses sequence parallelism distributes attention computation across GPUs.

Stage 1 · Reference-anchored teacher forcing

Each training block is conditioned on ground-truth clean history, converting the bidirectional DiT into a block-causal generator. A global Ref Sink keeps the reference-image latent visible to every generated block.

Stage 2 · Block-wise self-forcing distillation

A gradient-free self-forcing rollout is followed by block-wise DMD optimization, one replayed block at a time. Every block position sees a distribution-matching signal without retaining the full rollout graph, so 14B distillation fits on a single 8×80GB node and sampling drops to 3 steps.

Pose-Retrieval Sink Attention (PR-Sink)

A bounded KV cache with three parts: a Static Sink anchoring the first generated block, a Dynamic Sink holding the pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, its appearance context is restored — without keeping the whole sequence.

Systems co-design

Ulysses-style sequence parallelism and operator fusion distribute attention across GPUs, reaching 19.63 FPS streaming inference on two NVIDIA H100s — real time at billion scale.

Demo

Each demo pairs the driving pose stream (left) with the LiveAnimate output (right), generated causally from a single reference image. Only the input signal is constructed: to simulate long natural-motion sessions, the 30-minute driving signals are built from short source clips (about 10 s–3 min) whose pose controls are looped forward–reverse–forward (ping-pong concatenation). The output is not stitched — LiveAnimate generates the entire half hour as one continuous causal rollout. The four long-form rollouts are shown at 10× speed; the four ~3-minute rollouts are trimmed to their first 20 seconds at natural speed. Click a video to play.

Reference image: dancer in a street scene

Street dance · full body

36.0-minute rollout shown at 10× speed — the longest case, with fast full-body motion over a detailed outdoor scene.

10× timelapse

Reference image: speaker talking to camera

Talking monologue · upper body

30.0-minute rollout shown at 10× speed — face, clothing, and background remain consistent throughout.

10× timelapse

Stage performance · full body

28.8-minute rollout shown at 10× speed — appearance stays locked to the reference from the first minute to the last.

10× timelapse

Reference image: man giving a talk on a dark stage

Stage talk · upper body

28.8-minute rollout shown at 10× speed — identity stays sharp against a dark stage background.

10× timelapse

Shorter rollouts · first 20 seconds

Four ~3-minute rollouts, each trimmed to its first 20 seconds at natural speed — still one continuous rollout, useful for watching motion quality up close.