GHARP Breakthrough Enables Real-Time 3D Gaussian Head Animation on Mobile Devices

A team of researchers has introduced GHARP, an innovative method designed to animate three-dimensional human heads in real time utilizing just a few input images of a subject alongside a driving expression signal. Short for Real-time Gaussian Head Animation from Large-scale Reconstruction Prior, the breakthrough framework addresses several long-standing bottlenecks in digital avatar generation, achieving state-of-the-art performance on the Ava-256 benchmark while running up to thirteen times faster on an A100 graphics processing unit with significantly fewer Gaussian primitives.

The research effort brings together a broad team of authors, including Ali Benlalah, Sepehr Johari, Patricia Vitoria, Armin Kappeler, Artem Sevastopolsky, Alexander Jung, Gabriele Fanelli, Kevin Mader, Manuel Breitenstein, Claudia Plüss, Jan Rüegg, Simon Biland, Thomas Etterlin, Dmitry Kostiaev, Mathias Deschler, Brian Amberg, and Sebastian Martin. Their collective work tackles the complex trade-offs traditionally associated with digital human rendering, balancing high visual fidelity and robust geometric quality with the low latency required for mobile applications and real-time interactive environments.

Decoupling Identity and Animation for Optimal Performance

At the core of the GHARP framework is a strategic architectural decision: decoupling the avatar creation process into two distinct phases consisting of an offline identity stage and a runtime animation stage. This separation allows the system to establish a favorable trade-off regarding fidelity, quality, and computational execution speed.

During the identity stage, the system builds a comprehensive representation of the subject’s unique geometry and appearance. Because this phase occurs offline, it can handle computationally expensive processes to thoroughly capture fine details from the input data. Once this foundational identity model is established, the runtime animation stage takes over. This subsequent phase relies on a lightweight neural network that has been specifically optimized to operate efficiently on mobile devices, predicting expression-dependent residuals directly on top of the pre-computed identity representation.

To make this residual prediction efficient, the method performs animation within a semantically structured latent space belonging to a pretrained reconstruction model. Within this specialized latent space, expression changes remain spatially contained. This spatial containment ensures that modifications driven by facial movements do not inadvertently disrupt unrelated regions of the model. Furthermore, this reconstruction prior delivers a consistent spatial layout, which enables the system to seamlessly fuse multiple input views into a compact, fixed-size canonical Gaussian representation.

Addressing Body Pose and Clothing Ambiguity

Despite the structural advantages of this two-stage design, the approach initially encountered a challenge common to nearly all expression-driven avatar generation methods. Traditional expression codes are fundamentally designed to describe facial movements alone. Consequently, these codes omit broader body pose dynamics and the specific physical positioning of clothing. In scenarios where input data lacks explicit guidance for these areas, regions outside the face become underspecified.

When faced with such underspecified input, standard animation networks encounter an ill-posed mapping problem. Without sufficient data to determine exact positioning, the network typically resorts to averaging over conflicting body appearances across different frames. This averaging effect manifests visually as unwanted blur and noticeable temporal flickering, undermining the realism of the digital avatar.

To solve this persistent issue, the research team integrated a specialized body alignment network into the GHARP architecture. This component learns how to accurately align the person’s body within a target image with the corresponding input reference images. By effectively synchronizing these visual elements, the alignment network successfully removes ambiguity from the training signal, allowing the animation pipeline to render clothing and body posture with sharp clarity and temporal consistency.

Benchmarking Performance and Efficiency

When evaluated on the demanding Ava-256 benchmark, GHARP delivers state-of-the-art visual quality while demonstrating remarkable computational efficiency. The method operates up to thirteen times faster on an A100 GPU compared to conventional alternatives, achieving these performance gains while utilizing eight times fewer Gaussians. This reduction in primitive count not only accelerates rendering speeds but also makes the technology far more viable for deployment on resource-constrained hardware platforms.

The advancements introduced by GHARP build upon a broader landscape of ongoing research in neural rendering and 3D Gaussian reconstruction. Related scientific efforts at the forefront of the field continue to explore scalable feed-forward methods for handling complex multi-camera setups and dynamic environments. For instance, alternative frameworks like HeadsUp have been proposed to reconstruct high-quality 3D Gaussian heads from large-scale multi-camera captures. HeadsUp employs an efficient encoder-decoder architecture that compresses input views into a compact latent representation, which is subsequently decoded into a set of UV-parameterized 3D Gaussians anchored to a neutral head template, successfully decoupling the scale of 3D Gaussians from specific capture constraints.

Similarly, other foundational developments in neural rendering, such as Human Gaussian Splats, have vastly improved training and rendering speeds by orders of magnitude. While traditional photogrammetry techniques have historically excelled at capturing static scenes, they have frequently struggled to generalize effectively to freely moving humans within unstructured environments. Methods like HUGS address these limitations by representing animatable humans alongside their surrounding scenes using unified 3D Gaussian Splatting formulations. Within this evolving technological context, GHARP represents a significant step forward, bridging the gap between high-fidelity offline reconstruction and lightweight, real-time mobile deployment for interactive digital avatars.