REFACTOR-VLA Framework Introduces Wake-Sleep Architecture to Solve Long-Horizon Bottlenecks in Vision-Language-Action Models

The field of Embodied AI and robotics has made tremendous strides in recent years, largely driven by the development of Vision-Language-Action (VLA) models. Systems such as OpenVLA, $pi_0$, RT-2, and RDT-1B have demonstrated an impressive ability to interpret visual scenes and natural language instructions to generate basic robotic behaviors. However, a fundamental limitation has continued to hold back these state-of-the-art models: they are predominantly "monolithic" in design.

This architectural constraint means that current VLA models generate raw motor commands or exceptionally short sequences of actions directly, completely failing to organize complex behaviors into reusable, well-defined abstractions. Consequently, these systems perform poorly when confronted with long-horizon, multi-step tasks that require sustained planning and execution. Furthermore, this monolithic nature makes it exceptionally difficult for researchers to interpret what the models have actually learned during training.

To address these critical shortcomings, researchers Riyaaz Shaik and Chandru Venkataraman have introduced REFACTOR-VLA, a novel system designed to learn reusable skills through a biological-inspired "wake/sleep" architecture. The approach aims to bridge the gap between high-level language understanding and low-level motor execution by decomposing complex trajectories into manageable, interpretable program abstractions.

Overcoming the Behavioral-Equivalence Problem

In the realm of robotics, discovering discrete skills from continuous demonstration data has long been hampered by a core ambiguity: deciding when two distinct action sequences are genuinely "behavioral-equivalent." Prior attempts to tackle this problem have often fallen short. For instance, frameworks such as AtomicVLA and AtomSkill attempt to group action sequences by clustering their contrastive embeddings, a method that can struggle to capture the true dynamic outcomes of a physical movement.

On the other hand, systems like BLADE and LRLL rely heavily on off-the-shelf large language models (LLMs) to act as judges determining whether two action sequences achieve the same functional outcome. While flexible, these general-purpose LLMs are inherently uncalibrated to the specific physical dynamics, payloads, and kinematics of the target robot, leading to semantic judgments that may not hold up in physical execution.

REFACTOR-VLA bypasses these limitations by anchoring skill discovery in a grounded, learned latent world model, designated as $mathcalM_varpi$. By leveraging this internal dynamics model, the system can evaluate actions based on their actual simulated outcomes rather than relying solely on static embeddings or uncalibrated linguistic priors.

The Wake/Sleep Architecture and Skill Discovery

The REFACTOR-VLA framework operates on a continuous, dual-phase paradigm divided into a sleep phase and a wake phase, mirroring biological processes of consolidation and execution.

During the sleep phase, the system focuses on skill discovery. It clusters segments of motor programs using a specialized Behavioral-Equivalence Kernel (BEK). This kernel is fundamentally grounded in the outcomes of rolling out actions within the learned latent world model, $mathcalM_varpi$. By observing how trajectories unfold in this simulated latent space, the system can reliably group functionally equivalent behaviors into reusable skill primitives.

Conversely, during the wake phase, the system shifts its operational focus to policy optimization and execution. Here, REFACTOR-VLA generates typed lambda terms—representing simple, highly structured programs—drawn from a carefully curated vocabulary inspired by the classical Hindley-Milner type system. These lambda terms are then passed to a library-conditioned rectified-flow action decoder, which translates the structured abstractions into precise, executable motor commands.

Crucially, REFACTOR-VLA implements strict quality controls to prevent the proliferation of redundant or unhelpful behaviors. Only abstractions that successfully pass a rigorous Minimum Description Length (MDL) criterion alongside a return-preservation gate are formally accepted into the system’s growing library of reusable skills.

A Three-Phase Training Schedule

To effectively train the intricate components of REFACTOR-VLA, the researchers established a deliberate three-phase training schedule. Phase A serves as the world-model warmup, during which the latent world model $mathcalM_varpi$ is trained to accurately capture the environment’s dynamics. Phase B handles wake-phase policy optimization, fine-tuning the policy responsible for leveraging the established library of skills. Finally, Phase C executes the sleep-phase skill discovery, clustering action fragments into robust, reusable skill primitives based on the refined world model.

Evaluating the framework on the comprehensive LIBERO benchmark suite, the researchers uncovered critical empirical insights regarding scale and optimization objectives within embodied world models.

Benchmark Evaluation and Surprising Empirical Findings

The evaluation of REFACTOR-VLA across the LIBERO benchmark suite yielded two primary findings that challenge prevailing assumptions in robotic learning.

The first finding directly challenges the widespread industry heuristic that simply scaling up model size universally improves performance. When the researchers increased the size of the world model from 188 million parameters to 430 million parameters, performance actually worsened across four out of four tested benchmark suites. This outcome demonstrates that merely making the world model larger does not automatically translate to better downstream task execution or skill acquisition, highlighting the need for more efficient architectures rather than brute-force scaling.

The second finding underscores the vital importance of training objectives. The researchers discovered that introducing an auxiliary supervised contrastive loss—specifically, the InfoNCE loss—during the world-model warmup in Phase A dramatically improved the quality of skill clustering during the sleep phase in Phase C.

To rigorously quantify the quality of the learned skill clusters, the research team measured Normalized Mutual Information (NMI) under $n = 3$ multi-seeding protocols across the benchmark suites. The results demonstrated high clustering fidelity, recording an NMI of $0.462 pm 0.021$ on the Object suite, $0.867 pm 0.025$ on the Spatial suite, $0.915 pm 0.013$ on the Goal suite, and $0.754 pm 0.010$ on the LIBERO-10 suite. These high NMI scores validate that the combination of the latent world model, the Behavioral-Equivalence Kernel, and the auxiliary InfoNCE loss successfully guides the system toward discovering meaningful, distinct behavioral abstractions necessary for mastering complex, multi-step robotic tasks.

In parallel with advancements in modular skill discovery, ongoing research in Embodied AI continues to explore adjacent challenges. These include grounding multimodal large language models across diverse robotic embodiments and action spaces, as well as optimizing computational efficiency through parallelized causal inference techniques like KV-Runahead to accelerate prompt and decoding phases in large-scale deployments.