TR2026-127

LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models


    •  Yu, B.-Y., Peng, K.-C., Hsieh, J.-W., "LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models", European Conference on Computer Vision (ECCV), September 2026.
      BibTeX TR2026-127 PDF
      • @inproceedings{Yu2026sep,
      • author = {Yu, Bo-Yun and Peng, Kuan-Chuan and Hsieh, Jun-Wei},
      • title = {{LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models}},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2026,
      • month = sep,
      • url = {https://www.merl.com/publications/TR2026-127}
      • }
  • MERL Contact:
  • Research Areas:

    Artificial Intelligence, Computer Vision, Machine Learning

Abstract:

Vision-Language-Action (VLA) models have advanced rapidly, yet most gains still come from larger backbones, larger embodied datasets, or expensive iterative decoders, while action representation remains under-explored. We argue that action space design is a primary bottleneck and present LEAP (Latent-Enhanced Action Prototyping)-VLA, a two-stage framework that learns a structured action latent space from demonstrations via Multi-level Soft Residual Quantization (MSRQ), then trains a lightweight VLM-external aligner to predict actions by prototype-aware soft selection in that space. This design replaces discrete code assignment and multi-step denoising with fully differentiable single-pass latent prediction, preserving pretrained visionlanguage alignment without modifying the VLM. Empirically, LEAPVLA outperforms the state-of-the-art methods with substantially fewer trainable parameters and no embodied pretraining, showing that a wellstructured continuous action latent space can offset model scale and data requirements.