TR2026-127
LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models
-
- , "LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models", European Conference on Computer Vision (ECCV), Favaro, P. and Kukelova, Z. and Maki, A. and Rohrbach, A. and Schindler, K. and Tombari, F., Eds., DOI: 10.1007/978-3-032-37450-9_20, September 2026, vol. 17034, pp. 367–385.BibTeX TR2026-127 PDF
- @inproceedings{Yu2026sep,
- author = {Yu, Bo-Yun and Peng, Kuan-Chuan and Hsieh, Jun-Wei},
- title = {{LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models}},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2026,
- editor = {Favaro, P. and Kukelova, Z. and Maki, A. and Rohrbach, A. and Schindler, K. and Tombari, F.},
- volume = 17034,
- pages = {367–385},
- month = sep,
- publisher = {Springer},
- doi = {10.1007/978-3-032-37450-9_20},
- issn = {0302-9743},
- isbn = {978-3-032-37449-3},
- url = {https://www.merl.com/publications/TR2026-127}
- }
- , "LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models", European Conference on Computer Vision (ECCV), Favaro, P. and Kukelova, Z. and Maki, A. and Rohrbach, A. and Schindler, K. and Tombari, F., Eds., DOI: 10.1007/978-3-032-37450-9_20, September 2026, vol. 17034, pp. 367–385.
-
MERL Contact:
-
Research Areas:
Abstract:
Vision-Language-Action (VLA) models have advanced rapidly, yet most gains still come from larger backbones, larger embodied datasets, or expensive iterative decoders, while action representation remains under-explored. We argue that action space design is a primary bottleneck and present LEAP (Latent-Enhanced Action Prototyping)-VLA, a two-stage framework that learns a structured action latent space from demonstrations via Multi-level Soft Residual Quantization (MSRQ), then trains a lightweight VLM-external aligner to predict actions by prototype-aware soft selection in that space. This design replaces discrete code assignment and multi-step denoising with fully differentiable single-pass latent prediction, preserving pretrained visionlanguage alignment without modifying the VLM. Empirically, LEAPVLA outperforms the state-of-the-art methods with substantially fewer trainable parameters and no embodied pretraining, showing that a wellstructured continuous action latent space can offset model scale and data requirements.
