SLIM

Language-guided robot manipulation

SLIM: Grounding Robot Actions Through Self-Supervised Latent Interaction

A compact latent interaction policy that learns action-grounded, future-predictive visual representations.

Jingkai Wang1,2,* Zihan Tang3,2,* Gu Zhang4 Mingyu Cao2 Jiapeng Chen1 Jingjiao Zhao5,2 Xiansheng Chen2 Pengwei Wang2 Lemao Liu1 Dejing Dou1

1Fudan University   2Beijing Academy of Artificial Intelligence   3Tsinghua University   4IIIS, Tsinghua University   5Renmin University of China

*Equal contribution

Paper coming soon Code BibTeX
SLIM overview, efficiency comparison, latency, and real-world evaluation
SLIM combines masked latent trajectory learning with a compact Mixture-of-Transformers policy.

Abstract

Compact policies grounded in latent dynamics.

Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. SLIM is a compact 0.5B-parameter latent interaction policy that focuses this computation on observations, actions, and the transitions induced by actions.

SLIM learns action-grounded predictive latents through self-supervised masked trajectory prediction, alternating between action reconstruction and future-latent prediction. A compact Mixture-of-Transformers backbone then performs language-conditioned flow-matching action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative VLA and world-action-model baselines with fewer parameters, lower deployment latency, and substantially lower GPU memory usage.

Method

Masked trajectory learning for action-grounded latent interaction.

Action-masked and future-latent-masked training objectives
Stage 1 alternates action-masked reconstruction and future-latent prediction to ground visual representations in robot dynamics.
Mixture-of-Transformers observation and action streams
The MoT backbone uses joint observation-action attention, with language as a task-level condition and visual observations as state context.

Results

Strong performance with a compact policy backbone.

97.5%LIBERO overall success
77.45%LIBERO-plus zero-shot success
4.556 / 5CALVIN ABC to D average length
0.47Bmodel parameters

Benchmark comparisons

Competitive across robustness and long-horizon control.

Representative baselines from the current paper tables. Bar length encodes benchmark performance; the right column reports model parameters.

SLIM (ours) VLA-based WAM-based

LIBERO-plus

Zero-shot overall success (%)

77.45%
  1. VLA-JEPA WAM
    79.503B
  2. SLIM Ours
    77.450.47B
  3. OpenVLA-OFT VLA
    69.607B
  4. pi_0 VLA
    69.303.3B
  5. Fast-WAM WAM
    50.006B

CALVIN ABC to D

Average sequence length (max 5)

4.556
  1. SLIM Ours
    4.5560.47B
  2. FLOWER VLA
    4.5300.95B
  3. DreamVLA WAM
    4.440--
  4. UnifiedVLA VLA
    4.4108.5B
  5. VPP WAM
    4.3301.5B

Real-world evaluation

Robust multi-task manipulation on a physical robot.

67.8average real-world progress
77.3 msend-to-end policy latency
2.01 GiBpolicy-server GPU memory
SLIM real-world robot manipulation setup
One of five physical manipulation tasks used to evaluate nominal performance and visual robustness.
Average real-world progress across nominal and out-of-distribution settings
SLIM has the highest average progress across the nominal, distractor, lighting, and background settings.

Ablations

Trajectory objectives and EMA stabilize OOD performance.

Stage-1 masked trajectory learning improves both LIBERO-plus robustness and CALVIN long-horizon composition over Stage-2-only training.

An IDM:FDM loss ratio of 0.125:1 gives the strongest joint result. EMA raises LIBERO-plus success from 66.82% to 77.45% and CALVIN average length from 4.382 to 4.556.

Stage 1, masked-target, and EMA ablation curves
Stage-1, loss-ratio, and EMA ablations on LIBERO-plus and CALVIN.

Analysis

Trajectory learning focuses action evidence on manipulation-relevant regions.

Action-to-observation attention comparison for SLIM and a policy without Stage 1
Across two manipulation sequences, SLIM follows the manipulated object, gripper, and contact region more consistently than the step-matched policy without Stage 1.

Citation

BibTeX

@misc{slim2026,
  title        = {SLIM: Grounding Robot Actions Through Self-Supervised Latent Interaction},
  author       = {Jingkai Wang and Zihan Tang and Gu Zhang and Mingyu Cao and Jiapeng Chen and Jingjiao Zhao and Xiansheng Chen and Pengwei Wang and Lemao Liu and Dejing Dou},
  year         = {2026},
  note         = {Project page placeholder}
}