CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization

Excerpt

arXiv:2610.00727v1 Announce Type: cross Abstract: Controlling an agent with vision requires being able to separate useful information from irrelevant background information. JEPA-style latent world models seem like a natural approach for this, as they do not perform pixel-level reconstruction; however, they are still sensitive to these distractor signals and experience latent collapse. In this work, we introduce Controllability Factorized JEPA (CF-JEPA), a JEPA-style world model which splits the