The Content–Geometry Tradeoff in World Models: When JEPA Learns Latent Maps
Mengye Ren1, Ying Wang1*, Chris Hoang1*, Yilun Kuang1, and Yann LeCun1,2
1New York University, 2AMI Labs
*Equal contribution
Technical report
TL;DR: Sliced variance regularization learns smooth latent maps for planning, and slow channels with a gated residual predictor keep the task-relevant content.
Abstract
A latent world model for planning needs two properties: latent distances that smoothly track progress toward a goal (geometry), and task-relevant factors that remain decodable from the latent representation (content). Common approaches prevent representation collapse with a stop-gradient (SG) target or a distributional regularizer, but we find that both often yield poor latent geometry. In this paper, we introduce sliced variance regularization (SVReg), which imposes a variance floor on latent projections along random directions. SVReg learns a smooth map of the agent's state. Its latent trajectories are smooth, and its latent distances track progress in navigation and reaching. These geometric benefits, however, can come with a tradeoff in content. SVReg can discard slowly changing, hard-to-predict factors, such as the pose of a manipulated object, that SG variants tend to preserve. To address this tradeoff, we propose Slow SVReg, which favors slowly changing factors with a slowness term and a gated residual predictor. It keeps these factors and improves planning. This simple recipe has the highest planning success in the hard settings of five visual control tasks, and it learns latent maps from first-person mazes and city walks.
Overview

A latent world model for planning needs two properties. Latent distances should track progress toward a goal (geometry), and task-relevant factors should stay decodable from the latent (content). Training without collapse usually relies on a stop-gradient (SG) target or on a variance regularizer trained without SG. We find that both choices often give poor geometry.
Sliced variance regularization (SVReg) puts a variance floor on the latent along random directions. It learns a smooth map of the agent's state, but it can drop slowly changing, hard-to-predict factors such as the pose of a manipulated object. Slow SVReg adds a slowness term and a gated residual predictor, which favor these factors. It keeps them, plans best in the hard settings of five visual control tasks, and learns latent maps from first-person mazes and city walks.
Method

SVReg puts a one-sided floor on the standard deviation of the latent along random unit directions $\mathbf{u}$:
$$\mathcal{R}_{\mathrm{SVReg}}(z) = \mathbb{E}_{\mathbf{u}}\!\left[\max\!\left(0,\,1-\mathrm{std}_t(\mathbf{z}_t \cdot \mathbf{u})\right)^2\right].$$A small fixed set of slow channels $\mathbf{b}_t$ gets a variance-normalized slowness term, so persistent factors such as a resting cube keep room in the latent:
$$\mathcal{R}_{\mathrm{slow}}(\mathbf{b}) = \frac{\mathbb{E}_t\|\mathbf{b}_{t+1}-\mathbf{b}_t\|^2}{2\,\mathrm{tr}\,\mathrm{Var}(\mathbf{b})}.$$The predictor copies the current latent and learns the change, scaled by a per-dimension gate $\mathbf{c}$ initialized at 0.1:
$$\hat{\mathbf{z}}_{t+1} = \mathbf{z}_t + \mathbf{c}\odot f(\mathbf{z}_{\le t},\mathbf{a}_t).$$Latent maps
SVReg and Slow SVReg lay out the maze in the leading principal components of the latent, from top-down and first-person views alike.

Content
On Cube, SVReg loses the cube height in three of six seeds, the ones whose random initial encoder reads the least height. From the same starts, SG and Slow SVReg acquire it.

Planning
| Navigation and reaching | Object manipulation | ||||
|---|---|---|---|---|---|
| Method | U-maze | Medium | Reacher | Cube | PushT |
| Stop-gradient | |||||
| SG | 65.0±18.7 | 71.6±8.7 | 86.5±1.6 | 80.4±2.1 | 44.0±19.3 |
| SG + straightening | 88.0±3.1 | 72.8±5.6 | 86.6±2.9 | 85.4±1.0 | 62.1±7.7 |
| SG + SVReg | 95.8±1.1 | 62.7±6.2 | 96.3±1.6 | 77.2±1.3 | 76.9±4.3 |
| No stop-gradient | |||||
| VICReg | 94.3±2.3 | 77.2±2.3 | 93.1±1.7 | 69.6±5.8 | 55.0±3.6 |
| SIGReg-BN | 82.2±13.7 | 78.2±3.8 | 95.8±1.3 | 87.4±0.8 | 60.4±2.6 |
| SVReg (ours) | 96.4±1.3 | 81.3±2.3 | 98.7±0.9 | 76.9±8.8 | 75.0±8.6 |
| Slow SVReg (ours) | 98.7±0.7 | 84.7±2.7 | 99.6±0.4 | 89.1±1.6 | 87.3±3.8 |
City walks
We train on first-person walks rendered from 3D Gaussian-splat reconstructions of real city blocks (Wanderland), with actions and no poses. The walked layout appears in the top two principal components of the Slow SVReg latent.

BibTeX
@misc{ren2026content,
title = {The Content--Geometry Tradeoff in World Models: When {JEPA} Learns Latent Maps},
author = {Ren, Mengye and Wang, Ying and Hoang, Christopher and Kuang, Yilun and LeCun, Yann},
year = {2026}
}