Chapter 30 — The Gap Between the Training Objective and the Planning Objective

The big picture

A tiny story

A courier predicts every small step correctly. The goal is close across a wall, so the planner keeps choosing “walk toward the goal.” The courier presses into the wall instead of moving away toward the doorway.

Nothing in this story requires bad next-step prediction. Training asks, “Did this action lead to the correct next latent?” Planning asks, “Which whole sequence should be preferred?” Original LeWM v3 answers the second question with terminal squared latent Euclidean distance. A forecast and a route judgment are not the same object.

The real rule

Low mean squared prediction error cannot guarantee a good ordering because:

• common local motion can dominate rare doorway or contact decisions;
• a representation can predict nearby transitions while arranging wall-separated endpoints badly;
• planning evaluates self-fed, model-generated states, not only recorded contexts;
• Euclidean cost is symmetric and has no explicit direction, budget, support, or uncertainty argument.

Use oracle substitutions before changing the whole model:

oracle rollout + learned cost still ranks badly
  → inspect cost or representation geometry
learned rollout + oracle cost fails
  → inspect multi-step dynamics and support
both oracles succeed, deployed planner fails
  → inspect search, observation, action conversion, execution

An oracle is a laboratory instrument with privileged simulator access, not a deployable pixels-only component.

Two later proposals change different jobs. RC-aux replaces the original one-step prediction term with weighted multi-horizon open-loop prediction whose horizon set still includes step 1, and adds a budget-conditioned reachability head. Temporal-Distance JEPA learns a separate directed temporal cost and an (H=5) rollout-consistency loss matching each predicted latent to a stop-gradient encoding of the recorded future. That local detach belongs to the later method; original v3 targets remain connected.

Trajectory offsets are evidence of what one behavior did, not oracle shortest times. Cross-trajectory negatives can also be false.

The trick that can fool us

First change only the ruler. Use exact simulator endpoints for a fixed TwoRoom candidate bank. Rank them by straight-line distance and by wall-respecting graph distance. Remove the wall and repeat.

If graph distance changes the ranking only when the wall creates a detour, the topological-cost explanation survives this toy removal test. If both costs fail with exact endpoints, inspect candidate coverage or task definition. Then add learned encodings, reverse direction, vary the budget, and only afterward add learned rollout.

Do not call every later method the same “planning-aware objective”:

• Objective Bottleneck freezes one TwoRoom representation/predictor and swaps planner cost.
• Traj-LeWM adds a goal-conditioned latent trajectory cost, synthetic ranking negatives, and after each epoch mines failed endpoint-only executions to train that cost.
• Decision-Metric Alignment measures Plan–Real and CEM-stage Spearman, then adds training-only inverse-dynamics and goal-action heads while retaining Euclidean MPC cost.

These interventions target ruler, path, and representation geometry respectively.

Experiment receipt

The Objective Bottleneck follow-up reports latent L2 versus true distance at (r=0.426) despite a position ridge probe at (R^2=0.9922). On its checkpoint, an offset-100 temporal cost changes success from 26% to 98%; on the released checkpoint it changes 14% to 34%, while privileged decoded-position cost reaches 70%. This is one environment, one seed per checkpoint, and a non-v3-default long-goal protocol.

Traj-LeWM reports gains of 3, 14, 7, and 7 percentage points on PushT, Cube, Reacher, and TwoRoom over its LeWM comparison, averaged over three evaluation seeds. Its failure mining means the intervention is not purely offline-data training.

Decision-Metric Alignment reports one training run per configuration; its three seeds quantify evaluation variation, and its rank diagnostics require simulator rollout. These are author-reported results, not independent reproduction.

The baseline source remains LeWorldModel v3, commit 8edfeb3. No later stop-gradient, reachability, temporal, or action-head objective may be drawn backward into v3.

Quick check

1. How can oracle dynamics still produce bad TwoRoom planning?
2. What different jobs do multi-horizon prediction and reachability scoring perform?
3. Which oracle cell tells you to change the ruler before the predictor?