Chapter 18 — MPC: Do Not Trust the Model for Too Long at Once

The big picture

A tiny story

A map says “turn right,” but roadworks push you left. A sensible navigator looks again and draws a new route.

Model Predictive Control (MPC) does the same kind of thing. It imagines a finite plan, executes a declared prefix, receives a new real observation, and searches again. Feedback limits how long a prediction remains the only story. It does not guarantee that the encoder, predictor, cost, or search will interpret the new evidence correctly.

The real rule

Use three different clocks:

• (H): model-level action blocks imagined in each candidate.
• (K): model-level blocks actually executed before replanning, with (K\le H).
• raw environment timesteps inside one action block.

In the reported and frozen LeWM configuration, (H=5), (K=5), and each block contains five consecutive environment actions. So the whole five-block plan spans 25 environment timesteps and is executed before the next planning call. This is still MPC across the episode, but it is not the often-drawn (K=1) feedback loop.

freeze the trained encoder and predictor
while the task is active:
  encode the real observation history and separate goal
  run inner CEM over H-block candidates
  select a plan using the recorded return convention
  execute its first K blocks in the environment
  receive a new real observation and plan again

CEM and MPC are not synonyms. CEM changes an action proposal within one planning call. MPC is the outer observe–plan–execute–observe policy. Neither updates LeWM weights during evaluation, and unrealized candidates never receive real feedback.

The trick that can fool us

Call the new observation an epistemic reset, but keep the limit visible. It resets dependence on the old imagined endpoint. It does not reveal hidden velocity, contact force, or a privileged simulator state. A systematic across-wall cost error can simply be chosen again.

Use one controlled shove after the first action block. Compare:

1. an episode-open-loop controller;
2. a tutorial (K=1) controller;
3. the reported-style (K=5) controller.

Hold the start, goal, checkpoint, action bounds, total real-action budget, and seed schedule fixed. Record when each controller first sees the shove, how its next plan changes, planning calls, latency, and executed path. A shorter (K) receives fresher feedback but also more CEM calls. That is a feedback–compute trade-off, not a free win.

Experiment receipt

For each cycle, save the selected imagined plan, solid executed prefix, dashed unused suffix, returned observations, prefix forecast error, and replacement plan. Log both normalized model actions and raw environment actions; an adapter bug can imitate model failure.

The source boundary is LeWorldModel v3 and frozen TwoRoom evaluation config, commit 8edfeb3. The independent TwoRoom reimplementation also matches the five-block horizon and five-block execution interval in its own system.

The careful claim is: MPC can reduce reliance on long open-loop prediction when the deviation is observable and the remaining model, cost, search, and action path are useful. It is not a safety proof, a perfect-state reset, or evidence that shorter (K) is universally better.

Quick check

1. Why must a report separate model blocks from environment timesteps?
2. What does a new observation reset, and what can remain wrong?
3. How do the inner CEM clock and outer MPC clock differ?