Appendix H — Expert-Review Checklist

The big picture

A tiny story: eight inspectors

A manuscript is fluent, its figures are polished, and its table has neat decimals. Eight inspectors still stop it: one finds an invented teacher network; another finds a later preprint smuggled into v3; others ask what a probe proves, where benchmark conditions went, whether “caused” is justified, which theorem assumptions apply, where failed seeds are, and why the code link points to moving main.

A check mark is not evidence. Every pass points to a primary source, frozen implementation, controlled experiment, explicit assumption, raw result, or declared tutorial construction.

Before review, rewrite “LeWM works” as a checkable sentence with subject, mechanism, model version, environment, data, planner/budget, evidence type, and conclusion strength.

The eight-gate checklist

H.1 Is the original training graph accurate?

• One shared trainable visual encoder processes context and shifted next observations.
• The shifted target is encoded by that same model and is not detached; prediction gradients can reach both connected encoder routes.
• Actions align with transitions and condition the predictor; SIGReg regularizes observation-embedding populations.
• Original v3 has no stop-gradient target, EMA teacher, or pretrained visual encoder.
• Goal images, CEM, rewards, probes, and optional decoders stay outside the baseline training lane.

Passing this gate proves graph identity and possible gradient routes—not non-collapse, physics, rollout accuracy, or planning success.

H.2 Are baseline and later routes separated?

• Give every later method a passport: title, exact version/date, changed layer, targeted failure, evidence level, code status, and independent-reproduction status.
• Use LeJEPA only as motivation/theory where cited; never import a later detach, auxiliary head, cost, hierarchy, or teacher into v3.
• Distinguish “defines/proposes,” “authors report,” “theorem under assumptions,” “independently reproduced,” and “tutorial construction.”
• Treat recency, acceptance, and public author code as separate facts—not an evidence ladder.

H.3 Are probes and pictures interpreted with restraint?

• Record probe family/capacity, train/test split, preprocessing fit, episode leakage controls, shuffled labels, and simple/raw-feature baselines.
• Say “linearly decodable under this protocol,” not “the model learned/uses/understands the variable.”
• Treat t-SNE/UMAP, reconstructed images, latent trajectories, and VoE surprise as conditional views, not proof of semantics or intuitive physics.
• If claiming use or cause, add an intervention and disclose its side effects.

H.4 Do benchmark claims keep their conditions?

• Attach environment/task, data, model, planner, action/search/compute budgets, pretraining, seed policy, episode ledger, metric, and hardware timing.
• Inspect every speed multiplier’s denominator: model calls, batching, rendering, candidates, horizon, GPU, and whether pretraining is counted.
• Keep scope distinct: TwoRoom topology, PushT 2-D contact, simulated Reacher, and simulated OGBench-Cube do not by themselves establish general robotics, physical understanding, or safety.
• Preserve author-reported weaknesses and possible explanations as “possible,” not diagnosed causes; inspect goal/failure distributions, not only means.

H.5 Does language distinguish correlation from causation?

• Circle jumps such as “correlates with velocity, therefore uses velocity” or “SIGReg changed success, therefore Gaussianity caused physics.”
• Challenge the nearest alternative: hold physical state fixed while changing appearance; swap cost with dynamics fixed; swap dynamics with cost fixed; use oracle cells to localize bottlenecks.
• Remember that an ablation can change capacity, optimization, support, and search queries together. Match causal wording to the intervention actually isolated.

H.6 Are theorem assumptions attached?

• Name stationarity, transition/noise family, observation adequacy/invertibility, Gaussian/exact constraints, matched dimension, predictor expressivity, population/global optimum, signal separation, and state-conditioned action excitation where required.
• Keep the conclusion narrow: orthogonal recovery does not produce human-named axes; controlled theory identifies a conditional mean, not every multimodal future.
• List benchmark gaps separately: nonlinearity, partial observability, missing action support, drift, multimodality, finite data, approximate optimization, and misspecification.
• Say “diagnostic lens, not direct coverage” when assumptions are unknown or false.

H.7 Are positive, negative, null, and missing results visible?

• Show all declared seeds or the preregistered exclusion rule, uncertainty, per-episode outcomes, divergence/collapse/timeouts, unsuccessful ablations, and representative failures chosen by a stated policy.
• Use open for untested, null under this setup for measured non-separation, and failed under this setup for a missed criterion.
• Do not generalize a negative result beyond its task, method, support, budget, and protocol—or hide it because it weakens the story.

H.8 Does every implementation claim have a reproducible path?

• End at a frozen first-party file/SHA, pinned artifact plus hash/format, or clearly labeled tutorial construction and divergence.
• Pin dependencies separately, save composed configuration, data/checkpoint conversion, episode ledger, and analysis revision; avoid moving main when a SHA exists.
• Run the archive smoke test: load one checkpoint, trace one window to its episode, regenerate one summary. This tests handoff, not independent reproduction.
• Preserve history: current v3 TwoRoom is 25/50; the reproduction’s audited paper-side 100/150 belongs to v1/v2.

Evidence is coordinates, not a staircase

Decodability, predictability, controllability, generalization, causal identification, and planning utility ask different questions. A decoded position can be ignored by the predictor. Recorded-action accuracy can coexist with counterfactual failure. A planner can exploit a task shortcut without an identifiable state.

Ask which coordinates were tested and what remains open. Several indirect signals do not merge into the word “understanding.”

Try to break the claim

“LeWM stabilizes its pretrained visual teacher with EMA and stop-gradient, then proves through probes that its Gaussian state understands physics and makes CEM universally faster.”

This fluent sentence fails every gate. The graph is wrong; later mechanisms are imported; probes are promoted to understanding; speed lost hardware/budget conditions; correlation became causality; no theorem assumptions appear; failures vanished; and no frozen path is given.

Red-flag phrases trigger questions, not automatic rejection: proves understanding, guarantees physics, universally faster, distance is reachability, and the ablation shows the cause. Strong wording survives only when the operational definition, alternatives, budgets, assumptions, and direct evidence survive with it.

Evidence receipt

Baseline identity comes from LeWorldModel v3, frozen official code, and LeJEPA v3 for cited motivation. Theory comes from passive identifiability v1 and controlled identifiability v2. Independent evidence is limited to the TwoRoom reproduction v1 and its documented protocols. LeWM v3 was revised 2026-06-03, frozen code dated 2026-05-22, and this checklist’s cutoff is 2026-08-20. Passing all gates means the dated claim is accurately attributed, scoped, calibrated, balanced, and inspectable—not that the model is correct forever or that specialist peer review is unnecessary.

Three quick questions

1. Which direct facts expose an invented EMA teacher in a v3 diagram?
2. Why does a strong probe not establish that the planner uses the variable?
3. What is the strongest conclusion after all eight gates pass?