Chapter 8 — The whole LeVJEPA objective on one line

Two rules and one volume knob

Imagine a classroom with two marks for every set of cards. The first is awarded clip by clip: do the wide and close-up cards agree? The second looks across the room: do all cards form a healthy spread, or did everyone copy one answer? The second score is turned down to 0.02 and added to the first. No hidden teacher supplies perfect cards. Both sides revise together.

The shortest honest accounting

z_v = projector(shared_encoder(view_v)[CLS])
L_total = L_inv + 0.02 * L_SIGReg

view_0 is global; view_1 … view_V are local views of the same 16 frames. The paper writes the invariance term as:

L_inv = (1 / (V + 1)) * sum_{v=0..V} ||z_0 - z_v||²

The v = 0 term is zero, so the working constraint is each local embedding’s distance from the global one. Both arguments of that squared distance remain connected to the graph. Gradients from global and local paths update the same encoder/projector parameters.

L_SIGReg measures how the projected view embeddings differ from a standard normal along sampled directions. λ = 0.02 is the objective’s only trade-off hyperparameter, and the paper reports using that public default throughout rather than tuning it per experiment. “Only” does not erase the learning rate, batch size, crop count, token drop ratio, or model size.

The training box contains one shared Vision Transformer, one small shared projector, and a [CLS] readout per view. The projector gives SIGReg a space not pinned to the encoder’s final LayerNorm geometry. It is thrown away after pretraining; downstream tasks consume encoder features.

Outside that box: no target encoder, predictor, stop-gradient, or reconstruction of missing tokens. The official code does update a Polyak/EMA encoder after parameter steps and saves state_dict_ema for evaluation. Because that copy never supplies any z_v or loss target, it belongs outside the objective diagram.

One more absence matters. The equation contains no action a_t, future-state prediction, reward, cost, or search. Elegance does not turn the encoder into a planner.

Check the arithmetic

Suppose a log line shows L_inv = 0.30 and L_SIGReg = 5.0. Then L_total = 0.30 + 0.02 × 5.0 = 0.40. If someone computes 0.02 × L_inv + L_SIGReg, the knob is attached to the wrong term.

Objective records

• LeVJEPA v1: objective, bidirectional gradients, and λ = 0.02
• Pinned repository: full recipe and EMA evaluation note
• Pinned module.py: model and loss
• LeJEPA v3: the theoretical origin of SIGReg

Fold the equation into three lines

1. The objective is L_inv + 0.02 × L_SIGReg.
2. Global and local paths share the encoder/projector, and gradients flow through both sides.
3. EMA serves evaluation; there is no objective teacher, predictor, stop-gradient, or planner.

Final check for this part

1. Which loss does λ = 0.02 multiply?
2. Does the projector stay for downstream use?
3. How can the repository contain EMA weights while the objective has no EMA teacher?