Chapter 9 — How global and local views are paired

One flipbook, several cutouts

A teacher makes a 16-page flipbook. She photographs the whole desk for one edition, then prints close-up editions containing only the cup, hand, or table edge. Some close-ups are brighter or grayer. Every edition must still turn through the same 16 pages. One beginning on page 9 answers a different question.

Build the views carefully

For each training clip, LeVJEPA first fixes one 16-frame interval. It then makes 1 + V views. The global x_0 is 224 × 224 and covers more of the scene. Each local x_1 … x_V is 96 × 96, uses a tighter spatial crop, and receives photometric augmentation. The global view is the broad, mildly treated reference—not a label and not a stopped-gradient teacher.

V is an experimental setting, not a constant hidden in the method’s name. Unless stated otherwise, the paper’s controlled comparisons use four local views. In that setting, raising the count to ten improves the reported ImageNet probe; the curve approaches saturation around twelve. The public Walking Tours configuration defaults to ten. Always record the actual V; do not turn a repository default into a universal paper setting.

Why bother with a close-up? To align its [CLS] summary with the wide view, the encoder cannot rely entirely on one fixed coordinate or a complete background. It must find useful evidence in fragments. The wide view supplies a more stable, information-rich anchor than close-ups alone.

“Same time” is stronger than “same length.” Two clips can each contain 16 frames while starting at different stages of an action. LeVJEPA does not ask the first 16 frames to predict the next 16. Global and local views are simultaneous samples of one interval. Block-causal attention operates later, inside the encoder, and changes none of this pairing rule.

After augmentation, each view receives its own patch embedding and random token sample. Their keep-sets need not match. They share source times and network parameters, not necessarily visible patch coordinates.

Make a legal group

Let the source indices be 101–116. Use a 224 × 224 crop of those frames globally and three different 96 × 96 crops of those same frames locally. Change one local interval to 117–132. It remains 16 frames long, but the pairing is now invalid.

View-making records

• LeVJEPA v1: global/local views and local-count ablation
• Pinned default config: 224, 96, 16 frames, and ten local views
• Pinned data loader and multi-crop transform

Keep the window frame

1. Global 224 × 224 and local 96 × 96 views share exactly the same 16 frames.
2. Local-view count is part of the experiment; paper comparisons and repository defaults can differ.
3. Views may crop, augment, and drop tokens independently while sharing encoder/projector weights.

Window check

1. Can two 16-frame intervals with different start times form the positive pair?
2. Is the repository’s local-view count a fixed constant for every paper experiment?
3. Is the global view a stopped-gradient teacher?