Appendix B — The complete tensor-shape table

A shipping label for every tensor

An airport receives a case arranged as passenger–time–color–height–width. The next machine expects passenger–color–time–height–width. If someone changes the label but not the order, the program may still run; it will simply treat frames as color channels.

That is why “a 5-D tensor” is not a useful specification. Name every axis, then assert shape, type, range, and normalization on the first batch.

Symbol passport

“Common in paper studies” and “repository default” are different columns in an experiment ledger. In particular, V=4 and V=10 can both be correct when their protocols are named.

From a stored episode to several views

A raw clip sampled from one episode is usually:

[T,C,H_raw,W_raw]

The multi-view transform spatially crops the same T frames. After batching, the public loader produces:

Before the encoder, main.py changes the axis order:

global_video = rearrange(global_frame, "b t c h w -> b c t h w")
local_video = rearrange(local_frames, "b v t c h w -> (b v) c t h w")

The global input is now [B,C,T,224,224]; local input is [B*V,C,T,96,96]. Folding views into the batch lets one shared encoder process every local crop. The code unfolds that axis afterward.

Count the patches before dropping any

With tau=1, each token spans one frame and one 16×16 spatial patch:

global patches = 16 * (224/16) * (224/16) = 16 * 14 * 14 = 3136
local patches  = 16 * ( 96/16) * ( 96/16) = 16 *  6 *  6 =  576

After patch embedding and before adding [CLS]:

Changing tau to 2 would halve the temporal positions to 8 and make each token span two frames. LeVJEPA defaults to tau=1; quietly importing a tubelet-2 recipe from V-JEPA or VideoMAE changes the model.

What 95% dropping does to the sequence

The public code computes:

keep_len = round(N_patches * (1 - rho))

It samples an independent random keep-set for each item:

Those are training shapes. Evaluation and inference do not drop patches. On a 16×224×224 input, the released ViT-L therefore returns all 3136+1=3137 tokens. Using the sparse training length as the public checkpoint’s output length will misalign every dense-feature operation.

Encoder, summary, and projection shapes

At the default drop rate during training:

global_tokens : [B,       158,D]
local_tokens  : [B*V,      30,D]
global_cls    : [B,         1,D]
local_cls     : [B,         V,D]
all cls       : [B,       V+1,D]
projected z   : [B,       V+1,K]

The code keeps the global projection as embeddings[:, :1] with shape [B,1,K], broadcasts it, and compares it with [B,V+1,K]. The global-versus-itself term is always zero but remains inside the mean, matching the paper formula and public code.

Before SIGReg, axes change once more:

[B,V+1,K] -> [V+1,B,K]

Each view position is then checked across B different clips. The views of one clip are not treated as additional independent population samples.

Inside SIGReg

Let its input be [S,B,K], where S=V+1:

Distributed all-reduce aggregates [2,S,M,Q] characteristic-function statistics rather than transmitting every [B,K] embedding. That is why the communication size is largely independent of batch size and embedding width.

The released inference interface

The Hugging Face model expects:

pixel_values: [B,C,T,H,W]

For [1,3,16,224,224], its ViT-L/16 interface reports:

last_hidden_state: [1,3137,1024]  # CLS + 16*14*14 patches
pooler_output:     [1,1024]       # CLS

Repeating one image across 16 frames makes a valid input tensor, but it does not create motion. It is only an adapter from a still image to a video encoder.

Five assertions worth keeping in the test suite

assert global_frame.ndim == 5 and global_frame.shape[1:3] == (16, 3)
assert local_frames.ndim == 6 and local_frames.shape[2:4] == (16, 3)
assert global_frame.shape[-2:] == (224, 224)
assert local_frames.shape[-2:] == (96, 96)
assert torch.isfinite(loss).all()

Also log dtype/min/max/mean/std before and after normalization. A perfectly shaped uint8 tensor in the range 0–255 is still wrong when the model expects normalized floats.

Shape records

LeVJEPA v1 defines the architecture and notation. Pinned main.py, module.py, and data/loader.py define the executable shapes. The released model card defines the checkpoint interface.

Three labels not to swap

1. The loader uses [B,T,C,H,W]; the encoder uses [B,C,T,H,W].
2. The 95% drop happens during training; full inference returns 3,137 tokens.
3. SIGReg’s batch axis contains different clips, not the views of one clip.

Inspect the manifest

1. Why does the local branch enter the encoder as [B*V,C,T,96,96]?
2. Why is the global training sequence 158 tokens while the released checkpoint returns 3,137?
3. What goes wrong conceptually if [B,V+1,K] is flattened into [B*(V+1),K] for SIGReg?