Summarizing: JEPA

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture introduces I-JEPA, a self-supervised representation learning framework that avoids traditional data augmentations and pixel-level reconstruction. Instead, it learns semantic visual features by predicting representations of masked image regions directly in embedding space.

I-JEPA builds on the Joint-Embedding Predictive Architecture paradigm. Given a visible portion of an image, the model predicts embeddings of other regions from the same image. This differs from invariance-based approaches such as contrastive learning, which rely heavily on hand-crafted augmentations, and from generative masking methods such as MAE, which reconstruct raw pixels. The key idea is to predict high-level representations rather than low-level signals.

The architecture consists of three components. First, a context encoder implemented as a Vision Transformer that processes a large contiguous image region. Second, a target encoder, also a Vision Transformer, updated via exponential moving average to produce stable target representations. Third, a predictor network that takes the context representation along with positional information and predicts the embeddings of masked target blocks.

During training, multiple target blocks are sampled at semantic scales, and the model minimizes the average L2 distance between predicted embeddings and target embeddings. Because prediction occurs in representation space rather than pixel space, the model is encouraged to capture abstract semantic structure instead of texture statistics.

A notable property of I-JEPA is that it uses only a single image view without heavy augmentation pipelines. This simplifies training dynamics and reduces reliance on inductive biases introduced by transformation heuristics. It also avoids the computational overhead associated with reconstructing high-resolution pixel targets.

Empirically, I-JEPA achieves strong performance on ImageNet-1K linear evaluation, outperforming prior non-augmentation methods such as MAE and data2vec while using less compute. In low-shot classification settings with limited labeled data, it matches or exceeds competitive baselines. On transfer benchmarks including CIFAR100 and Places205, it demonstrates robust semantic representations. It also performs competitively on dense prediction tasks such as object counting and depth estimation, indicating that spatial structure is preserved.

By predicting structured representations instead of pixels or augmented views, I-JEPA provides evidence that semantic visual representations can emerge from predictive learning in embedding space. The framework offers a scalable and compute-efficient direction for self-supervised learning in vision and suggests broader applicability to multimodal and world-model architectures.