Skip to content

dew.config.jepa

The JEPA run, as one typed record: I-JEPA over images, V-JEPA over video.

The encoder is the run’s model. The predictor takes the encoder’s width and heads, and its own depth and width from predictor: a predictor as wide as the encoder makes the objective too easy. Optional probes score the frozen encoder beside the validation loss.

NameSummary
JepaRunConfigA run, plus the JEPA objective’s own knobs.

dataclass source

class JepaRunConfig(
model: ModelConfig = (lambda: ModelConfig('jepa_encoder', {'precision': 'default'}))(),
data: Annotated[Dataset | None, tyro.conf.Suppress] = None,
validation: Annotated[Dataset | Mapping[str, Dataset] | None, tyro.conf.Suppress] = None,
data_seed: int = 0,
source: SourceOptions = SourceOptions(),
optim: OptimConfig = (lambda: OptimConfig(learning_rate=0.001))(),
trainer: TrainerConfig = TrainerConfig(),
objective: ObjectiveConfig = (lambda: ObjectiveConfig('jepa'))(),
lora: Annotated[LoRA, tyro.conf.subcommand(lora)] | None = None,
val_metrics: Annotated[tuple[MetricConfig, ...], record_argument(tuple[MetricConfig, ...])] | None = None,
val_every: Annotated[Mapping[str, int | str], record_argument(Mapping[str, int | str])] = dict(),
val_summaries: Annotated[tuple[Summary, ...], record_argument(tuple[Summary, ...])] = (),
predictor: JsonDict = dict(),
num_target_blocks: int = 4,
target_scale: tuple[float, float] = (0.15, 0.2),
target_aspect: tuple[float, float] = (0.75, 1.5),
probe_classes: int | None = None,
knn_k: int = 20,
)

A run, plus the JEPA objective’s own knobs.

objective: ObjectiveConfig

The JEPA objective and its arguments (--objective.momentum 0.996 1.0). Its EMA ramps over the whole run unless it names momentum_steps.

predictor: JsonDict

Predictor kwargs, over the encoder’s shared ones.

probe_classes: int | None

Number of classes for optional frozen-encoder probes; without them, validation scores the loss alone.

def sample() -> Field

The batch field the encoder reads, at the resolution the data comes in (Dataset.facts): a clip’s video, else an image. Raises ValueError for a chain of neither, or one whose size varies.

def build_run() -> Prepared

The encoder, the predictor over its patch grid, the target-block mask and the probes, around the objective the run names. The predictor computes as the encoder does, its compute dtype and attention kernel.

def default_metrics(*, validates: bool) -> tuple[MetricConfig, ...]

A linear and a k-nearest-neighbour probe of the frozen encoder, where probe_classes names the labels’ classes, whatever the data holds: a run without a validation split refuses them.