Skip to content

dew.training.quantization

The training step that quantizes the modules an objective trains (dew.quantization).

NameSummary
quantize_trunkQuantize the trunk matmuls of the modules objective trains, in place.

function source

def quantize_trunk(objective: Objective[LossT, EffectsT], spec: Quantization) -> None

Quantize the trunk matmuls of the modules objective trains, in place.

This is RunConfig.train’s step, not a user’s. Quantization.apply wraps a module before an objective is built, which is what a recipe or a script that builds its own model does. A run that names --trainer.quantization has handed RunConfig.train the objective already, so the wrap lands on the modules the objective runs instead (Objective.substitute), before anything has initialised or traced them; each wrapped module is a copy of the same class, so what the objective read off it at construction still holds. A frozen teacher or reference keeps its own numerics.

An objective that trains no module has nothing to wrap and is refused by name.