dew.objectives.diffusion.guidance_distillation
Guidance distillation: a student that reads the guidance scale as an input.
Meng et al. 2023 (“On Distillation of Guided Diffusion Models”, stage one) train a student ε_s(z_t, w) to match the teacher’s classifier-free guided prediction at a scale w drawn per example. One student evaluation then replaces the teacher’s two. FLUX.1 [dev] ships such a student, with its guidance embedded beside the time. No training code is published for it, so this module follows the paper’s equation.
| Name | Summary |
|---|---|
GuidanceDistillationObjective | Distills a teacher’s classifier-free guidance into this model. |
with_guidance | Return conditions with every conditioning record’s guidance set to scale, one value per row. |
GuidanceDistillationObjective
Section titled “GuidanceDistillationObjective”class GuidanceDistillationObjective( model: nn.Module, process: Process | Preset, inputs: InputSpec, *, teacher: nn.Module | DiffusionObjective | None = None, teacher_variables: Variables | None = None, scales: tuple[float, float] = (1.0, 8.0), **kwargs={},)Distills a teacher’s classifier-free guidance into this model.
The teacher is teacher: an objective, as a run config rebuilds the
teacher’s run from its record (DiffusionRunConfig.teacher_run), or a
model over this objective’s process and inputs, by default this one
without its adapter. Its whole variables tree is held frozen under
TEACHER: the starting tree’s when it holds one, as a saved student’s
does, else teacher_variables, the tree the teacher’s run published,
else the model the starting tree holds. The teacher reads its own
conditioning of the batch, the conditioning its run trained on. Each row
draws a scale uniformly from scales. The student reads that scale as its
conditioning record’s guidance, and regresses its raw output onto the
teacher’s guided raw output at that scale, both on the same noised sample,
under the process’s weighting. The two must share the process’s schedule and prediction and
the data’s latent geometry, and the student’s conditioning must have a
guidance input, as a guidance-embedded Flux’s does. Sampling is unguided.
GuidanceDistillationObjective.program_key
Section titled “GuidanceDistillationObjective.program_key”def program_key() -> tuple[ProgramModule, ...]The student, then the teacher’s modules, which the step does not train.
GuidanceDistillationObjective.towers
Section titled “GuidanceDistillationObjective.towers”def towers() -> tuple[Held, ...]The student’s towers, then the teacher’s, whose own encoders condition its targets.
GuidanceDistillationObjective.substitute
Section titled “GuidanceDistillationObjective.substitute”def substitute(modules: Sequence[nn.Module]) -> NoneGuidanceDistillationObjective.held_variables
Section titled “GuidanceDistillationObjective.held_variables”def held_variables(*, beside: Variables | None = None) -> VariablesGuidanceDistillationObjective.loss
Section titled “GuidanceDistillationObjective.loss”def loss(variables, batch, step: Step)with_guidance
Section titled “with_guidance”def with_guidance(conditions: dict, scale: jax.Array) -> dictReturn conditions with every conditioning record’s guidance set to scale, one value per row.