Skip to content

dew.objectives.diffusion.guidance_distillation

Guidance distillation: a student that reads the guidance scale as an input.

Meng et al. 2023 (“On Distillation of Guided Diffusion Models”, stage one) train a student ε_s(z_t, w) to match the teacher’s classifier-free guided prediction at a scale w drawn per example. One student evaluation then replaces the teacher’s two. FLUX.1 [dev] ships such a student, with its guidance embedded beside the time. No training code is published for it, so this module follows the paper’s equation.

NameSummary
GuidanceDistillationObjectiveDistills a teacher’s classifier-free guidance into this model.
with_guidanceReturn conditions with every conditioning record’s guidance set to scale, one value per row.

class source

class GuidanceDistillationObjective(
model: nn.Module,
process: Process | Preset,
inputs: InputSpec,
*,
teacher: nn.Module | DiffusionObjective | None = None,
teacher_variables: Variables | None = None,
scales: tuple[float, float] = (1.0, 8.0),
**kwargs={},
)

Distills a teacher’s classifier-free guidance into this model.

The teacher is teacher: an objective, as a run config rebuilds the teacher’s run from its record (DiffusionRunConfig.teacher_run), or a model over this objective’s process and inputs, by default this one without its adapter. Its whole variables tree is held frozen under TEACHER: the starting tree’s when it holds one, as a saved student’s does, else teacher_variables, the tree the teacher’s run published, else the model the starting tree holds. The teacher reads its own conditioning of the batch, the conditioning its run trained on. Each row draws a scale uniformly from scales. The student reads that scale as its conditioning record’s guidance, and regresses its raw output onto the teacher’s guided raw output at that scale, both on the same noised sample, under the process’s weighting. The two must share the process’s schedule and prediction and the data’s latent geometry, and the student’s conditioning must have a guidance input, as a guidance-embedded Flux’s does. Sampling is unguided.

def program_key() -> tuple[ProgramModule, ...]

The student, then the teacher’s modules, which the step does not train.

def towers() -> tuple[Held, ...]

The student’s towers, then the teacher’s, whose own encoders condition its targets.

def substitute(modules: Sequence[nn.Module]) -> None

GuidanceDistillationObjective.held_variables

Section titled “GuidanceDistillationObjective.held_variables”
def held_variables(*, beside: Variables | None = None) -> Variables
def loss(variables, batch, step: Step)

function source

def with_guidance(conditions: dict, scale: jax.Array) -> dict

Return conditions with every conditioning record’s guidance set to scale, one value per row.