dew.objectives.diffusion.consistency
Continuous-time consistency distillation (sCM) combined with distribution matching (DMD2), which together make rCM.
sCM (Lu & Song 2025, “Simplifying, Stabilizing and Scaling Continuous-Time
Consistency Models”) trains a student F on TrigFlow, x_t = cos(t) x_0 +
sin(t) z, toward the tangent of the teacher’s probability-flow ODE. One JVP
gives the student’s time derivative along the ODE, and the loss pulls F
toward F plus the normalized tangent g. DMD2 (Yin et al. 2024, “Improved
Distribution Matching Distillation for Fast Image Synthesis”) moves the
student’s few-step samples along the difference between a fake score,
trained on those samples, and the teacher’s score. rCM (Zheng et al. 2025,
“Large Scale Diffusion Distillation via Score-Regularized Continuous-Time
Consistency”) combines the two. The official code is NVlabs/rcm’s
T2VDistillModel_rCM, which tools/rcm_reference.py runs.
Every network here is a rectified-flow velocity model on Dew’s Flow
process. trig_prediction reads one on TrigFlow the way rCM’s
RectifiedFlow_TrigFlowWrapper does, at rf time sin t / (cos t + sin t).
| Name | Summary |
|---|---|
FAKE_SCORE | Where rCM’s fake score network lives in params and constants. |
TEACHER | The collection of a distilling objective’s frozen teacher variables. |
ConsistencyDistillationObjective | Trains rCM: sCM distillation of a flow teacher, regularized by DMD2. |
backward_simulation | Return the student’s few-step sample from x_T, starting at t = pi/2 (rCM’s backward_simulation). |
consistency_loss | Return sCM’s per-row loss times scale (rCM’s _student_scm_step). |
critic_loss | Return the fake score’s per-row denoising loss on the student’s samples at TrigFlow t, divided by sin^2 t (rCM’s training_step_critic). |
discrete_consistency_loss | Return rCM’s discrete consistency (dCM) per-row loss (rCM’s _student_dcm_step). |
distribution_matching_loss | Return DMD2’s per-row generator loss (rCM’s _student_dmd_step). |
guided | Return the teacher’s guided prediction (guide). |
trig_prediction | Return (x_0, F) of a velocity model at TrigFlow (x, t). |
FAKE_SCORE
Section titled “FAKE_SCORE”FAKE_SCORE = 'fake_score'Where rCM’s fake score network lives in params and constants.
TEACHER
Section titled “TEACHER”TEACHER = 'teacher'The collection of a distilling objective’s frozen teacher variables.
ConsistencyDistillationObjective
Section titled “ConsistencyDistillationObjective”class ConsistencyDistillationObjective( model: nn.Module, process: Process | Preset, inputs: InputSpec, *, consistency_weight: float = 100.0, dmd_weight: float = 1.0, teacher_guidance: float = 1.0, tangent_warmup: int = 0, student_update_freq: int = 5, max_simulation_steps: int = 4, student_times: tuple[float, float] = (-0.8, 1.6), critic_times: tuple[float, float] = (0.0, 1.6), consistency: Literal['continuous', 'discrete'] = 'continuous', discrete_steps: int = 48, discrete_skip: int = 1, discrete_shift: float = 5.0, steps: int = 3, **kwargs={},)Trains rCM: sCM distillation of a flow teacher, regularized by DMD2.
The teacher (FlowDistillationObjective) is the model the fake score
runs too, and the student and the fake score start from its weights.
consistency_weight is sCM’s loss scale (100, as
in rCM; 0 trains DMD2 alone), dmd_weight is DMD2’s (1; 0 trains sCM
alone), and teacher_guidance is the teacher’s classifier-free guidance
scale in both losses.
For the first tangent_warmup steps, the tangent’s warmup ratio rises
from 0 to 1 and only the student trains. After that, one step in
student_update_freq trains the student and the others train the fake
score, the way rCM alternates its two optimizers. Each network gets its
own copy of the optimizer given to the trainer and steps only on its own
updates (optimizer). The EMA averages only the student’s updates, and
its decay reads the student’s own update count (averages); with
ema_decay=dew.averaging.power_decay(0.1) it is rCM’s power EMA at
rate 0.1. The student’s DMD2 sample takes 1 to max_simulation_steps
steps, cycling with the step count. Training times are rCM’s log-normals
in rf time: student_times for sCM, and critic_times for DMD2 and the
critic.
With consistency="discrete", the objective trains rCM’s discrete
consistency (dCM, the consistency distillation of Song et al. 2023) in
place of sCM. dCM compares the student’s x_0 at a point of a
discrete_steps grid in rf time, shifted by discrete_shift, with its
own stopped x_0 discrete_skip teacher Euler steps later.
The loss is the mean over rows, while rCM’s trainer backpropagates their
sum. Under Adam the two take the same steps when the epsilon is rCM’s
divided by the batch size. The phases alternate from one update to the
next, each read off the committed updates (Step.updates), so every
microbatch an update accumulates is of its phase. Sampling runs the
student’s multistep consistency solver, Consistency.
sCM’s loss differentiates the student with respect to time. The
recommended training default is simple_dit(time_scale=0.002); other
continuous Fourier scales remain differentiable.
ConsistencyDistillationObjective.optimizer
Section titled “ConsistencyDistillationObjective.optimizer”def optimizer(tx: optax.GradientTransformation) -> optax.GradientTransformationReturn an optimizer that gives the student and the fake score each their own copy of tx.
Each copy steps only on its own network’s updates, as rCM’s two
optimizers do. The two alternate update by update, and every
microbatch an update accumulates is of its update’s phase: the loss
reads the committed updates (Step.updates), the clock these masks
count, however many microbatches an update takes.
ConsistencyDistillationObjective.averages
Section titled “ConsistencyDistillationObjective.averages”def averages(update: jax.Array) -> jax.ArrayConsistencyDistillationObjective.program_key
Section titled “ConsistencyDistillationObjective.program_key”def program_key() -> tuple[ProgramModule, ...]The student, the frozen teacher, then the fake score, which trains.
ConsistencyDistillationObjective.substitute
Section titled “ConsistencyDistillationObjective.substitute”def substitute(modules: Sequence[nn.Module]) -> NoneConsistencyDistillationObjective.network_variables
Section titled “ConsistencyDistillationObjective.network_variables”def network_variables(key: jax.Array, state: Variables) -> VariablesThe fake score, started as the teacher too.
ConsistencyDistillationObjective.loss
Section titled “ConsistencyDistillationObjective.loss”def loss(variables, batch, step: Step)backward_simulation
Section titled “backward_simulation”def backward_simulation( student: Callable[[jax.Array, jax.Array], jax.Array], x_T, times, noises, live: jax.Array | None = None, trained: Callable[[jax.Array, jax.Array], jax.Array] | None = None,) -> jax.ArrayReturn the student’s few-step sample from x_T, starting at t = pi/2 (rCM’s backward_simulation).
At each time in times, capped per row at the time before it, the clean
prediction is noised again with the next entry of noises. The last
clean prediction is the sample, and only it keeps its gradient.
student(x, t) returns the student’s x_0. live marks the steps that
run; the other steps leave the state and the time unchanged, so a traced
step count can run over a fixed number of draws. trained, given, is the
student for that last prediction, the one call a gradient flows through.
consistency_loss
Section titled “consistency_loss”def consistency_loss( student: Callable[[jax.Array, jax.Array], jax.Array], x, t, teacher_F, warmup: float | jax.Array, scale: float, trained: Callable[[jax.Array, jax.Array], jax.Array] | None = None,) -> jax.ArrayReturn sCM’s per-row loss times scale (rCM’s _student_scm_step).
One JVP gives the student’s F-derivative along the teacher’s ODE, with tangents (cos t sin t F_teacher, cos t sin t). At warmup ratio r the tangent is
g = -cos t sqrt(1 - r^2 sin^2 t) (F_sg - F_teacher) - (r cos t sin t x + dF/dt)normalized by its norm plus 0.1, and the loss is ||F - F_sg - g||^2.
student(x, t) returns the student’s F; trained, where given, is the student whose F the loss
trains, the JVP’s apart (one that records what its call sows). The noised sample and its time, built
from float32 draws,
are widened to the teacher’s dtype, exactly, so a float64 twin takes its JVP in float64, its
tangents in the primals’ dtypes as jax.jvp asks.
critic_loss
Section titled “critic_loss”def critic_loss(generated, fake, t) -> jax.ArrayReturn the fake score’s per-row denoising loss on the student’s samples
at TrigFlow t, divided by sin^2 t (rCM’s training_step_critic).
discrete_consistency_loss
Section titled “discrete_consistency_loss”def discrete_consistency_loss( student: Callable[[jax.Array, jax.Array], jax.Array], teacher: Callable[[jax.Array, jax.Array], jax.Array], x0, noise, u, steps: int, skip: int, shift: float, scale: float,) -> jax.ArrayReturn rCM’s discrete consistency (dCM) per-row loss (rCM’s _student_dcm_step).
On a grid of steps points in shifted rf time, the loss compares the
student’s x_0 at a point u with its own stopped x_0 after skip teacher
Euler steps of F. student(x, t) returns the student’s x_0 and
teacher(x, t) the guided teacher’s F, both at TrigFlow time.
distribution_matching_loss
Section titled “distribution_matching_loss”def distribution_matching_loss(generated, fake, teacher, scale: float) -> jax.ArrayReturn DMD2’s per-row generator loss (rCM’s _student_dmd_step).
The target is the sample moved along fake - teacher, normalized by the mean distance to the teacher’s prediction (at least 1e-5). The loss regresses the sample onto that stopped target.
guided
Section titled “guided”def guided(unconditional: jax.Array, conditional: jax.Array, scale: float) -> jax.ArrayReturn the teacher’s guided prediction (guide). Guidance applies only at a scale above one.
trig_prediction
Section titled “trig_prediction”def trig_prediction( velocity: Velocity, x: jax.Array, t: jax.Array,) -> tuple[jax.Array, jax.Array]Return (x_0, F) of a velocity model at TrigFlow (x, t).
The input and the time are first scaled to rf. Then x_0 = x / (cos + sin) - rf v, and F = (cos x - x_0) / sin.