dew.config.diffusion
The diffusion run as one typed record, and the one function that builds it.
DiffusionRunConfig is what a diffusion recipe parses from its command line
and writes as run.json next to the checkpoints. build() is the single
function that turns it into a DiffusionObjective: the recipe trains the
objective it returns, and TextToImage.from_run samples from the objective it
returns for the same file.
| Name | Summary |
|---|---|
DiffusionRunConfig | A diffusion run’s configuration: the shared run fields plus the diffusion objective’s own settings. |
FlowRolloutConfig | A run’s flow RL rollout: groups samples of each prompt over steps time points, each scored by the image metric reward names, an alias or an import path, against its own prompt, higher being better (clip_score). |
DiffusionRunConfig
Section titled “DiffusionRunConfig”class DiffusionRunConfig( model: ModelConfig = (lambda: ModelConfig('unet', dict(DEFAULT_MODEL_CONFIG)))(), data: Annotated[Dataset | None, tyro.conf.Suppress] = None, validation: Annotated[Dataset | Mapping[str, Dataset] | None, tyro.conf.Suppress] = None, data_seed: int = 0, source: SourceOptions = SourceOptions(), optim: OptimConfig = OptimConfig(), trainer: TrainerConfig = TrainerConfig(), objective: ObjectiveConfig = (lambda: ObjectiveConfig('diffusion', {'solver': to_record(EulerAncestral(), Solver)}))(), lora: Annotated[LoRA, tyro.conf.subcommand(lora)] | None = None, val_metrics: Annotated[tuple[MetricConfig, ...], record_argument(tuple[MetricConfig, ...])] | None = None, val_every: Annotated[Mapping[str, int | str], record_argument(Mapping[str, int | str])] = dict(), val_summaries: Annotated[tuple[Summary, ...], record_argument(tuple[Summary, ...])] = (), preset: PresetSpec | None = EDM(), text: TextCondition | None = TextCondition(), audio: AudioCondition | None = None, autoencoder: PretrainedAutoencoder | None = None, pretrained: str | None = None, teacher_run: str | None = None, rollout: Annotated[FlowRolloutConfig, tyro.conf.subcommand('rollout')] | None = None,)A diffusion run’s configuration: the shared run fields plus the diffusion objective’s own settings.
preset: PresetSpec | None-
The convention the model is trained and sampled with.
None uses the one the
pretrainedpipeline’s scheduler reads, which a preset may restate. objective: ObjectiveConfig-
The objective and its arguments: the denoising loss (
diffusion) with its EMA, condition dropout and validation sampling (--objective.ema-decay,--objective.guidance,--objective.steps), or a loss of its own in its place, as--objective mean_flow --objective.omega 2or--objective rcm --teacher-run runs/teacher. Each objective refuses a preset or a guidance its loss cannot train or sample with. text: TextCondition | None-
The text condition, passed under its encoder’s keyword (
ConditionEncoder.keyword).None trains unconditionally, or on
audio. audio: AudioCondition | None-
The audio condition, passed under its encoder’s keyword in place of text.
It needs
textto be None and a chain of clips that hold their audio (dew.data.presets.clips(sample_rate=)). autoencoder: PretrainedAutoencoder | None-
The autoencoder for latent diffusion; None trains in pixel space.
pretrained: str | None-
A published diffusion pipeline to fine-tune, or a saved diffusion run to start from.
It is a Hub repo,
repo@revisionor a local directory in the diffusers layout, or a saved Dew run’s directory, which loads as its own sampling task at the step--source.stepselects (load_diffusion_source). The checkpoint decides the model, its text conditioning and its autoencoder, so--modelholds only the precision settings andtextandautoencoderstay unset. teacher_run: str | None-
The run directory a distillation’s teacher trained in. Its published weights are the frozen teacher’s (
teacher_variables=), unless the starting tree holds the teacher; guidance distillation also rebuilds the teacher’s objective from the run’s record (teacher=). Only a distillation takes one. rollout: Annotated[FlowRolloutConfig, tyro.conf.subcommand('rollout')] | None-
The rollout a Flow-GRPO run trains on: its reward metric, group size, time points and trained transitions,
--rollout.reward clip_score --rollout.groups 4. A Flow-GRPO run without one takesFlowRolloutConfig(), which its record then states; another objective trains on its data and takes none. context: TextCondition | AudioCondition | None-
Return the condition the model reads, text or audio, if any.
DiffusionRunConfig.derived_preset
Section titled “DiffusionRunConfig.derived_preset”def derived_preset() -> Preset | NoneThe run’s preset as it builds its process, what it leaves to the run derived from the run as it
stands, so an override of what it derives from (the images’ size, data.images.size, or the
autoencoder) derives it afresh, where the run records the preset as given: EDM’s sigma draw the
space’s, pixels without an autoencoder and latents with one, and a Flow resolution shift’s token
count the data’s images’ (ResolutionShift.at). A regime, sigmas or token count the preset states
wins, a model whose grid is not the 16-pixel one stating its own.
DiffusionRunConfig.sample_field
Section titled “DiffusionRunConfig.sample_field”def sample_field() -> FieldReturn the batch field the model generates, at the resolution the data comes in
(Dataset.facts): a clip’s video, else an image. Raises ValueError for a chain of neither, or
one whose size varies.
DiffusionRunConfig.mask_field
Section titled “DiffusionRunConfig.mask_field”def mask_field( sample: Field, wanted: Field, chain: Dataset | None = None, name: str = 'the chain',) -> FieldThe batch field an inpainting pipeline reads its mask from (wanted, the pipeline’s own): the
chain’s wanted.key (the run’s data, or chain, a validation split name) at the samples’ height
and width, one channel, as dew.data.images.RandomMask draws one or a dataset’s own column decodes
one. A chain that carries none is refused, naming RandomMask: an inpainting pipeline paints the
region its mask marks, and a plain image chain marks none.
DiffusionRunConfig.model_fields
Section titled “DiffusionRunConfig.model_fields”def model_fields(autoencoder: AutoEncoder | None) -> dict[str, Configured]Return the fields the run derives for the model, which ModelConfig.build
takes over the model’s arguments.
These are the channels the model denoises when the architecture takes them
as output_channels. The published families name theirs as their sources
do, in model.fields. On the model those build, an IntervalModel embeds
the duration under an interval process, and MeanFlow, whose loss
differentiates in time, turns a TimeScaled model’s time features at
SMOOTH_TIME_SCALE unless model.fields names a scale.
DiffusionRunConfig.build
Section titled “DiffusionRunConfig.build”def build(*, variables: Variables | None = None) -> DiffusionObjectiveBuild the objective, with each component built around its parameters.
Supplied variables are the authoritative saved snapshot. The encoders and the VAE read only configuration and tokenizer metadata, and use their subtrees of the supplied variables without loading source weights or casting storage.
A lora goes on the denoiser that pretrained loads, or, from scratch, on a
fresh draw of it from the run’s key. The objective then trains the adapter’s
factors and any loss head of its own.
DiffusionRunConfig.build_run
Section titled “DiffusionRunConfig.build_run”def build_run() -> PreparedThe objective build makes, with the run’s Hub sources pinned
(pinned), on its data, whose captions the objective’s conditions
tokenize. val_metrics score each validation, and Flow-GRPO samples
the batches it trains on (rollout).
DiffusionRunConfig.pinned
Section titled “DiffusionRunConfig.pinned”def pinned() -> DiffusionRunConfigReturn this run with its Hub sources pinned to the commits they resolve to now: an imported
adapter’s (RunConfig.pinned), pretrained and the alignment’s encoder, so the record names
the weights the run started from.
DiffusionRunConfig.default_metrics
Section titled “DiffusionRunConfig.default_metrics”def default_metrics(*, validates: bool) -> tuple[MetricConfig, ...]CLIP for an image run whose data validates, and nothing otherwise.
DiffusionRunConfig.metric
Section titled “DiffusionRunConfig.metric”def metric(config: MetricConfig) -> Metricconfig built, loading its own weights: a video run scores a VideoGrid against its video
field, which a PSNR or SSIM that states no field of its own reads there, and an image-only
metric raises a ValueError naming it here, before the trainer starts.
FlowRolloutConfig
Section titled “FlowRolloutConfig”class FlowRolloutConfig( reward: str = 'clip_score', groups: int = 4, steps: int = 11, train_steps: int | None = None,)A run’s flow RL rollout: groups samples of each prompt over steps time points, each scored by
the image metric reward names, an alias or an import path, against its own prompt, higher being
better (clip_score). Flow-GRPO trains the first train_steps transitions of each trajectory (all
of them when None) and DiffusionNFT each clean sample at the first train_steps times of its
sampler’s grid (the released 9 when None). build makes the objective’s rollout, a FlowRollout
or an NFTRollout.
FlowRolloutConfig.build
Section titled “FlowRolloutConfig.build”def build( objective: FlowGRPOObjective | DiffusionNFTObjective,) -> FlowRollout | NFTRolloutThe rollout over objective, scoring each sample with the metric’s per-sample measure.