dew.training.budget
How long a run trains and what ends it early: its one budget, in steps, epochs, examples or tokens
(run_length, run_steps), the stop policies Trainer.fit takes beside it (Plateau, Pause), and
its cadences resolved against it (every_step).
| Name | Summary |
|---|---|
Pause | Ends the run once it has trained fraction of its budget, its state and data position written in full, so a run resumed from that checkpoint trains on as one that never paused: the budget still sizes the learning-rate schedule. |
Plateau | Stops training after evals eligible evaluations without an improvement larger than min_delta. |
every_step | value, a cadence of steps or a share of the run such as 5% (dew._records.share), resolved once against the run’s planned length: a share of steps microsteps, accumulation an update, the whole updates nearest it and at least one; of a token budget alone, a cadence of that many committed tokens (Counted); of a run bounded to examples no pass counts steps ahead of, a cadence of that many consumed examples. |
recorded | cadence as a run’s plan records it: its steps, its committed tokens or its duration. |
run_length | Refuse a run whose length is named twice, by two of steps, epochs and examples, or, where required, not at all; tokens caps any of them or bounds a run alone. |
run_steps | The microsteps a run takes over loader, the one policy for a fit’s and a config’s run length: steps, epochs of it, whose ramp stages begin where updates do, as the stream cuts them, or the steps that read the examples it is bounded to (Loader.example_steps), a read with no record count bounded to its epochs’ source examples among them, or None where tokens alone bounds it. |
run_tokens | The run’s token budget (budget_tokens), refused where the objective counts no tokens. |
class Pause(fraction: float)Ends the run once it has trained fraction of its budget, its state and data position
written in full, so a run resumed from that checkpoint trains on as one that never paused: the
budget still sizes the learning-rate schedule. A budget of steps, epochs or examples pauses at
the whole update nearest the fraction, a token budget at the first update whose tokens reach
it, and with both the first reached. Resume without it, or with a later fraction, to go on;
with the same one the run is already where it pauses, so nothing trains. A run bounded in examples
that counts no steps ahead pauses at the first step whose batch brings its consumed examples to the
fraction.
Pause.at
Section titled “Pause.at”def at( steps: int | None, tokens: int | None, accumulation: int, examples: int | None = None,) -> tuple[int | None, int | None, int | None]The step, the real tokens and the consumed examples a run of steps microsteps,
accumulation an update, of tokens and, where no pass counts its steps ahead, bounded to
examples, pauses at, each None where that budget is not set. Refuses a pause before the first
update or at the budget’s end, where the run ends anyway.
Plateau
Section titled “Plateau”class Plateau( metric: Annotated[Metric | TrainingScalar | Summary | types.MethodType | str, tyro.conf.arg(constructor=str)], evals: int = 5, min_delta: float = 0.0, mode: Literal['min', 'max'] | None = None, split: str | None = None,)Stops training after evals eligible evaluations without an improvement larger than min_delta. A
Summary is eligible at the evaluations that form it, those that scored every split it weighs. The
metric is the object fit scores, or its recorded name, as Best takes one (val/loss).
every_step
Section titled “every_step”def every_step( value: Cadence | str, steps: int | None, tokens: int | None, accumulation: int, *, name: str, examples: int | None = None,) -> Cadencevalue, a cadence of steps or a share of the run such as 5% (dew._records.share), resolved
once against the run’s planned length: a share of steps microsteps, accumulation an update,
the whole updates nearest it and at least one; of a token budget alone, a cadence of that many
committed tokens (Counted); of a run bounded to examples no pass counts steps ahead of, a cadence
of that many consumed examples. A share of a run with no planned length is refused by
name.
recorded
Section titled “recorded”def recorded(cadence: Cadence | str | datetime.timedelta) -> JSONcadence as a run’s plan records it: its steps, its committed tokens or its duration.
run_length
Section titled “run_length”def run_length( steps: int | None, epochs: int | None, tokens: int | None, examples: int | None = None, *, required: bool = True,) -> NoneRefuse a run whose length is named twice, by two of steps, epochs and examples, or, where
required, not at all; tokens caps any of them or bounds a run alone.
run_steps
Section titled “run_steps”def run_steps( loader: Loader, steps: int | None, epochs: int | None, tokens: int | None, accumulation: int, examples: int | None = None,) -> int | NoneThe microsteps a run takes over loader, the one policy for a fit’s and a config’s run length:
steps, epochs of it, whose ramp stages begin where updates do, as the stream cuts them, or the
steps that read the examples it is bounded to (Loader.example_steps), a read with no record count
bounded to its epochs’ source examples among them, or None where tokens alone bounds it. Refuses a
run with no length or two, and epochs of a stream with no count of either.
run_tokens
Section titled “run_tokens”def run_tokens(loader: Loader, tokens: int | None, *, counts_tokens: bool) -> int | NoneThe run’s token budget (budget_tokens), refused where the objective counts no tokens.