Skip to content

dew.training.budget

How long a run trains and what ends it early: its one budget, in steps, epochs, examples or tokens (run_length, run_steps), the stop policies Trainer.fit takes beside it (Plateau, Pause), and its cadences resolved against it (every_step).

NameSummary
PauseEnds the run once it has trained fraction of its budget, its state and data position written in full, so a run resumed from that checkpoint trains on as one that never paused: the budget still sizes the learning-rate schedule.
PlateauStops training after evals eligible evaluations without an improvement larger than min_delta.
every_stepvalue, a cadence of steps or a share of the run such as 5% (dew._records.share), resolved once against the run’s planned length: a share of steps microsteps, accumulation an update, the whole updates nearest it and at least one; of a token budget alone, a cadence of that many committed tokens (Counted); of a run bounded to examples no pass counts steps ahead of, a cadence of that many consumed examples.
recordedcadence as a run’s plan records it: its steps, its committed tokens or its duration.
run_lengthRefuse a run whose length is named twice, by two of steps, epochs and examples, or, where required, not at all; tokens caps any of them or bounds a run alone.
run_stepsThe microsteps a run takes over loader, the one policy for a fit’s and a config’s run length: steps, epochs of it, whose ramp stages begin where updates do, as the stream cuts them, or the steps that read the examples it is bounded to (Loader.example_steps), a read with no record count bounded to its epochs’ source examples among them, or None where tokens alone bounds it.
run_tokensThe run’s token budget (budget_tokens), refused where the objective counts no tokens.

dataclass source

class Pause(fraction: float)

Ends the run once it has trained fraction of its budget, its state and data position written in full, so a run resumed from that checkpoint trains on as one that never paused: the budget still sizes the learning-rate schedule. A budget of steps, epochs or examples pauses at the whole update nearest the fraction, a token budget at the first update whose tokens reach it, and with both the first reached. Resume without it, or with a later fraction, to go on; with the same one the run is already where it pauses, so nothing trains. A run bounded in examples that counts no steps ahead pauses at the first step whose batch brings its consumed examples to the fraction.

def at(
steps: int | None,
tokens: int | None,
accumulation: int,
examples: int | None = None,
) -> tuple[int | None, int | None, int | None]

The step, the real tokens and the consumed examples a run of steps microsteps, accumulation an update, of tokens and, where no pass counts its steps ahead, bounded to examples, pauses at, each None where that budget is not set. Refuses a pause before the first update or at the budget’s end, where the run ends anyway.

dataclass source

class Plateau(
metric: Annotated[Metric | TrainingScalar | Summary | types.MethodType | str, tyro.conf.arg(constructor=str)],
evals: int = 5,
min_delta: float = 0.0,
mode: Literal['min', 'max'] | None = None,
split: str | None = None,
)

Stops training after evals eligible evaluations without an improvement larger than min_delta. A Summary is eligible at the evaluations that form it, those that scored every split it weighs. The metric is the object fit scores, or its recorded name, as Best takes one (val/loss).

function source

def every_step(
value: Cadence | str,
steps: int | None,
tokens: int | None,
accumulation: int,
*,
name: str,
examples: int | None = None,
) -> Cadence

value, a cadence of steps or a share of the run such as 5% (dew._records.share), resolved once against the run’s planned length: a share of steps microsteps, accumulation an update, the whole updates nearest it and at least one; of a token budget alone, a cadence of that many committed tokens (Counted); of a run bounded to examples no pass counts steps ahead of, a cadence of that many consumed examples. A share of a run with no planned length is refused by name.

function source

def recorded(cadence: Cadence | str | datetime.timedelta) -> JSON

cadence as a run’s plan records it: its steps, its committed tokens or its duration.

function source

def run_length(
steps: int | None,
epochs: int | None,
tokens: int | None,
examples: int | None = None,
*,
required: bool = True,
) -> None

Refuse a run whose length is named twice, by two of steps, epochs and examples, or, where required, not at all; tokens caps any of them or bounds a run alone.

function source

def run_steps(
loader: Loader,
steps: int | None,
epochs: int | None,
tokens: int | None,
accumulation: int,
examples: int | None = None,
) -> int | None

The microsteps a run takes over loader, the one policy for a fit’s and a config’s run length: steps, epochs of it, whose ramp stages begin where updates do, as the stream cuts them, or the steps that read the examples it is bounded to (Loader.example_steps), a read with no record count bounded to its epochs’ source examples among them, or None where tokens alone bounds it. Refuses a run with no length or two, and epochs of a stream with no count of either.

function source

def run_tokens(loader: Loader, tokens: int | None, *, counts_tokens: bool) -> int | None

The run’s token budget (budget_tokens), refused where the objective counts no tokens.