Skip to content

dew.laprop

DreamerV3’s optimizer (laprop): LaProp’s direction (scale_by_laprop) of the update clipped adaptively (clip_by_agc), at DreamerV3’s learning rate. dew.optim registers laprop among its optimizers.

NameSummary
clip_by_agcAdaptive gradient clipping (Brock et al., arXiv 2102.06171) as DreamerV3’s code takes it (embodied/jax/opt.py:109-123): each parameter’s update scaled down until its norm is at most clip times the parameter’s own, which floor keeps from reaching zero with the parameter (a bias drawn at zero still moves), one norm over the whole array.
lapropDreamerV3’s optimizer (dreamerv3/agent.py:342-379): the update clipped adaptively to agc of each parameter’s norm (clip_by_agc), LaProp’s direction of it (scale_by_laprop, momentum b1, mean square b2, eps), stepped at learning_rate.
scale_by_lapropLaProp’s direction (Ziyin et al., arXiv 2002.04839) as DreamerV3’s code forms it (embodied/jax/opt.py:126-164): each gradient over the root of its bias-corrected mean square (decay b2, eps added outside the root), then the bias-corrected momentum of decay b1 of those quotients.

function source

def clip_by_agc(clip: float = 0.3, floor: float = 0.001) -> optax.GradientTransformation

Adaptive gradient clipping (Brock et al., arXiv 2102.06171) as DreamerV3’s code takes it (embodied/jax/opt.py:109-123): each parameter’s update scaled down until its norm is at most clip times the parameter’s own, which floor keeps from reaching zero with the parameter (a bias drawn at zero still moves), one norm over the whole array. A clip of 0 clips nothing. The update reads the parameters.

function source

def laprop(
learning_rate: float | optax.Schedule | None = None,
b1: float = 0.9,
b2: float = 0.999,
eps: float = 1e-20,
agc: float = 0.3,
) -> optax.GradientTransformation

DreamerV3’s optimizer (dreamerv3/agent.py:342-379): the update clipped adaptively to agc of each parameter’s norm (clip_by_agc), LaProp’s direction of it (scale_by_laprop, momentum b1, mean square b2, eps), stepped at learning_rate. Where that is None it is the code’s, 4e-5 ramped linearly from zero over the first 1000 updates.

function source

def scale_by_laprop(
b1: float = 0.9,
b2: float = 0.999,
eps: float = 1e-20,
) -> optax.GradientTransformation

LaProp’s direction (Ziyin et al., arXiv 2002.04839) as DreamerV3’s code forms it (embodied/jax/opt.py:126-164): each gradient over the root of its bias-corrected mean square (decay b2, eps added outside the root), then the bias-corrected momentum of decay b1 of those quotients. Adam divides the momentum of the gradients; LaProp takes the momentum of the divided ones. Both bias corrections form in double (bias_corrected). The state is optax.ScaleByAdamState’s, its moments in float32 at least, as the code keeps them.