dew.laprop
DreamerV3’s optimizer (laprop): LaProp’s direction (scale_by_laprop) of the update clipped adaptively
(clip_by_agc), at DreamerV3’s learning rate. dew.optim registers laprop among its optimizers.
| Name | Summary |
|---|---|
clip_by_agc | Adaptive gradient clipping (Brock et al., arXiv 2102.06171) as DreamerV3’s code takes it (embodied/jax/opt.py:109-123): each parameter’s update scaled down until its norm is at most clip times the parameter’s own, which floor keeps from reaching zero with the parameter (a bias drawn at zero still moves), one norm over the whole array. |
laprop | DreamerV3’s optimizer (dreamerv3/agent.py:342-379): the update clipped adaptively to agc of each parameter’s norm (clip_by_agc), LaProp’s direction of it (scale_by_laprop, momentum b1, mean square b2, eps), stepped at learning_rate. |
scale_by_laprop | LaProp’s direction (Ziyin et al., arXiv 2002.04839) as DreamerV3’s code forms it (embodied/jax/opt.py:126-164): each gradient over the root of its bias-corrected mean square (decay b2, eps added outside the root), then the bias-corrected momentum of decay b1 of those quotients. |
clip_by_agc
Section titled “clip_by_agc”def clip_by_agc(clip: float = 0.3, floor: float = 0.001) -> optax.GradientTransformationAdaptive gradient clipping (Brock et al., arXiv 2102.06171) as DreamerV3’s code takes it
(embodied/jax/opt.py:109-123): each parameter’s update scaled down until its norm is at most clip
times the parameter’s own, which floor keeps from reaching zero with the parameter (a bias drawn at
zero still moves), one norm over the whole array. A clip of 0 clips nothing. The update reads the
parameters.
laprop
Section titled “laprop”def laprop( learning_rate: float | optax.Schedule | None = None, b1: float = 0.9, b2: float = 0.999, eps: float = 1e-20, agc: float = 0.3,) -> optax.GradientTransformationDreamerV3’s optimizer (dreamerv3/agent.py:342-379): the update clipped adaptively to agc of each
parameter’s norm (clip_by_agc), LaProp’s direction of it (scale_by_laprop, momentum b1, mean
square b2, eps), stepped at learning_rate. Where that is None it is the code’s, 4e-5 ramped
linearly from zero over the first 1000 updates.
scale_by_laprop
Section titled “scale_by_laprop”def scale_by_laprop( b1: float = 0.9, b2: float = 0.999, eps: float = 1e-20,) -> optax.GradientTransformationLaProp’s direction (Ziyin et al., arXiv 2002.04839) as DreamerV3’s code forms it
(embodied/jax/opt.py:126-164): each gradient over the root of its bias-corrected mean square (decay
b2, eps added outside the root), then the bias-corrected momentum of decay b1 of those quotients.
Adam divides the momentum of the gradients; LaProp takes the momentum of the divided ones. Both bias
corrections form in double (bias_corrected). The state is optax.ScaleByAdamState’s, its moments in
float32 at least, as the code keeps them.