libreasr.lib.optimizer

A collection of optimizers.

Functions

AdaHessian	A Optimizer for Adam with lr, mom, sqr_mom, eps and params
RAdamAdabelief	A Optimizer for Adam with lr, mom, sqr_mom, eps and params
adahessian_step	Step for Adam with lr on p
asqg
average_sqr_diag_hessian
radam_adabelief_step	Step for RAdam with lr on p
ranger_adabelief	Convenience method for Lookahead with RAdam

Classes

Apollo

Implements Atom algorithm. :param params: iterable of parameters to optimize or dicts defining parameter groups :type params: iterable :param lr: learning rate :type lr: float :param beta: coefficient used for computing running averages of gradient (default: 0.9) :type beta: float, optional :param eps: term added to the denominator to improve numerical stability (default: 1e-4) :type eps: float, optional :param warmup: number of warmup steps (default: 0) :type warmup: int, optional :param init_lr: initial learning rate for warmup (default: 0.01) :type init_lr: float, optional :param wd: weight decay coefficient (default: 0) :type wd: float, optional.