libreasr.lib.optimizer.Apollo

class libreasr.lib.optimizer.Apollo(params, lr, beta=0.9, eps=0.0001, warmup=100, init_lr=0.01, wd=0)

Bases: fastai2.optimizer.Optimizer

Implements Atom algorithm. :param params: iterable of parameters to optimize or dicts defining

parameter groups

Parameters

• lr (float) – learning rate

• beta (float, optional) – coefficient used for computing running averages of gradient (default: 0.9)

• eps (float, optional) – term added to the denominator to improve numerical stability (default: 1e-4)

• warmup (int, optional) – number of warmup steps (default: 0)

• init_lr (float, optional) – initial learning rate for warmup (default: 0.01)

• wd (float, optional) – weight decay coefficient (default: 0)

Methods

all_params
clear_state
freeze
freeze_to
load_state_dict
set_freeze
set_hyper
set_hypers
state_dict
step	Performs a single optimization step. :param closure: A closure that reevaluates the model and returns the loss. :type closure: callable, optional.
unfreeze
zero_grad

Attributes

param_groups

step(closure=None)

Performs a single optimization step. :param closure: A closure that reevaluates the model

and returns the loss.