espnet2.schedulers.cosine_anneal_warmup_restart.CosineAnnealingWarmupRestarts

source

class espnet2.schedulers.cosine_anneal_warmup_restart.CosineAnnealingWarmupRestarts(optimizer: Optimizer, first_cycle_steps: int, cycle_mult: float = 1.0, max_lr: float = 0.1, min_lr: float = 0.001, warmup_steps: int = 0, gamma: float = 1.0, last_epoch: int = -1)

Bases: _LRScheduler, AbsBatchStepScheduler

Cosine Annealing Warmup Restart.

optimizer (Optimizer): Wrapped optimizer. first_cycle_steps (int): First cycle step size. cycle_mult(float): Cycle steps magnification. Default: -1. max_lr(float): First cycle’s max learning rate. Default: 0.1. min_lr(float): Min learning rate. Default: 0.001. warmup_steps(int): Linear warmup step size. Default: 0. gamma(float): Decrease rate of max learning rate by cycle. Default: 1. last_epoch (int): The index of last epoch. Default: -1.

get_lr()

init_lr()

step(epoch=None)