espnet2.schedulers.warmup_step_lr.WarmupStepLR

source

class espnet2.schedulers.warmup_step_lr.WarmupStepLR(optimizer: Optimizer, warmup_steps: int | float = 25000, steps_per_epoch: int = 10000, step_size: int = 1, gamma: float = 0.1, last_epoch: int = -1)

Bases: _LRScheduler, AbsBatchStepScheduler

The WarmupStepLR scheduler.

This scheduler is the combination of WarmupLR and StepLR:

WarmupLR: : lr = optimizer.lr * warmup_step ** 0.5 : * min(step ** -0.5, step * warmup_step ** -1.5)

WarmupStepLR: : if step <= warmup_step: : lr = optimizer.lr * warmup_step ** 0.5 : * min(step ** -0.5, step * warmup_step ** -1.5)
else: : lr = optimizer.lr * (gamma ** (epoch//step_size))

Note that the maximum lr equals to optimizer.lr in this scheduler.

get_lr()