espnet2.schedulers.warmup_reducelronplateau.WarmupReduceLROnPlateau

source

class espnet2.schedulers.warmup_reducelronplateau.WarmupReduceLROnPlateau(optimizer: Optimizer, warmup_steps: int | float = 25000, mode='min', factor=0.1, patience=10, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0, eps=1e-08, verbose=False)

Bases: AbsBatchStepScheduler, AbsValEpochStepScheduler

The WarmupReduceLROnPlateau scheduler.

This scheduler is the combination of WarmupLR and ReduceLROnPlateau:

WarmupLR: : lr = optimizer.lr * warmup_step ** 0.5 : * min(step ** -0.5, step * warmup_step ** -1.5)

WarmupReduceLROnPlateau: : if step <= warmup_step: : lr = optimizer.lr * warmup_step ** 0.5 : * min(step ** -0.5, step * warmup_step ** -1.5)
else: : lr = ( : optimizer.lr * factor if no improvement for a ‘patience’ number of epochs else optimizer.lr <br/> )

Note that the maximum lr equals to optimizer.lr in this scheduler.

property in_cooldown

is_better(a, best)

load_state_dict(state_dict)

state_dict()

step(metrics=None, epoch=None)