espnet3.components.schedulers.linear_warmup_decay.LinearWarmupDecayLR

source

class espnet3.components.schedulers.linear_warmup_decay.LinearWarmupDecayLR(optimizer: Optimizer, warmup_steps: int | float = 20000, total_steps: int | float = 600000, start_factor: float = 1e-08, end_factor: float = 1e-08, last_epoch: int = -1)

Bases: _LRScheduler, AbsBatchStepScheduler

Linear warmup followed by linear decay.

This reproduces F5-TTS’s training schedule (https://github.com/SWivid/F5-TTS/blob/main/src/f5_tts/model/trainer.py): a linear warmup start_factor -> 1.0 over warmup_steps followed by a linear decay 1.0 -> end_factor over the remaining total_steps - warmup_steps updates. Stepped per optimizer update (scheduler_interval: step).

Unlike ESPnet’s WarmupLR (inverse-sqrt, non-terminating), the linear decay needs to know the total number of updates, so total_steps must be set to the planned training length (epochs * updates_per_epoch), matching how upstream computes decay_updates = total_updates - warmup_updates. Once total_steps is passed the learning rate is clamped at end_factor * base_lr; it never rises again.

Configure the schedule.

• Parameters:
  • optimizer – Optimizer whose learning rates are scheduled.
  • warmup_steps – Updates spent ramping start_factor up to 1.0. 0 starts at the base lr and decays immediately.
  • total_steps – Planned training length in optimizer updates. Must be greater than warmup_steps.
  • start_factor – Multiplier applied to the base lr at step 0. Must be in (0, 1].
  • end_factor – Multiplier the lr decays to, and is clamped at. Must be in [0, 1].
  • last_epoch – Index of the last update, -1 to start fresh.

Example

scheduler:
  _target_: espnet3.components.schedulers.                    linear_warmup_decay.LinearWarmupDecayLR
  warmup_steps: 20000
  total_steps: 600000
scheduler_interval: step

• Raises:ValueError – If warmup_steps is negative, if total_steps is not greater than warmup_steps, if start_factor is not in (0, 1], or if end_factor is not in [0, 1].

NOTE

total_steps is the planned horizon, not a stopping condition: training past it holds the floor rather than going negative. Because the schedule is stepped per update, scheduler_interval: step is required; leaving it at the epoch default stretches the warmup over the whole run.

get_lr()

Return the learning rate for every parameter group at this step.

• Returns: One learning rate per parameter group.

NOTE

Called by step(); do not call it directly. Except at the warmup handover it derives the next value from each group’s CURRENT lr rather than from base_lrs, so an extra manual call compounds the factor and corrupts the schedule. Read param_groups[i]["lr"] instead. This multiplicative form is what reproduces the original SequentialLR(LinearLR, LinearLR) bit-for-bit.