espnet2.optimizers.optim_groups.add_optimizer_hooks

source

espnet2.optimizers.optim_groups.add_optimizer_hooks(model, bias_weight_decay=False, normalization_weight_decay=False)

Set zero weight decay for some params

Set weight_decay=0.0 for parameters in model.no_weight_decay, for parameters with attribute _no_weight_decayTrue, for bias parameters if bias_weight_decayFalse, for normalization parameters if normalization_weight_decay==False

See: https://discuss.pytorch.org/t/weight-decay-only-for-weights-of-nn-linear-and-nn-conv/114348 # noqa