espnet.asr.asr_utils.add_gradient_noise

source

espnet.asr.asr_utils.add_gradient_noise(model, iteration, duration=100, eta=1.0, scale_factor=0.55)

Add noise from a standard normal distribution to the gradients.

The standard deviation (sigma) is controlled by the three hyper-parameters below. sigma goes to zero (no noise) with more iterations.

• Parameters:
  • model (torch.nn.model) – Model.
  • iteration (int) – Number of iterations.
  • duration (int) – Number of durations to control the interval of the sigma change.
  • eta (float) – The magnitude of sigma.
  • scale_factor (float) – The scale of sigma.