espnet2.asr_transducer.normalization.ScaleNorm

source

class espnet2.asr_transducer.normalization.ScaleNorm(normalized_shape: int, eps: float = 1e-05)

Bases: Module

ScaleNorm module definition.

Reference: https://arxiv.org/pdf/1910.05895.pdf

• Parameters:
  • normalized_shape – Expected size.
  • eps – Value added to the denominator for numerical stability.

Construct a ScaleNorm object.

forward(x: Tensor) → Tensor

Compute scale normalization.

• Parameters:x – Input sequences. (B, T, D_hidden)
• Returns: Output sequences. (B, T, D_hidden)