espnet2.asr_transducer.decoder.modules.mega.feed_forward.NormalizedPositionwiseFeedForward

source

class espnet2.asr_transducer.decoder.modules.mega.feed_forward.NormalizedPositionwiseFeedForward(size: int, hidden_size: int, normalization: ~torch.nn.modules.module.Module = <class 'torch.nn.modules.normalization.LayerNorm'>, activation: ~torch.nn.modules.module.Module = <class 'torch.nn.modules.activation.ReLU'>, dropout_rate: float = 0.0)

Bases: Module

NormalizedPositionFeedForward module definition.

• Parameters:
  • size – Input/Output size.
  • hidden_size – Hidden size.
  • normalization – Normalization module.
  • activation – Activation function.
  • dropout_rate – Dropout rate.

Construct an NormalizedPositionwiseFeedForward object.

forward(x: Tensor) → Tensor

Compute feed-forward module.

• Parameters:x – NormalizedPositionwiseFeedForward input sequences. (B, L, size)
• Returns: NormalizedPositionwiseFeedForward output sequences. (B, L, size)
• Return type: x

reset_parameters(val: float = 0.0, std: float = 0.02) → None

Reset module parameters.

• Parameters:
  • val – Initialization value.
  • std – Standard deviation.