espnet.nets.pytorch_backend.conformer.encoder.Encoder

source

class espnet.nets.pytorch_backend.conformer.encoder.Encoder(idim, attention_dim=256, attention_heads=4, linear_units=2048, num_blocks=6, dropout_rate=0.1, positional_dropout_rate=0.1, attention_dropout_rate=0.0, input_layer='conv2d', normalize_before=True, concat_after=False, positionwise_layer_type='linear', positionwise_conv_kernel_size=1, macaron_style=False, pos_enc_layer_type='abs_pos', selfattention_layer_type='selfattn', activation_type='swish', use_cnn_module=False, zero_triu=False, cnn_module_kernel=31, padding_idx=-1, stochastic_depth_rate=0.0, intermediate_layers=None, ctc_softmax=None, conditioning_layer_dim=None)

Bases: Module

Conformer encoder module.

• Parameters:
  • idim (int) – Input dimension.
  • attention_dim (int) – Dimension of attention.
  • attention_heads (int) – The number of heads of multi head attention.
  • linear_units (int) – The number of units of position-wise feed forward.
  • num_blocks (int) – The number of decoder blocks.
  • dropout_rate (float) – Dropout rate.
  • positional_dropout_rate (float) – Dropout rate after adding positional encoding.
  • attention_dropout_rate (float) – Dropout rate in attention.
  • input_layer (Union *[*str , torch.nn.Module ]) – Input layer type.
  • normalize_before (bool) – Whether to use layer_norm before the first block.
  • concat_after (bool) – Whether to concat attention layer’s input and output. if True, additional linear will be applied. i.e. x -> x + linear(concat(x, att(x))) if False, no additional linear will be applied. i.e. x -> x + att(x)
  • positionwise_layer_type (str) – “linear”, “conv1d”, or “conv1d-linear”.
  • positionwise_conv_kernel_size (int) – Kernel size of positionwise conv1d layer.
  • macaron_style (bool) – Whether to use macaron style for positionwise layer.
  • pos_enc_layer_type (str) – Encoder positional encoding layer type.
  • selfattention_layer_type (str) – Encoder attention layer type.
  • activation_type (str) – Encoder activation function type.
  • use_cnn_module (bool) – Whether to use convolution module.
  • zero_triu (bool) – Whether to zero the upper triangular part of attention matrix.
  • cnn_module_kernel (int) – Kernerl size of convolution module.
  • padding_idx (int) – Padding idx for input_layer=embed.
  • stochastic_depth_rate (float) – Maximum probability to skip the encoder layer.
  • intermediate_layers (Union *[*List *[*int ] , None ]) – indices of intermediate CTC layer. indices start from 1. if not None, intermediate outputs are returned (which changes return type signature.)

Construct an Encoder object.

forward(xs, masks)

Encode input sequence.

• Parameters:
  • xs (torch.Tensor) – Input tensor (#batch, time, idim).
  • masks (torch.Tensor) – Mask tensor (#batch, 1, time).
• Returns: Output tensor (#batch, time, attention_dim). torch.Tensor: Mask tensor (#batch, 1, time).
• Return type: torch.Tensor