espnet2.asr.encoder.beats_encoder.TransformerSentenceEncoderLayer

source

class espnet2.asr.encoder.beats_encoder.TransformerSentenceEncoderLayer(embedding_dim: float = 768, ffn_embedding_dim: float = 3072, num_attention_heads: float = 8, dropout: float = 0.1, attention_dropout: float = 0.1, activation_dropout: float = 0.1, activation_fn: str = 'relu', layer_norm_first: bool = False, deep_norm: bool = False, has_relative_attention_bias: bool = False, num_buckets: int = 0, max_distance: int = 0, rescale_init: bool = False, gru_rel_pos: bool = False, encoder_layers: int = 0)

Bases: Module

Transformer encoder layer.

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor, self_attn_mask: Tensor | None = None, self_attn_padding_mask: Tensor | None = None, need_weights: bool = False, pos_bias=None)

Forward pass.