espnet2.asr.decoder.transformer_decoder.BaseTransformerDecoder

source

class espnet2.asr.decoder.transformer_decoder.BaseTransformerDecoder(vocab_size: int, encoder_output_size: int, dropout_rate: float = 0.1, positional_dropout_rate: float = 0.1, input_layer: str = 'embed', use_output_layer: bool = True, pos_enc_class=<class 'espnet.nets.pytorch_backend.transformer.embedding.PositionalEncoding'>, normalize_before: bool = True)

Bases: AbsDecoder, BatchScorerInterface, MaskParallelScorerInterface

Base class of Transfomer decoder module.

• Parameters:
  • vocab_size – output dim
  • encoder_output_size – dimension of attention
  • attention_heads – the number of heads of multi head attention
  • linear_units – the number of units of position-wise feed forward
  • num_blocks – the number of decoder blocks
  • dropout_rate – dropout rate
  • self_attention_dropout_rate – dropout rate for attention
  • input_layer – input layer type
  • use_output_layer – whether to use output layer
  • pos_enc_class – PositionalEncoding or ScaledPositionalEncoding
  • normalize_before – whether to use layer_norm before the first block
  • concat_after – whether to concat attention layer’s input and output if True, additional linear will be applied. i.e. x -> x + linear(concat(x, att(x))) if False, no additional linear will be applied. i.e. x -> x + att(x)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

batch_score(ys: Tensor, states: List[Any], xs: Tensor, return_hs: bool = False) → Tuple[Tensor, List[Any]]

Score new token batch.

• Parameters:
  • ys (torch.Tensor) – torch.int64 prefix tokens (n_batch, ylen).
  • states (List *[*Any ]) – Scorer states for prefix tokens.
  • xs (torch.Tensor) – The encoder feature that generates ys (n_batch, xlen, n_feat).
• Returns: Tuple of : batchfied scores for next token with shape of (n_batch, n_vocab) and next state list for ys.
• Return type: tuple[torch.Tensor, List[Any]]

batch_score_partially_AR(ys: Tensor, states: List[Any], xs: Tensor, yseq_lengths: Tensor) → Tuple[Tensor, List[Any]]

forward(hs_pad: Tensor, hlens: Tensor, ys_in_pad: Tensor, ys_in_lens: Tensor, return_hs: bool = False, return_all_hs: bool = False) → Tuple[Tensor, Tensor]

Forward decoder.

• Parameters:

  • hs_pad – encoded memory, float32 (batch, maxlen_in, feat)
  • hlens – (batch)
  • ys_in_pad – input token ids, int64 (batch, maxlen_out) if input_layer == “embed” input tensor (batch, maxlen_out, #mels) in the other cases
  • ys_in_lens – (batch)
  • return_hs – (bool) whether to return the last hidden output before output layer
  • return_all_hs – (bool) whether to return all the hidden intermediates

• Returns: tuple containing:

  x: decoded token score before softmax (batch, maxlen_out, token) : if use_output_layer is True,

  olens: (batch, )

• Return type: (tuple)

forward_one_step(tgt: Tensor, tgt_mask: Tensor, memory: Tensor, memory_mask: Tensor | None = None, *, cache: List[Tensor] | None = None, return_hs: bool = False) → Tuple[Tensor, List[Tensor]]

Forward one step.

• Parameters:
  • tgt – input token ids, int64 (batch, maxlen_out)
  • tgt_mask – input token mask, (batch, maxlen_out) dtype=torch.uint8 in PyTorch 1.2- dtype=torch.bool in PyTorch 1.2+ (include 1.2)
  • memory – encoded memory, float32 (batch, maxlen_in, feat)
  • memory_mask – encoded memory mask (batch, 1, maxlen_in)
  • cache – cached output list of (batch, max_time_out-1, size)
  • return_hs – dec hidden state corresponding to ys, used for searchable hidden ints
• Returns: NN output value and cache per self.decoders. y.shape` is (batch, maxlen_out, token)
• Return type: y, cache

forward_partially_AR(tgt: Tensor, tgt_mask: Tensor, tgt_lengths: Tensor, memory: Tensor, cache: List[Tensor] | None = None) → Tuple[Tensor, List[Tensor]]

Forward one step.

• Parameters:
  • tgt – input token ids, int64 (n_mask * n_beam, maxlen_out)
  • tgt_mask – input token mask, (n_mask * n_beam, maxlen_out) dtype=torch.uint8 in PyTorch 1.2- dtype=torch.bool in PyTorch 1.2+ (include 1.2)
  • tgt_lengths – (n_mask * n_beam, )
  • memory – encoded memory, float32 (batch, maxlen_in, feat)
  • cache – cached output list of (batch, max_time_out-1, size)
• Returns: NN output value and cache per self.decoders. y.shape` is (batch, maxlen_out, token)
• Return type: y, cache

score(ys, state, x, return_hs=False)

Score.