espnet2.asr.state_spaces.block.SequenceResidualBlock

source

class espnet2.asr.state_spaces.block.SequenceResidualBlock(d_input, i_layer=None, prenorm=True, dropout=0.0, tie_dropout=False, transposed=False, layer=None, residual=None, norm=None, pool=None, drop_path=0.0)

Bases: SequenceModule

Residual block wrapper for black box layer.

The SequenceResidualBlock class implements a generic (batch, length, d_input) -> (batch, length, d_input) transformation

• Parameters:
  • d_input – Input feature dimension
  • i_layer – Layer index, only needs to be passed into certain residuals like Decay
  • dropout – Dropout for black box module
  • tie_dropout – Tie dropout mask across sequence like nn.Dropout1d/nn.Dropout2d
  • transposed – Transpose inputs so each layer receives (batch, dim, length)
  • layer – Config for black box module
  • residual – Config for residual function
  • norm – Config for normalization layer
  • pool – Config for pooling layer per stage
  • drop_path – Drop ratio for stochastic depth

Initializes internal Module state, shared by both nn.Module and ScriptModule.

property d_output

Output dimension of model.

This attribute is required for all SequenceModule instantiations. It is used by the rest of the pipeline (e.g. model backbone, decoder) to track the internal shapes of the full model.

property d_state

Return dimension of output of self.state_to_tensor.

default_state(*args, **kwargs)

Create initial state for a batch of inputs.

forward(x, state=None, **kwargs)

Forward pass.

A sequence-to-sequence transformation with an optional state.

Generally, this should map a tensor of shape (batch, length, self.d_model) to (batch, length, self.d_output)

Additionally, it returns a “state” which can be any additional information For example, RNN and SSM layers may return their hidden state, while some types of transformer layers (e.g. Transformer-XL) may want to pass a state as well

property state_to_tensor

Return a function mapping a state to a single tensor.

This method should be implemented if one wants to use the hidden state insteadof the output sequence for final prediction. Currently only used with the StateDecoder.

step(x, state, **kwargs)

Step the model recurrently for one step of the input sequence.

For example, this should correspond to unrolling an RNN for one step. If the forward pass has signature (B, L, H1) -> (B, L, H2), this method should generally have signature (B, H1) -> (B, H2) with an optional recurrent state.