espnet2.asr.state_spaces.model.SequenceModel

source

class espnet2.asr.state_spaces.model.SequenceModel(d_model, n_layers=1, transposed=False, dropout=0.0, tie_dropout=False, prenorm=True, n_repeat=1, layer=None, residual=None, norm=None, pool=None, track_norms=True, dropinp=0.0, drop_path=0.0)

Bases: SequenceModule

Isotropic deep sequence model backbone, in the style of ResNets / Transformers.

The SequenceModel class implements a generic (batch, length, d_input) -> (batch, length, d_output) transformation

• Parameters:
  • d_model – Resize input (useful for deep models with residuals)
  • n_layers – Number of layers
  • transposed – Transpose inputs so each layer receives (batch, dim, length)
  • dropout – Dropout parameter applied on every residual and every layer
  • tie_dropout – Tie dropout mask across sequence like nn.Dropout1d/nn.Dropout2d
  • prenorm – Pre-norm vs. post-norm
  • n_repeat – Each layer is repeated n times per stage before applying pooling
  • layer – Layer config, must be specified
  • residual – Residual config
  • norm – Normalization config (e.g. layer vs batch)
  • pool – Config for pooling layer per stage
  • track_norms – Log norms of each layer output
  • dropinp – Input dropout
  • drop_path – Stochastic depth for each residual path

Initializes internal Module state, shared by both nn.Module and ScriptModule.

property d_state

Return dimension of output of self.state_to_tensor.

default_state(*batch_shape, device=None)

Create initial state for a batch of inputs.

forward(inputs, *args, state=None, **kwargs)

Forward pass.

A sequence-to-sequence transformation with an optional state.

Generally, this should map a tensor of shape (batch, length, self.d_model) to (batch, length, self.d_output)

Additionally, it returns a “state” which can be any additional information For example, RNN and SSM layers may return their hidden state, while some types of transformer layers (e.g. Transformer-XL) may want to pass a state as well

property state_to_tensor

Return a function mapping a state to a single tensor.

This method should be implemented if one wants to use the hidden state insteadof the output sequence for final prediction. Currently only used with the StateDecoder.

step(x, state, **kwargs)

Step the model recurrently for one step of the input sequence.

For example, this should correspond to unrolling an RNN for one step. If the forward pass has signature (B, L, H1) -> (B, L, H2), this method should generally have signature (B, H1) -> (B, H2) with an optional recurrent state.