espnet2.asr_transducer.activation.Mish

source

class espnet2.asr_transducer.activation.Mish(softplus_beta: float = 1.0, softplus_threshold: int = 20, use_builtin: bool = False)

Bases: Module

Mish activation definition.

Mish(x) = x * tanh(softplus(x))

Reference: https://arxiv.org/abs/1908.08681.

• Parameters:
  • softplus_beta – Beta value for softplus activation formulation. (Usually 0 > softplus_beta >= 2)
  • softplus_threshold – Values above this revert to a linear function. (Usually 10 > softplus_threshold >= 20)
  • use_builtin – Whether to use PyTorch activation function if available.

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor) → Tensor

Forward computation.