espnet2.asr_transducer.activation.Swish

source

class espnet2.asr_transducer.activation.Swish(beta: float = 1.0, use_builtin: bool = False)

Bases: Module

Swish activation definition.

Swish(x) = (beta * x) * sigmoid(x) : where beta = 1 defines standard Swish activation.

References

https://arxiv.org/abs/2108.12943 / https://arxiv.org/abs/1710.05941v1. E-swish variant: https://arxiv.org/abs/1801.07145.

• Parameters:
  • beta – Beta parameter for E-Swish. (beta >= 1. If beta < 1, use standard Swish).
  • use_builtin – Whether to use PyTorch function if available.

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor) → Tensor

Forward computation.