espnet2.asr_transducer.activation.Smish

source

class espnet2.asr_transducer.activation.Smish(alpha: float = 1.0, beta: float = 1.0)

Bases: Module

Smish activation definition.

Smish(x) = (alpha * x) * tanh(log(1 + sigmoid(beta * x))) : where alpha > 0 and beta > 0

Reference: https://www.mdpi.com/2079-9292/11/4/540/htm.

• Parameters:
  • alpha – Alpha value for Smish activation fomulation. (Usually, alpha = 1. If alpha <= 0, set value to 1).
  • beta – Beta value for Smish activation formulation. (Usually, beta = 1. If beta <= 0, set value to 1).

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor) → Tensor

Forward computation.