espnet2.asr_transducer.activation.FTSwish

source

class espnet2.asr_transducer.activation.FTSwish(threshold: float = -0.2, mean_shift: float = 0)

Bases: Module

Flatten-T Swish activation definition.

FTSwish(x) = x * sigmoid(x) + threshold : where FTSwish(x) < 0 = threshold

Reference: https://arxiv.org/abs/1812.06247

• Parameters:
  • threshold – Threshold value for FTSwish activation formulation. (threshold < 0)
  • mean_shift – Mean shifting value for FTSwish activation formulation. (applied only if != 0, disabled by default)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor) → Tensor

Forward computation.