espnet2.asr_transducer.activation.get_activation

source

espnet2.asr_transducer.activation.get_activation(activation_type: str, ftswish_threshold: float = -0.2, ftswish_mean_shift: float = 0.0, hardtanh_min_val: int = -1.0, hardtanh_max_val: int = 1.0, leakyrelu_neg_slope: float = 0.01, smish_alpha: float = 1.0, smish_beta: float = 1.0, softplus_beta: float = 1.0, softplus_threshold: int = 20, swish_beta: float = 1.0) → Module

Return activation function.

• Parameters:
  • activation_type – Activation function type.
  • ftswish_threshold – Threshold value for FTSwish activation formulation.
  • ftswish_mean_shift – Mean shifting value for FTSwish activation formulation.
  • hardtanh_min_val – Minimum value of the linear region range for HardTanh.
  • hardtanh_max_val – Maximum value of the linear region range for HardTanh.
  • leakyrelu_neg_slope – Negative slope value for LeakyReLU activation formulation.
  • smish_alpha – Alpha value for Smish activation fomulation.
  • smish_beta – Beta value for Smish activation formulation.
  • softplus_beta – Beta value for softplus activation formulation in Mish.
  • softplus_threshold – Values above this revert to a linear function in Mish.
  • swish_beta – Beta value for Swish variant formulation.
• Returns: Activation function.