espnet2.asr.frontend.espnet_ssl.ESPnetSSLFrontend

source

class espnet2.asr.frontend.espnet_ssl.ESPnetSSLFrontend(fs: int | str = 16000, frontend_conf: dict | None = {'badim': 320, 'bdropout_rate': 0.0, 'blayers': 3, 'bnmask': 2, 'bprojs': 320, 'btype': 'blstmp', 'bunits': 300, 'delay': 3, 'ref_channel': -1, 'taps': 5, 'use_beamformer': False, 'use_dnn_mask_for_wpe': True, 'use_wpe': False, 'wdropout_rate': 0.0, 'wlayers': 3, 'wprojs': 320, 'wtype': 'blstmp', 'wunits': 300}, masking_conf: dict | None = {}, multilayer_feature: bool = False, layer: int = -1, freeze_encoder_steps: int = 0, mask_feats: bool = True, use_final_output: bool = True)

Bases: AbsFrontend

Frontend wrapper for SSL models trained in ESPnet.

• Parameters:
  • fs (int) – unused.
  • frontend_conf (dict) – make sure path_or_url has a value
  • multilayer_feature (bool) – whether to use weighted sum
  • layer (int) – 0-indexed layer to use if not using weighted sum
  • mask_feats (int) – whether to mask input feats to encoder
  • use_final_output (bool) – use final normalized output instead of last layer output. For post-LN model archs.

forward(input: Tensor, input_lengths: Tensor) → Tuple[Tensor, Tensor]

Defines the computation performed at every call.

Should be overridden by all subclasses.

NOTE

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size() → int

reload_pretrained_parameters()