espnet2.spk.encoder.ecapa_tdnn_encoder.EcapaTdnnEncoder

source

class espnet2.spk.encoder.ecapa_tdnn_encoder.EcapaTdnnEncoder(input_size: int, block: str = 'EcapaBlock', model_scale: int = 8, ndim: int = 1024, output_size: int = 1536, num_blocks: int | None = None, dilations: Sequence[int] | None = None)

Bases: AbsEncoder

ECAPA-TDNN encoder. Extracts frame-level ECAPA-TDNN embeddings from

mel-filterbank energy or MFCC features. Paper: B Desplanques at el.,

``

ECAPA-TDNN: Emphasized Channel Attention,

Propagation and Aggregation in TDNN Based Speaker Verification,’’ in Proc. INTERSPEECH, 2020.

• Parameters:
  • input_size – input feature dimension.
  • block – type of encoder block class to use.
  • model_scale – scale value of the Res2Net architecture.
  • ndim – dimensionality of the hidden representation.
  • output_size – output embedding dimension.
  • num_blocks – number of SE-Res2 blocks, with dilations 2, 3, …, num_blocks + 1. The default is the paper’s three blocks (2, 3, 4). Four blocks (2, 3, 4, 5) follow the IDLab VoxSRC-20 system, which adds a fourth SE-Res2Block with dilation 5 (J. Thienpondt et al., The IDLAB VoxSRC-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification,'' arXiv:2010.11255, Sec. 2). For five or more, choosing the dilations with ``dilations is recommended.
  • dilations – dilation of each SE-Res2 block, such as [2, 3, 4, 2, 3, 4]; the number of blocks is its length. Dilations do not change the parameter shapes, so build the model from the configuration it was trained with (the encoder_conf saved in its config.yaml).

Every block takes the first convolution’s output plus the outputs of all earlier blocks, and the 1x1 convolution layer4 aggregates all block outputs. The first three blocks keep the names layer1-layer3 and further blocks are extra_layers, so three-block checkpoints load unchanged.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x: Tensor)

Calculate forward propagation.

• Parameters:x (torch.Tensor) – Input tensor (#batch, L, input_size).
• Returns: Output tensor (#batch, L, output_size).
• Return type: torch.Tensor

output_size() → int