espnet2.universa.espnet_model.ESPnetUniversaModel

source

class espnet2.universa.espnet_model.ESPnetUniversaModel(universa: AbsUniversa, frontend: AbsFrontend)

Bases: AbsESPnetModel

ESPnet model for Universa.

Initialize ESPnet model for Universa.

collect_feats(audio: Tensor, audio_lengths: Tensor, ref_audio: Tensor | None = None, ref_audio_lengths: Tensor | None = None, **kwargs) → Dict[str, Tensor]

forward(audio: Tensor, audio_lengths: Tensor, metrics: Dict[str, Tensor], ref_audio: Tensor | None = None, ref_audio_lengths: Tensor | None = None, ref_text: Tensor | None = None, ref_text_lengths: Tensor | None = None, **kwargs) → Tuple[Tensor, Dict[str, Tensor], Tensor]

Extract input/reference features and compute the predictor’s loss.

Audio tensors are (batch, samples), text is (batch, tokens), and each length tensor has one entry per utterance. Metrics map names to scalar target batches. Returns loss, detached statistics, and batch weight.

inference(audio: Tensor, audio_lengths: Tensor, ref_audio: Tensor | None = None, ref_audio_lengths: Tensor | None = None, ref_text: Tensor | None = None, ref_text_lengths: Tensor | None = None, **kwargs) → Dict[str, Any]

Return predicted output as a dict.