espnet2.train.preprocessor.SpeechLMPreprocessor

source

class espnet2.train.preprocessor.SpeechLMPreprocessor(token_list: List, token_bias: Dict, encoder_decoder_format: bool = False, codec_token_per_frame: int = 1, codec_token_in_use: int | None = None, unk_symbol: str = '<unk>', space_symbol: str = '<space>', non_linguistic_symbols: Path | str | Iterable[str] | None = None, g2p_type: str | None = None, bpemodel: Path | str | Iterable[str] | None = None, bpe_encode_kwargs: Dict | None = None, text_cleaner: str | None = None, speaker_prompt_length: int = 1800)

Bases: AbsPreprocessor

Preprocessor specifically for SpeechLM models

diagnose(data)

Only for debug

modality_specific_processing(value, modality)

special_token(token)