espnet2.speechlm.model.speechlm.speechlm_job.SpeechLMJobTemplate
Less than 1 minute
espnet2.speechlm.model.speechlm.speechlm_job.SpeechLMJobTemplate
class espnet2.speechlm.model.speechlm.speechlm_job.SpeechLMJobTemplate(config: Dict[str, Any], is_train: bool = False, load_pretrained: bool = True)
Bases: AbsJobTemplate
Job template for SpeechLM training tasks.
This class implements the specific model and data processing configurations for speech language modeling tasks.
Initialize the SpeechLM job template.
- Parameters:
- config – Dictionary containing job configuration parameters.
- is_train – Whether the job is for training (True) or inference.
- load_pretrained – Load the pretrained weights of the frozen audio encoder and of the base LM. False builds both from their configs alone, leaving the LM weights uninitialised, for a caller that loads a checkpoint strictly over the whole model right after - as both inference entry points do. Nothing else may pass False.
build_model(parallel_dims=None) → Module
Build the SpeechLM model.
When parallel_dims is provided and PP is enabled, automatically selects the PP model variant (model_choice + "_pp") and reads pp_layout from self.config["trainer"]["titan_config"].
- Parameters:parallel_dims – TorchTitan ParallelDims. If provided and
pp_enabled, the PP model variant is used. - Returns: A SpeechLM model instance (full or PP-stage).
build_preprocessor() → Callable
Build the data collation function for SpeechLM.
- Returns: A callable function for collating SpeechLM batch data.
