espnet2.asr.encoder.beats_encoder.init_bert_params

source

espnet2.asr.encoder.beats_encoder.init_bert_params(module)

Initialize the weights specific to the BERT Model. This overrides the default initializations depending on the specified arguments.

1. If normal_init_linear_weights is set then weights of linear layer will be initialized using the normal distribution and bais will be set to the specified value.
2. If normal_init_embed_weights is set then weights of embedding layer will be initialized using the normal distribution.
3. If normal_init_proj_weights is set then weights of in_project_weight for MultiHeadAttention initialized using the normal distribution (to be validated).