espnet2.spk.loss.aamsoftmax.sine_from_cosine

source

espnet2.spk.loss.aamsoftmax.sine_from_cosine(cosine: Tensor) → Tensor

Compute sqrt(1 - cos^2) with finite gradients at

|cos|

= 1.

sqrt has an infinite derivative at 0, so the plain expression gives NaN gradients when an embedding is exactly aligned with a class weight (

|cos|

== 1), which bf16 rounding makes common near convergence; values of

|cos|

slightly above 1 from rounding would also give NaN in the forward. Entries with 1 - cos^2 <= 0 are set to 0 and their sqrt argument is replaced by 1, so the masked branch has a finite (zero) gradient. Forward values equal sqrt((1 - cos^2).clamp(0, 1)) for all inputs.

• Parameters:cosine – Cosine similarities of any shape.
• Returns: Sine values of the same shape as cosine.