espnet2.asr_transducer.decoder.modules.mega.positional_bias.RotaryRelativePositionBias

source

class espnet2.asr_transducer.decoder.modules.mega.positional_bias.RotaryRelativePositionBias(size: int, max_positions: int = 2048)

Bases: Module

RotaryRelativePositionBias module definition.

• Parameters:
  • size – Module embedding size.
  • max_positions – Maximum number of relative positions.

Construct a RotaryRelativePositionBias object.

forward(length: int) → Tensor

Compute rotary relative position bias.

• Parameters:length – Sequence length.
• Returns: Rotary relative position bias. (L, L)
• Return type: bias

static get_sinusoid_embeddings(max_positions: int, size: int) → Tuple[Tensor, Tensor]

Compute sinusoidal positional embeddings.

• Parameters:
  • max_positions – Maximum number of positions.
  • size – Input size.
• Returns: Sine elements. (max_positions, size // 2) : Cos elements. (max_positions, size // 2)

reset_parameters(val: float = 0.0, std: float = 0.02) → None

Reset module parameters.

• Parameters:
  • val – Initialization value.
  • std – Standard deviation.

rotary(x: Tensor) → Tensor

Compute rotary positional embeddings.

• Parameters:x – Input sequence. (L, size)
• Returns: Rotary positional embeddings. (L, size)
• Return type: x