Rotary Positional Embeddings (RoPE)
A positional encoding method that applies a rotation matrix to Query and Key vectors in self-attention layers.
RoPE implements relative positional encoding by rotating Query and Key vectors in the complex plane. This preserves positional information over long context windows, allowing models to scale to larger input sequences.
Historical figures and technical concepts for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Official Documentation.