Attention & Representation

The Approximation Rank of Softmax Attention

Studies the approximation complexity of softmax attention through support geometry and softmax-visible interaction dimension.

2026 · preprint

Related publications

2026preprint

The Approximation Rank of Softmax Attention: Sharp Geometric Laws and Robust Interaction Dimension

Yuhe Sui, Jianing Zhang

arXiv · 2026