The Approximation Rank of Softmax Attention: Sharp Geometric Laws and Robust Interaction Dimension
NeurIPS 2026 NeurReps Extended Abstracts · 2026
Yuhe Sui
I study the mathematical structure of learning and decision-making, and the design of reliable AI systems for research. About me →
Research interests
- Attention & Representation
- Mathematical structure, approximation and representation complexity, intrinsic geometry, and invariances of attention mechanisms and learned representations.
- Learning & Decision-Making
- Reinforcement learning, policy optimisation, in-context learning of algorithms, stochastic control, and structured decision procedures.
- Scientific AI Systems
- Reliable long-horizon AI systems for research, with emphasis on hierarchical execution, verification, provenance, recoverability, and human control.
- Mathematical & Quantitative Methods
- Probability, optimisation, discrete mathematics, statistical learning, and quantitative applications.
Flagship project
All projects →QRS research programme · Fine-tuning · Attention
GeoAFT
Geometric Attention Fine-Tuning
Adapting how representations relate to one another, not only the representations themselves.
A pretrained Transformer learns representations and a rule for comparing them. Fine-tuning usually changes the first. GeoAFT adapts the second.
Selected publications
All publications →NTU-QRS AlphaAllocator: Bounded Transformer News Overlays for Chinese Macro-Asset Allocation
NLPCC 2026 Shared Task 4 · Poster + Oral · 2026
Shared Task 4: Track 1 winner; Track 2 6th.
No matching results. Try another keyword or clear the filters.
Explore the archive
Projects
Research projects, public platforms, and independent tools.
Notes
Ideas, explanations, and working notes in mathematics and computing.
Beyond Research
Travel, food, music, art, reading, and writing outside my academic work.