Attention & Representation

Representation-Relative Attention Geometry

Studies which named geometric descriptions of attention are intrinsic and which depend on admissible changes of query/key representation.

2026 · ongoing

Project lead

GeoAttention

QRS project

Overview

Attention geometry, representation-relative identifiability, and geometry-adaptive prediction.

My role

I lead this project within QRS. Associated research outputs carry their own author lists.

Current scope

Active research, as recorded in the QRS project registry reviewed on 10 October 2026.

Public resources

QRS project

Source and status

This description is grounded in the QRSNTU project registry and its generated public project page, reviewed on 10 October 2026. Project status and paper status are separate; related publication records retain their own visibility and metadata.

Related projects

2026preprint

The Approximation Rank of Softmax Attention

Sharp Geometric Laws and Robust Interaction Dimension

Studies the approximation complexity of softmax attention through support geometry and softmax-visible interaction dimension.

Project lead

arXiv QRS project Research: The Approximation Rank of Softmax Attention: Sharp Geometric Laws and Robust Interaction Dimension

ongoing

GeoAFT

Research on post-training attention adaptation via geometry-adaptive fine-tuning.

Project lead

QRS project Research: Geometric Attention Fine-Tuning Is Representation-Relative: Autonomous ML Research with AI-Native Workflow

← All projects