[thw20]

Good work! This is very interesting. Here's a related work that construct low-rank approximation for attention: https://arxiv.org/abs/2505.12942.

Maybe the idea of Query calibration matrix Rxx is of interest to the author!

[jchandra]

Thanks, really appreciate the pointer. Will dig into it.