FourierQK: Filter Shape, Admissibility and the Leakage-Coverage Law
Frequency-collapse attention [Zeris, 2026e] achieves large gains over standard dot-product attention by replacing the Q/K dot product with a bandpass-filtered inner product at a learned frequency. A natural follow-up question is: which filter shape works best, and why?
arXiv cs.LG2 min read