I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]
ALHR - Adaptive Learnable Hierarchical Routing, uses static binary trees and learnable functions to minimize the amount of keys to be read.
Reddit - r/ML1 min read