RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State

Excerpt

arXiv:2602.11958v2 Announce Type: replace Abstract: Linear attention offers an efficient alternative to full attention with a fixed-size recurrent state. However, this state is shared by all tokens, so information from distinct tokens becomes superposed within it and produces inter-token interference that degrades long-range fine-grained recall. To address this issue, we propose RAM-Net, which replaces dense access to a shared state with sparse address-based access. RAM-Net organizes the recurre