ASRN Adaptive Sparse Recurrence Network [N]

Excerpt

A copy layer for language models that finds earlier occurrences of the current context with learned hash tables and copies what came next — with memory linear in sequence length. submitted by /u/Mean-Disaster8380 [link] [comments]