Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

Excerpt

arXiv:2605.16048v2 Announce Type: replace Abstract: State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Language Models (LLMs). These three advantages are a direct consequence of the time recurrence inherent in the SSMs architecture. Here, we further improve this recurrent architecture by positively answering two previously underexplored, orthogonal questions: (1) Can we re