On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D

Excerpt

arXiv:2610.00820v1 Announce Type: new Abstract: General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires task-specific training or adaptation. We ask whether they can instead be generated directly from a few demonstrations. Using ARC-1D as a controlled testbed, we show that individual transformations can be represented by tiny specialist models, and that a hypernetwork can ge