Closing the Context Gap: Activation Alignment for Tabular In-Context Learning

Excerpt

arXiv:2610.06679v1 Announce Type: cross Abstract: Tabular foundation models perform in-context learning (ICL) by conditioning predictions on labeled training examples provided as context. Unlike traditional models that separate training from inference, these models must process all training examples in every forward pass, making each prediction expensive. Restricting the number of training examples reduces this cost but substantially degrades performance. Instead of discarding context, we propos