Activation-Based Active Learning for In-Context Learning: Challenges and Insights

Excerpt

arXiv:2606.05134v2 Announce Type: replace Abstract: Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations. In this paper, we test the hypothesis that model activations could provide a fine-grained signal to optimise the selection of in-context examples. We present a comprehensive analysis of MLP activation-based deep active learning methods applied to in-context learning