Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures

Excerpt

arXiv:2610.07754v1 Announce Type: new Abstract: Adversarial training is one of the most reliable defenses against adversarial attacks, but its high computational cost must generally be paid anew for each task. Robust foundation models offer a promising alternative: adversarially pretrain a model once and then transfer its robustness to downstream tasks through lightweight adaptation. However, a fundamental question remains open: can robustness acquired during pretraining transfer to unseen tasks