Generalization through Lexical Abstraction in Transformer Models: The Case of Functional Words

Excerpt

arXiv:2609.19887v1 Announce Type: new Abstract: Pronouns, adverbs and other functional words (such as they, her, somewhere, there) are often used in language to replace concrete nouns or phrases, when their properties - such as gender, grammatical number - provide sufficient information for the given context. Do pretrained transformer models encode such functional words in a manner that allows them to be used like humans do? Can language models recognize the syntactic and semantic parallelism of