Information Satisfaction: A Reader-Centered Axis for Summarization Evaluation

Excerpt

arXiv:2608.14457v1 Announce Type: new Abstract: The majority of work on summarization evaluation focuses on general summary quality (e.g., ROUGE, BERTScore) or specific desired properties (e.g., readability, factuality). However, these metrics fail to measure the utility of a summary to an individual user. For example, a biomedical researcher learning about the latest vaccine research will have different informational needs from a family doctor. Query-focused summarization captures part of this