Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model

Excerpt

arXiv:2610.01006v1 Announce Type: cross Abstract: Decision models return probabilities intended for routing, abstention and automated action. Calibration makes those probabilities useful on average, but does not establish whether low confidence reflects chance or missing knowledge, nor whether confidence falls when a model moves beyond what it knows. We audit this distinction in Jev, a decision model, with over 15 public datasets and 6 generated task families, with paired interventions that vary