Return to Colloquia & Seminar listing
Uncertainty Quantification Beyond Calibration
Joint Math/CS Theory| Speaker: | Charlotte Peale, Stanford |
| Location: | 1131 Kemper |
| Start time: | Mon, Oct 12 2026, 4:10PM |
Description
Calibration has emerged as a standard formal approach to uncertainty quantification, but the precise guarantees it provides are not always clear. This talk explores three recent works that establish more rigorous connections between calibration and the concrete goals of uncertainty quantification in modern machine learning.
First, we establish a tight connection between calibration and error diagnosis, the ability of a model to accurately assess its own limitations. We characterize the precise conditions under which a model's uncertainty estimates can compete with externally-trained error predictors, revealing when self-assessment is and isn't possible.
Second, we address uncertainty decomposition into epistemic (model-based) and aleatoric (data-based) components. This distinction is vital for understanding prediction errors in complex, subjective tasks, and for determining whether collecting more data will actually improve performance. However, standard calibration is insufficient to provide rigorous decomposed uncertainty estimates. We introduce higher-order calibration, a strengthened notion that provides theoretical guarantees for uncertainty decomposition, and discuss its application to model routing.
