Isaac Gibbs (University of California, Berkley)
Title: Uncertainty quantification for black-box models with conditional guarantees
Abstract:
A central problem in the uncertainty quantification literature is designing methods that are both distribution-free and individualized to the test sample at hand. Prior work has shown that it is impossible to achieve finite-sample conditional validity without modelling assumptions. Thus, canonical methods in, e.g., the conformal inference literature, typically only issue marginal guarantees over a random draw of the test covariates. In this talk, I will outline a framework that bridges this gap by recasting the conditional objective as a set of robustness criteria over a class of covariate shifts. By relaxing the target class of covariate shifts, I will define a spectrum of problems that range between marginal and exact conditional validity and give methods that provide precise guarantees in between these extremes. This framework has broad applications and I will show how it can be used to construct prediction sets around the outputs of black-box regression models and filter out false information from the responses of large language models. This talk is based on joint work with John Cherian and Emmanuel Cand猫s.
馃敆 Zoom:
