skip to main content
Caltech

H.B. Keller Colloquium

Monday, February 3, 2020
4:00pm to 5:00pm
Add to Cal
Annenberg 105
Algorithms for Eliciting Machine Learning Metrics
Sanmi Koyejo, Assistant Professor, Department of Computer Science, University of Illinois at Urbana-Champaign,

Given a prediction problem with real-world tradeoffs, which cost function should the machine learning model be trained to optimize? Unfortunately, typical default metrics in machine learning, such as accuracy applied to binary classifiers, may not capture tradeoffs relevant to the problem at hand. This talk proposes metric elicitation as a formal strategy to address the metric selection problem, specifically by automatically discovering implicit preferences from an expert or an expert panel via interactive feedback. I will primarily focus on algorithms for eliciting classification metrics, showing that simple algorithms are efficient for metric elicitation under broad assumptions. Finally, I will briefly outline early work on metric selection for measuring group fairness in classification problems with sensitive groups.

For more information, please contact Diana Bohler by phone at 6263951768 or by email at [email protected].