“Mind the Gap: From Predictions to ML-Informed Decisions”
Machine learning (ML) is increasingly being used to support decision-making in many organizational settings. However, there is currently a gap between the design and evaluation of ML algorithms and the functional role of these algorithms as tools for decision support. The first part of the talk will highlight the role of humans-in-the-loop, and the importance of evaluating decisions instead of predictions, through a study of the adoption of a risk assessment tool in child maltreatment hotline screenings. The second part of the talk will focus on the gap between the construct of interest and the proxy that the algorithm optimizes for. Using a proposed machine learning methodology that extracts knowledge from experts’ historical decisions, De-Arteaga shows that in the context of child maltreatment hotline screenings (1) there are high-risk cases whose risk is considered by the experts but not wholly captured in the target labels used to train a deployed model, and (2) we can bridge this gap if we purposefully design with this goal in mind.
About the Speaker
Maria De-Arteaga is an assistant professor at the Information, Risk and Operation Management (IROM) Department at the University of Texas at Austin, where she is also a core faculty member in the Machine Learning Laboratory and an affiliated faculty of Good Systems. She holds a joint Ph.D. in Machine Learning and Public Policy and a M.Sc. in Machine Learning, both from Carnegie Mellon University, and a. B.Sc. in Mathematics from Universidad Nacional de Colombia. Her research focuses on the risks and opportunities of using machine learning to support experts’ decisions in high-stakes settings, with a particular interest in algorithmic fairness and human-AI collaboration. Her work has been featured by UN Women and Global Pulse and has received best paper awards at NAACL’19 and Data for Policy’16, and research awards from Google and Microsoft Research.
About the AI for Social Impact Seminar Series
The AI for Social Impact Seminar Series brings together researchers and thought leaders from a variety of fields to explore the diverse applications of artificial intelligence for a societal benefit. Through the series, the Center for Socially Responsible Artificial Intelligence aims to inspire new ideas and collaborations and to identify novel approaches that can advance discovery in the field at Penn State and beyond.