Dalla Lana School of Public Health

Links

• For more information

Join us at the Statistical Sciences Applied Research and Education Seminar (ARES) with

Martha White
Associate Professor
Department of Computing Science,
Alberta Machine Intelligence Institute (Amii)
University of Alberta

Free Virtual Event | Registration Required

Talk Title
Reinforcement Learning in the Real World: Making Predictions Online for Water Treatment

Abstract
In this talk I will discuss how we used reinforcement-learning based prediction approaches for a real drinking-water treatment plant. I will first describe this dataset, and highlight challenges with seasonality, non- stationarity, partial observability, and heterogeneity across sensors and operation modes of the plant. I will then explain General Value
Function (GVF) predictions—discounted cumulative sums of observations–and highlight why they might be preferable to classical n-horizon predictions common in time series prediction. One important conclusion from this work is to demonstrate the importance of learning in deployment: an agent trained purely offline with no online updating performs
more poorly than an agent that learns online. I will conclude with some general learnings about using reinforcement learning for real systems.

Speaker Profile
Martha White is an Associate Professor of Computing Science at the University of Alberta. Martha is a PI of Amii–the Alberta Machine Intelligence Institute–which is one of the top machine learning centres in the world, and a director of RLAI–the Reinforcement Learning and Artificial Intelligence Lab at the University of Alberta. She holds a Canada CIFAR AI Chair and received IEEE’s AIs 10 to Watch in 2020. She has authored more than 65 papers in top journals and conferences. Martha has served as an area chair for top conferences in AI and ML, including ICML,
NeurIPS, ICLR, AAAI and IJCAI, as well as co-program chair for ICLR in 2020, and is an associate editor for JMLR and TMLR.