Technical - Partner Event
AI Seminar – Yi Wan
When
Feb. 25, 2022 - Feb. 25, 2022
12:00 PM – 1:00 PM MST
Where
Online
Online
Presenter: Yi Wan
Title: Towards Adaptive Model-Based Reinforcement Learning
Abstract: In recent years, a growing number of deep model-based reinforcement learning (RL) methods have been introduced. The interest in deep model-based RL is not surprising, given their many potential benefits, such as higher sample efficiency and the potential for fast adaption to changes in the environment. However, we demonstrate, using an improved version of the recently introduced Local Change Adaptation (LoCA) setup, that the well-known model-based methods such as PlaNet and DreamerV2 perform poorly in their ability to adapt to local environmental changes. Combined with prior work that made a similar observation about the other popular model-based method, MuZero, a trend appears to emerge suggesting that current deep model-based methods have serious limitations. We dive deeper into the causes of this poor performance, by identifying elements that hurt adaptive behavior and linking these to underlying techniques frequently used in deep model-based RL. We empirically validate these insights in the case of linear function approximation by demonstrating that a modified version of linear Dyna achieves effective adaptation to local changes. Furthermore, we provide detailed insights into the challenges of building an adaptive non-linear model-based method, by experimenting with a non-linear version of Dyna.
Bio: Yi Wan is a fifth-year Ph.D. candidate in Computing Science at the University of Alberta, focusing on reinforcement learning, which he believes is the most promising way to artificial general intelligence. His Ph.D. supervisor is Professor Rich Sutton. His long-term research goal is to build simple, general, and scalable learning and planning algorithms for reinforcement learning problems. He is particularly interested in designing these algorithms 1) for the average reward problem setting, 2) with function approximation, and 3) with temporal abstractions. Previously, he earned a Bachelor degree in Electrical and Computer Engineering (ECE) from Shanghai Jiao Tong University (SJTU), where he worked in SJTU Speech Lab, advised by Professor Kai Yu. After that, he obtained a master degree, also in ECE, from University of Michigan. During his master, he worked in Intelligent Robotics Lab, advised by Professor Ben Kuipers. During his master and Ph.D., he interned as a researcher in Mila, Huawei, and Tusimple and also interned as an engineer in Yitu.
The University of Alberta Artificial Intelligence (AI) Seminar is a weekly meeting where researchers (including students, developers, and professors) interested in AI can share their current research. Presenters include local speakers from the University of Alberta and industry as well as other institutions. The seminars discuss a wide range of topics related in any way to Artificial Intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems are of interest. Learn more at the AI Seminar website and by subscribing to the mailing list!
Looking to build AI capacity? Need a speaker at your event?
