Technical - Partner Event
AI Seminar – Shi Dong
When
April 8, 2022 - April 8, 2022
12:00 PM – 1:00 PM MST
Where
Online
Online
Presenters: Shi Dong, Stanford University
Title: Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent States
Abstract: We design a simple reinforcement learning (RL) agent that implements an optimistic version of Q-learning and establish through regret analysis that this agent can operate with some level of competence in any environment. While we leverage concepts from the literature on provably efficient RL, we consider a general agent-environment interface and provide a novel agent design and analysis. This level of generality positions our results to inform the design of future agents for operation in complex real environments. We establish that, as time progresses, our agent performs competitively relative to policies that require longer times to evaluate. The time it takes to approach asymptotic performance is polynomial in the complexity of the agent's state representation and the time required to evaluate the best policy that the agent can represent. Notably, there is no dependence on the complexity of the environment. The ultimate per-period performance loss of the agent is bounded by a constant multiple of a measure of distortion introduced by the agent's state representation. This work is the first to establish that an algorithm approaches this asymptotic condition within a tractable time frame.
Bio:
Shi Dong is currently a Ph.D. candidate in the Department of Electrical Engineering at Stanford University, where he is advised by Prof. Benjamin Van Roy. He is interested in using theoretical tools to understand the essential elements in practical reinforcement learning (RL) agent design, and to help bring the benefits of RL to real life. He received his bachelor’s degree from Tsinghua University, and master’s degree from the Department of Statistics at Stanford University. His industrial experiences include assisting ByteDance in improving their news and video recommendation system, and research internships at Google, DeepMind, and Microsoft. One of his works was awarded winner in the 2021 INFORMS George Nicholson Student Paper Competition.
The University of Alberta Artificial Intelligence (AI) Seminar is a weekly meeting where researchers (including students, developers, and professors) interested in AI can share their current research. Presenters include local speakers from the University of Alberta and industry as well as other institutions. The seminars discuss a wide range of topics related in any way to Artificial Intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems are of interest. Learn more at the AI Seminar website and by subscribing to the mailing list!
Looking to build AI capacity? Need a speaker at your event?
