Caltech Logo

Control Meets Learning Seminar

Wednesday, March 10, 2021
9:00am to 10:00am
Add to Cal
Online Event
Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent State
Benjamin Van Roy, Professor, Department of Electrical Engineering, and of Management Science and Engineering, Stanford University,

I will describe a reinforcement learning agent that, with specification only of agent state dynamics and a reward function, can operate with some degree of competence in any environment. The agent applies an optimistic version of Q-learning to update value predictions that are based on the agent's actions and aleatoric states. We establish a regret bound demonstrating convergence to near-optimal per-period performance, where the time required is polynomial in the number of actions and aleatoric states, as well as the reward averaging time of the best policy among those for which actions depend on history only through aleatoric state. Notably, there is no further dependence on the number of environment states or averaging times associated with other policies or statistics of history.

For more information, please contact Jolene Brink by email at [email protected] or visit Control Meets Learning Website.