Optimal Control with Learning on The Fly: System with Unknown Drift

Speaker

Debdipta Goswami

Affiliation

Assistant Professor
Department of Mechanical and Aerospace Engineering
The Ohio State University

Abstract

Most techniques developed during the past century for estimation and control of dynamical systems require precise knowledge of an underlying mathematical model. But recently, there has been renewed interest in controlling systems for which a reliable model is unknown or the model parameters can vary. Here an optimal control strategy is developed for a simple problem where the drift parameter is unknown and must be learned on the fly while controlling the system. This scenario requires a control strategy from the initial time and cannot accommodate distinct phases of parameter learning and control update. The cost function is posed over a finite time interval, and the state is subject to random disturbances and measured by a noisy sensor. Several different versions of the problem are studied. In the Bayesian version, a prior distribution on the unknown drift parameter is assumed. In the ‘‘agnostic’’ version, no such assumption is made and the performance is compared against an adversary with exact knowledge of the parameter. This gives rise to several notions of performance penalty or ‘‘regret’’. The optimal strategy is chosen to minimize the ‘‘worst-case regret’’ arising from the most unfavorable value of the parameter for that strategy. In every case, the optimal strategy turns out to be a Bayesian strategy or a limit of Bayesian strategies. This is a joint work with Clarence W. Rowley and Charles L. Fefferman (Princeton University).

Bio

Debdipta Goswami joined the Department of Mechanical and Aerospace Engineering, the Ohio State University, in 2022 as an assistant professor. He received his Ph.D. degree in Electrical and Computer Engineering from the University of Maryland in 2020 under the supervision of Prof. Derek A. Paley. Between 2020 and 2022, he has worked as a postdoctoral research associate at the Department of Mechanical and Aerospace Engineering in Princeton University where he worked with Prof. Clancy Rowley. His research interests lie at the intersection of control systems and machine learning with a focus on motion planning and agile control of aerial robots. He has worked on data-driven discovery and control of dynamical systems using operator-theoretic methods and reservoir computers. His current research focuses on the structured learning of control systems from data with guaranteed performance and simultaneous learning and control of dynamical systems. He also works on model predictive control and motion planning for unmanned aerial vehicles.