Policy Gradient Method over the Input-Output History Model

Speaker

Tomonori Sadamoto

Affiliation

Assistant Professor
Department of Mechanical Engineering and Intelligent Systems
The University of Electro-Communications, Japan

Abstract

In this talk, we will discuss a policy gradient method (PGM) over the so-called input-output history (IOH) representation and its application to the linear quadratic Gaussian (LQG) dynamic output feedback control synthesis. First, we establish the equivalence between the dynamic output feedback and the static partial state feedback under a new system representation characterized by the finite-length IOH. Using this equivalence, we search for the optimal dynamic output feedback controller via the search for the optimal partial state feedback gain. Due to the sparsity constraint on the feedback gain matrix, the latter problem belongs to the class of static output feedback design problems, which by itself is a well-recognized challenging problem. Nevertheless, by exploring a low-dimensional representation of the closed-loop system, we show that the cost function is smooth and exhibits a gradient dominance property under a few mild conditions, ensuring linear convergence of the PGM to the global optimum. This is a joint work with Dr. Tomonori Sadamoto.

Bio

Tomonori Sadamoto received his Ph.D. degree from the Tokyo Institute of Technology, Tokyo, Japan, in 2015. From 2015 to 2016, he was a visiting researcher at the School of Electrical Engineering, Royal Institute of Technology (KTH), Stockholm, Sweden. From 2016 to November 2018, he was a specially appointed assistant professor in the Department of Systems and Control Engineering, Graduate School of Engineering, Tokyo Institute of Technology. From November 2018 to October 2023, he served as an assistant professor in the Department of Mechanical and Intelligent Systems Engineering, University of Electro-Communications. Since November 2023, he has been an associate professor in the same department. In 2014, he was named a finalist for the Best Student Paper Award at the 13th European Control Conference. In 2020, he received the IEEE Control Systems Magazine Outstanding Paper Award.