PROJECT: Constrained Stochastic Linear-Quadratic Control: A Direct-Comparison Based Approach

Constrained Stochastic Linear-Quadratic Control: A Direct-Comparison Based Approach.

This project is completed during my visiting in University of Illinois at Champaign-Urbana, IL, USA, cooperated with Weiping Wu and Ruo-Bing Xue, from Shanghai Jiao Tong University, Shanghai, China.

We study the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints in infinite horizon, considering both additive and multiplicative noises. Stochastic control systems can be formulated as Markov decision problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance potentials for the LQ problem by utilizing the state separation property of the system structure. Based on it, we successfully derive the optimality condition and the stationary optimal feedback control. By introducing bias optimization, we establish a general framework for studying infinite horizon control problems with both average and total rewards. In addition, we verify that the proposed optimal control solves both of these problems. Our work provides a new perspective in linear-quadratic control problems; based on this approach, learning based algorithms can be developed without identifying all the system structure parameters.