Reinforcement Learning based Control Systems and its Stability Analysis

Abstract

In present paper Reinforcement learning (RL) based control is implemented with Actor-critic based technique. Generation of reinforcement signal is described earlier with binary sign function for Actor-Critic based Reinforcement Learning. In the present paper two more discriminate functions introduced to generate reinforcement signal. It is proved with the results that these functions have faster convergence then earlier method to obtain reinforcement signal. In this paper, 2-link robot manipulator system’s dynamics is used to simulate results. It is assumed that the system has a certain ‘canonical’ structure. Present paper also shows reinforcement learning based controller achieves faster convergence then conventional controller based on loop gains. Convergence proof of proposed algorithm is defined with Lyapunov stability theory. Also we have proposed generalized Lyapunov function for any discriminant functions and system expressed in canonical form.