Enhancing Human Action Recognition using Machine Learning Techniques

Abstract

Human Action Recognition (HAR) represents an understanding of actions performed by a human in a video. Action(s) modeling involves mapping action to a label that describes an instance of that action. Different agents (i.e., humans) can perform actions under varying speed, lighting conditions, and diverse viewpoints. Moreover, fully-automated HAR systems have several challenges such as background clutter, occlusion, scale, and appearance. Further, there has been increasing interest in the analysis of 3D data due to advancements in sensing technology. 3D data acquisition is made easier by costeffective devices. Thereby, in a scene, information about depth can be obtained from various objects and persons in the form of depth images. Moreover, 3D position of body joints is also available in the form of skeleton data. In this thesis, we address the recognition of actions from a sequence of depth maps and 3D skeleton data. To address issues and challenges of HAR such as background clutter and illumination invariance, in this thesis, we have proposed different techniques. In our proposed framework, we have used an improved learning algorithm named Selfadaptive Differential Evolution-Extreme Learning Machine (SaDE-ELM) for action classification. In the proposed approach, we have used Depth Motion Maps-Local Binary Pattern (DMM-LBP) for feature extraction and SaDE-ELM for action classification. Later, an approach is proposed to improve performance of action recognition for depth-based input with Single Layer Feed-forward Network (SLFN) using Self-adaptive Differential Evolution with knowledge-based control parameter-Extreme Learning Machine (SKPDE-ELM). Further, an approach is proposed for depth-based data using various feature combinations. For action classification, we have used Kernel-based Extreme Learning Machine (KELM) classifier. Moreover, to reduce misclassification, majority voting-based technique is applied. Recent advancements in computer vision field using neural networks have resulted in development of end-to-end systems, for applications, such as object recognition, image captioning, scene understanding, and action recognition. We explore an end-to-end system for action recognition using skeleton data and we improve the performance of state-of-the-art methods using Coefficient of Variation (CoV)-based vi weight initialization for LSTM, GRU, and CNN architectures. Later, we propose a method based on data augmentation using motion data for skeleton-based action recognition using ResNet architecture. Finally, we concluded the thesis with important discussions and future directions in the area of HAR.