Vision Based Robotic Arm Control System with Biometric Authentication

Abstract

The main objective of this effort is to explore the utility of a human hand for natural Human-Robot Interaction and Biometric Authentication. Hand gestures are a powerful communication channel and can be used as an interface device to achieve a natural and immersive Human-Robot Interaction or Human-Machine Interaction. Also various geometric features of hand can be used for the biometric authentication. Overall aim of this thesis is to develop a vision based system for controlling a 5 degree of freedom robotic arm with the help of static hand gestures and implement the hand geometry based biometric authentication so that only the authorized user can access the system. In the proposed method hand geometry and hand gesture images are acquired using HD web-camera and then passed through three stages: Image preprocessing, feature extraction, and recognition. In preprocessing stage some of the image processing operations are applied to extract the hand region from its background and prepare the image for the feature extraction stage. For hand geometry recognition various geometric features of the hand like length and width of the fingers, Palm width and distances between the valley points of fingers are extracted and utilize for the user authentication. For hand gesture recognition, Chain Code of the hand contour is extracted and use to construct the feature vector, and finally a supervised back-propagation multilayered feed-forward neural network is used for the classification of hand gestures. Once a hand gesture is recognized, an appropriate command is sent to a robotic arm to perform a pre-defined task. The proposed system has been extensively tested with success. The average performance of the system to recognize hand gestures is more than 98% and the robotic arm is able to do a jobs by using hand gesture commands as its input.