Explainable Artificial Intelligence for Facial Emotions in Deep Learning Environment

Abstract

Explainable Artificial Intelligence for Facial Emotions in Deep Learning Environment Abstract: Deep Learning models have shown excellent results in Autism Spectrum Disorder (ASD) diagnosis but they are quite complex and lack transparency. This study is intended to combine Deep learning (DL) and Explainable Artificial Intelligence (XAI) for ASD diagnosis using five pre-trained models (VGG16, InceptionV3, ResNet50, MobileNet, EfficientNet). We have utilized two publicly available datasets, Facial Expression Dataset and Facial Dataset Bangladesh, for training these models. Among these models ResNet50 has highest training accuracy of 99%, 100% and test accuracy of 86%, 96% for Facial Expression Dataset and Facial Dataset Bangladesh respectively. We have applied two Explainable Artificial Intelligence (XAI) techniques, Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanation (SHAP) on both the Datasets for identifying the facial regions that have the most impact while making predictions. Experimental findings show that the pixels highlighted by LIME and SHAP mostly contain the eye and the mouth region.