Pose Correctional Captioning

Abstract

Our study addresses the issue of offering real-time assistance for improving exercise form by establishing an advanced architecture that comprehends visual relationships between source and target images. Our method incorporates employing MaxVit Vision Transformers, layered cross-attention layers, and an encoder-transformer decoder fusion to construct intelligent position-correcting captions. We conducted a comprehensive literature review to inform our work, focusing on evaluating MaxVit Vision Transformers for precise pose correctional captions, assessing the impact of stacked cross-attention layers, and quantifying the advantages of transformer decoders and pretrained GPT over traditional models. Rigorous testing against numerous image datasets verifies the proposed architecture’s usefulness in providing users with descriptive information to transition seamlessly between different workout poses. This research builds upon prior breakthroughs, as we have also done rigorous assessments on metrics such as Bleu-4, Rough-L, Meteor, and Cidar, attaining considerably better results surpassing previous standards with a Bleu-4 of 16.2, Rough-L score of 44.06, Meteor score of 36.80, and Cidar score of 11.39. Our research gives a novel alternative to the absence of real-time workout form recommendations. The recommended solution demonstrates promising results, giving a practical way to increasing training routines by eliminating mistakes and reducing the possibility of accidents using intelligent posture corrections captions.