Enhancing Sign Language Classification through Reservoir Computing and Depth Camera Technology

The depth camera has emerged as an efficacious tool for facilitating interactions between individuals and machines. Its advantageous features, including a lightweight design, high durability, completeness, superior image quality, and versatility, have prompted researchers to explore its integration into advanced applications. Currently, the prevailing methods for classifying sign language exhibit certain limitations, such as suboptimal accuracy and inefficiency in gesture recognition technologies. In this study, we conduct a comprehensive examination of hand language characteristics rooted in American Sign Language (ASL), utilizing gesture-based interactions and augmenting sensing performance through preprocessing and the application of reservoir computing in the training model. We conduct a comprehensive evaluation of the gesture characteristics captured by the depth camera, elucidating a methodology encompassing preprocessing, machine learning, and the technical prediction of salient gesture features derived from conventional depth camera video footage. This endeavor aims to establish a robust signal model for enhanced understanding and representation of the analyzed gestures. Our machine-learning models adeptly predict American Sign Language signs. Correlation values derived from the depth camera data demonstrate a pronounced alignment with naturally occurring variations in metrics observed within specific gestures. Our methodologies for quantifying gestures with depth cameras contribute to heightened accessibility in quantitative motion analysis. The outcomes of the reservoir computing implementation exhibit the successful classification of three hand language signs, thereby manifesting high precision and recall. The robust performance of the classification is pivotal for practical applications and underscores the efficacy of our approach. © 2024 IEEE.