Daffa Izzalhaqqi, Muhammad Yusuf and Wahyono, Wahyono (2023) Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models. In: 3rd International Workshop on Intelligent Systems, IWIS 2023, 9 - 11 August 2023, Ulsan.
4. Gesture_recognition_for_Indonesian_Sign_Language_B.pdf - Published Version
Restricted to Registered users only
Download (1MB) | Request a copy
Abstract
Sign language comprises unique hand gestures used to communicate between the hearing and hearing-impaired communities, with variations based on regions or countries. Due to the increasing global prevalence of hearing loss, there is a growing need for efficient sign language recognition systems. This paper presents the development of a hybrid CNN-LSTM model specifically designed to recognize static alphabetical gestures in Indonesian Sign Language (BISINDO). Primary data was collected to address the issue of publicly available dataset. These self-collected images underwent augmentation and preprocessing stages prior to model training. The architecture and hyperparameter configuration of the hybrid model were fine-tuned using the Randomized Search CV method. Performance analysis was conducted on both the hybrid model, its constituent models, and state-of-the-art model. The experimental results revealed remarkable performance by the hybrid model with excellent accuracies of 99.60%, 84.87%, and 98.00% on the training, validation, and testing sets respectively, along with a macro average score of 0.98, indicating high precision, recall, and F1-score across all the classes. However, it was observed that the VGG-16 and CNN exhibited slightly superior performance compared to the hybrid model, while the LSTM demonstrated the least favorable performance among the models.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Uncontrolled Keywords: | BISINDO; CNN; Deep Learning; Hybrid Model; LSTM; Sign Language Recognition |
Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
Divisions: | Faculty of Mathematics and Natural Sciences > Computer Science & Electronics Department |
Depositing User: | Ismu WIDARTO |
Date Deposited: | 03 Sep 2024 02:46 |
Last Modified: | 03 Sep 2024 02:46 |
URI: | https://ir.lib.ugm.ac.id/id/eprint/6128 |