Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models

Daffa Izzalhaqqi, Muhammad Yusuf and Wahyono, Wahyono (2023) Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models. In: 3rd International Workshop on Intelligent Systems, IWIS 2023, 9 - 11 August 2023, Ulsan.

[thumbnail of 4. Gesture_recognition_for_Indonesian_Sign_Language_B.pdf] Text
4. Gesture_recognition_for_Indonesian_Sign_Language_B.pdf - Published Version
Restricted to Registered users only

Download (1MB) | Request a copy

Abstract

Sign language comprises unique hand gestures used to communicate between the hearing and hearing-impaired communities, with variations based on regions or countries. Due to the increasing global prevalence of hearing loss, there is a growing need for efficient sign language recognition systems. This paper presents the development of a hybrid CNN-LSTM model specifically designed to recognize static alphabetical gestures in Indonesian Sign Language (BISINDO). Primary data was collected to address the issue of publicly available dataset. These self-collected images underwent augmentation and preprocessing stages prior to model training. The architecture and hyperparameter configuration of the hybrid model were fine-tuned using the Randomized Search CV method. Performance analysis was conducted on both the hybrid model, its constituent models, and state-of-the-art model. The experimental results revealed remarkable performance by the hybrid model with excellent accuracies of 99.60%, 84.87%, and 98.00% on the training, validation, and testing sets respectively, along with a macro average score of 0.98, indicating high precision, recall, and F1-score across all the classes. However, it was observed that the VGG-16 and CNN exhibited slightly superior performance compared to the hybrid model, while the LSTM demonstrated the least favorable performance among the models.

Item Type: Conference or Workshop Item (Paper)
Uncontrolled Keywords: BISINDO; CNN; Deep Learning; Hybrid Model; LSTM; Sign Language Recognition
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Mathematics and Natural Sciences > Computer Science & Electronics Department
Depositing User: Ismu WIDARTO
Date Deposited: 03 Sep 2024 02:46
Last Modified: 03 Sep 2024 02:46
URI: https://ir.lib.ugm.ac.id/id/eprint/6128

Actions (login required)

View Item
View Item