AUGMENTATIVE AND ALTERNATIVE COMMUNICATION METHOD BASED ON TONGUE CLICKING FOR MUTE DISABILITIES

NIK NUR WAHIDAH NIK HASHIM; MUHAMMAD AMIRUL AMIN AZMI; HAZLINA MD. YUSOF

doi:10.31436/iiumej.v20i1.1021

Authors

NIK NUR WAHIDAH NIK HASHIM INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
MUHAMMAD AMIRUL AMIN AZMI
HAZLINA MD. YUSOF https://orcid.org/0000-0002-6349-2028

DOI:

https://doi.org/10.31436/iiumej.v20i1.1021

Keywords:

Mute, Speech, Alternative Communication, Classification, Features

Abstract

This paper presents a pilot study for a novel application of converting tongue clicking sound to words for people with the inability to speak. 15 features of speech that are related to speech timing patterns, amplitude modulation, zero crossing and peak detection were extracted. The experiments were conducted with three different patterns using binary Support Vector Machine (SVM) classification with 10 recordings as training data and 10 recordings as development data. Peak size outperformed all features with 85% classification rate for pattern P1-P3 whereas multiple features produced 100% classification rate for P1-P2 and P2-P3. A GUI based system was developed to validate the trained classifier. Multiclass SVM were constructed based on the best features obtained from binary SVM classification outcome, namely peak size and skewness amplitude modulation, and then tested on 15 recordings. The GUI based multiclass SVM obtained a satisfying performance of 67% correct classification of the test data set.

ABSTRAK: Kertas ini membentangkan panduan kajian kepada aplikasi terkini dalam menukar bunyi klik pada lidah kepada perkataan untuk orang yang mempunyai kehilangan upaya dalam bertutur. 15 ciri khas berkaitan pertuturan adalah pola masa, modulasi nilai tertinggi, tiada titik persilangan dan nilai terpilih yang dikesan. Eksperimen telah dijalankan dengan tiga corak berlainan menggunakan perduaan Mesin Vektor Sokongan (SVM) klasifikasi dengan 10 rakaman sebagai data terlatih dan 10 rakaman sebagai data yang dibina. Saiz tertinggi yang melebihi semua ciri-ciri pada 85% kadar klasifikasi dilihat pada corak P1-P3, sedangkan ciri-ciri pelbagai telah terhasil pada 100% kadar klasifikasi P1-P2 dan P2-P3. Sistem berdasarkan GUI telah dibina bagi menilai ciri terlatih. Kelas pelbagai SVM telah dibina berdasarkan ciri-ciri terbaik dan dihasilkan daripada klasifikasi perduaan SVM, iaitu saiz tertinggi dan modulasi saiz tertinggi tidak linear, dan telah diuji dengan 15 rakaman. Kelas pelbagai SVM yang didapati melalui GUI ini adalah memberangsangkan iaitu 67% klasifikasi adalah tepat pada set data yang diuji.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

World Federation of Deaf, Retrieved December 10, 2018, from https://wfdeaf.org/

Statistik Pendaftaran OKU. Retrieved December 10, 2018, from http://www.data.gov.my/data/ms_MY/dataset/statistik-pendaftaran-oku

Dilbagi A. (2014). Talk breathes new life into the alternative communication device market. Available at: http://newatlas.com/talk-morse-code-speech/33042/.

Kerr D, Bouazza-Marouf K, Gaur A, Sutton A, Green R. (2016). A breath controlled AAC system. Journal of Communication Matters, 30(3): 11-13.

Mukherjee K, Chatterjee D. (2015). Augmentative and Alternative Communication Device Based on Eye-Blink Detection and Conversion to Morse-Code to Aid Paralyzed Individuals, Int. Conf. On Communication, Information and Computing Technology (ICCICT), pp. 1-5. doi: 10.1109/ICCICT.2015.7045754 DOI: https://doi.org/10.1109/ICCICT.2015.7045754

Nik Nur Wahidah NH, Wilkes M, Salomon R, Meggs J, and France DJ. (2016). Evaluation of Voice Acoustics as Predictors of Clinical Depression Scores, Journal of Voice, 31(2): 256.e1-256.e6.

Nagori NP, Malode V. (2016). Communication Interface for Deaf-Mute People using Microsoft Kinect. Int. Conf. Autom. Control Dyn. Optim. Tech., pp. 640-644. DOI: https://doi.org/10.1109/ICACDOT.2016.7877664

Ozdas A, Shiavi RG, Silverman SE, Silverman MK, Wilkes DM. (2004). Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk. IEEE Trans. Biomed. Eng., 51(9): 1530-1540.

Vijayalakshmi P, Aarthi M. (2016). Sign language to speech conversion. Fifth Int. Conf. Recent Trends Inf. Technol., pp. 1-6. DOI: https://doi.org/10.1109/ICRTIT.2016.7569545

Shorrock TH, MacKay JC, Ball CJ. (2004). Efficient Communication by Breathing, Proceedings of the First International Conference on Deterministic and Statistical Methods in Machine Learning, pp. 88-97. doi: 10.1007/11559887_5 DOI: https://doi.org/10.1007/11559887_5