[논문]A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors

Cheng, Juan; Chen, Xun; Liu, Aiping; Peng, Hu

doi:10.3390/s150923303

A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors 원문보기

Sensors, v.15 no.9, 2015년, pp.23303 - 23324

Cheng, Juan (Department of Biomedical Engineering, Hefei University of Technology, 193 Tunxi Road, Hefei 230009, China) , Chen, Xun (E-Mails: chengjuan@hfut.edu.cn (J.C.)) , Liu, Aiping (xun.chen@hfut.edu.cn (X.C.)) , Peng, Hu (hpeng@hfut.edu.cn (H.P.))

Abstract ▼ AI-Helper

Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing “Palm Downward” sign gestures from “Palm Inward” ones. Only the “Palm Inward” gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

주제어

참고문헌 (62)

1. Ong S.C.W. Ranganath S. Automatic sign language analysis: A survey and the future beyond lexical meaning IEEE Trans. Pattern Anal. Mach. Intell. 2005 27 873 891 10.1109/TPAMI.2005.112 15943420

상세보기
2. Kong W.W. Ranganath S. Towards subject independent continuous sign language recognition: A segment and merge approach Pattern Recognit. 2014 47 1294 1308 10.1016/j.patcog.2013.09.014

상세보기
3. Fagiani S.S.M. Principi E. Piazza F. Signer independent isolated Italian sign recognition based on hidden Markov models Pattern Anal. Appl. 2015 18 385 402 10.1007/s10044-014-0400-z

상세보기
4. Regina E. Chuan C. Guardino C. American sign language recognition using leap motion sensor Proceedings of the 13th International Conference on Machine Learning and Applications (ICMLA) Detroit, MI, USA 3–5 December 2014 541 544
5. Ding L. Martinez A.M. Modelling and recognition of the linguistic components in American sign language Image Vis. Comput. 2009 27 1826 1844 10.1016/j.imavis.2009.02.005 20161003

상세보기
6. Oz C. Leu M.C. American sign language word recognition with a sensory glove using artificial neural networks Eng. Appl. Artif. Intell. 2011 24 1204 1213 10.1016/j.engappai.2011.06.015

상세보기
7. Starner T. Pentland A. Real-time American sign language recognition from video using hidden Markov models Proceedings of the International Symposium on Computer Vision Coral Gables, FL, USA 19–21 November 1995 265 270
8. Weaver J. Starner T. Pentland A. Real-time American sign language recognition using desk and wearable computer based video IEEE Trans. Pattern Anal. Mach. Intell. 1998 20 1371 1375

상세보기
9. Vogler C. Metaxes D. A framework for recognizing the simultaneous aspects of American sign language Comput. Vis. Image Underst. 2001 81 358 384 10.1006/cviu.2000.0895

상세보기
10. Vogler C. Goldenstein S. Facial movement analysis in ASL Univers. Access Inf. Soc. 2008 6 363 374 10.1007/s10209-007-0096-6

상세보기
11. Deriche M. Mohandes M. Liu J. Image-based and sensor-based approaches to Arabic sign language recognition IEEE Trans. Hum. Mach. Syst. 2014 44 551 557
12. Fanaswala M. Amin F. Assaleh K. Shanableh T. Bajaj H. Continuous Arabic sign language recognition in user dependent mode J. Intell. Learn. Syst. Appl. 2010 2 19 27

상세보기
13. Assaleh K. Shanableh T. Al-Rousan M. Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language IEEE Trans. Syst. Man Cybern. Part B Cybern. 2007 37 641 650
14. Kosmidou V.E. Hadjileontiadis L.J. Intrinsic mode entropy: An enhanced classification means for automated Greek sign language gesture recognition Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society Vancouver, BC, USA 20–25 August 2008 5057 5060
15. Kosmidou V.E. Hadjileontiadis L.J. Sign language recognition using intrinsic-mode sample entropy on sEMG and accelerometer data IEEE Trans. Biomed. Eng. 2009 56 2879 2890 10.1109/TBME.2009.2013200 19174329

상세보기
16. Petrantonakis P.C. Kosmidou V.E. Hadjileontiadis L.J. Enhanced sign language recognition using weighted intrinsic-mode entropy and signer’s level of deafness IEEE Trans. Syst. Man Cybern. Part B Cybern. 2011 41 1531 1543
17. Kim J.S. Jang W. Bien Z. A dynamic gesture recognition system for the Korean sign language (KSL) IEEE Trans. Syst. Man Cybern. Part B Cybern. 1996 26 354 359
18. Park G. Jang W. Kim J. Lee C. Bien Z. Kim S. Real-time recognition system of Korean sign language based on elementary components Proceedings of the Sixth IEEE International Conference on Fuzzy Systems Barcelona, Spain 1–5 July 1997 Volume 3 1463 1468
19. Bang W. Kim J. Park K. Bien Z.Z. Continuous gesture recognition system for Korean sign language based on fuzzy logic and hidden Markov model Proceedings of the 2002 IEEE International Conference on Fuzzy Systems Honolulu, HI, USA 12–17 May 2002
20. Wang C. Gao W. Shan S. An approach based on phonemes to large vocabulary Chinese sign language recognition Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition Washington, DC, USA 21–21 May 2002
21. Zhang X. Wang K. Li Y. Chen X. Wang Z.J. A sign-component-based framework for Chinese sign language recognition using accelerometer and sEMG data IEEE Trans. Biomed. Eng. 2012 59 2695 2704 22438511

상세보기
22. Wang C. Kong D. Wang R. Yin B. Similarity assessment model for Chinese sign language videos IEEE Trans. Multimed. 2014 16 751 761 10.1109/TMM.2014.2298382

상세보기
23. China Disabled Persons’ Federation The Number of Persons with Disabilities in the End of 2010 Available online: http://www.cdpf.org.cn/sjzx/cjrgk/201206/t20120626_387581.shtml (accessed on 4 May 2015)
24. China Association of the Deaf Chinese Sign Language (Revised Edition) Hua Xia Press Beijing, China 2003 (In Chinese)
25. Katsamanis A. Theodorakis S. Maragos P. Product-HMMs for automatic sign language recognition Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing Taipei, Taiwan 19–24 April 2009 1601 1604
26. Fang G. Gao W. A SRN/HMM system for signer-independent continuous sign language recognition Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition Washington, DC, USA 21–21 May 2002 312 317
27. Zhang T. Sun C. Xu C. Latent support vector machine modeling for sign language recognition with Kinect Acm Trans. Intell. Syst. Technol. 2015 6 10.1145/2629481

상세보기
28. Tang A. Huang J. Lu K. Wang Y.F. Li H. A real-time hand posture recognition system using deep neural networks Acm Trans. Intell. Syst. Technol. 2015 6 10.1145/2735952

상세보기
29. China Disabled Persons’ Federation Graphic Book of Phonology- and Radical-Coded Getures Available online: http://club.china.com/data/thread/1011/2717/35/28/5_1.html (accessed on 5 April 2015)
30. Silanon K. Suvonvorn N. Finger-spelling recognition system using fuzzy finger shape and hand appearance features Proceedings of the Fourth International Conference on Digital Information and Communication Technology and It’s Applications (DICTAP) Bangkok, Thailand 6–8 May 2014 419 424
31. Dahmani D. Larabi S. User-independent system for sign language finger spelling recognition J. Vis. Commun. Image Represent. 2014 25 1240 1250 10.1016/j.jvcir.2013.12.019

상세보기
32. Asselin P.K. Allen J.M. Foulds R. American sign language finger spelling recognition system Proceedings of the IEEE 29th Annual Bioengineering Conference Newark, NJ, USA 22–23 March 2003 285 286
33. Grobel K. Assan M. Isolated sign language recognition using hidden Markov models Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Computational Cybernetics and Simulation Orlando, FL, USA 12–15 October 1997 Volume 1 162 167
34. Shanableh T. Tubaiz N. Assaleh K. Glove-based continuous Arabic sign language recognition in user-dependent mode IEEE Trans. Hum. Mach. Syst. 2015 45 526 533
35. Fang G. Gao W. Zhao D. Large vocabulary sign language recognition based on fuzzy decision trees IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2004 34 305 314 10.1109/TSMCA.2004.824852

상세보기
36. Sarkar S. Yang R. Loeding B. Handling movement epenthesis and hand segmentation ambiguities in continuous sign language recognition using nested dynamic programming IEEE Trans. Pattern Anal. Mach. Intell. 2010 32 462 477 20075472

상세보기
37. Fang G. Gao W. Zhao D. Large-vocabulary continuous sign language recognition based on transition-movement models IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2007 37 1 9 10.1109/TSMCA.2006.886347

상세보기
38. McDonald J. Kell D. Markham C. Weakly supervised training of a sign language recognition system using multiple instance learning density matrices IEEE Trans. Syst. Man Cybern. Part B Cybern. 2011 41 526 541
39. Roy S.H. Moore J. de Luca G. Nawab S.H. de Luca C.J. Cheng M.S. Chang S. A combined sEMG and accelerometer system for monitoring functional activity in stroke IEEE Trans. Neural Syst. Rehabil. Eng. 2009 17 585 594 10.1109/TNSRE.2009.2036615 20051332

상세보기
40. Li Y. Lantz V. Wang K. Zhang X. Chen X. Yang J. A framework for hand gesture recognition based on accelerometer and EMG sensors IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2011 41 1064 1076
41. Morris D. Saponas T.S. Tan D.S. Balakrishnan R. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Boston, MA, USA 15–17 March 2008 515 524
42. Liang R.H. Ouhyoung M. A real-time continuous gesture recognition system for sign language Proceedings of the International Conference Automatic Face and Gesture Recognition Nara, Japan 14–16 April 1998 558 565
43. Zhou P. Li X. Aruin A.S. Teager-kaiser energy operation of surface EMG improves muscle activity onset detection Ann. Biomed. Eng. 2007 35 1532 1538 17473984

상세보기
44. Zhang X. Zhou P. Sample entropy analysis of surface EMG for improved muscle activity onset detection against spurious background spikes J. Electromyogr. Kinesiol. 2012 22 901 907 10.1016/j.jelekin.2012.06.005 22800657

상세보기
45. Xu Q. Quan Y. Yang L. He J. An adaptive algorithm for the determination of the onset and offset of muscle contraction by emg signal processing IEEE Trans. Neural Syst. Rehabil. Eng. 2013 21 65 73 10.1109/TNSRE.2012.2226916 23193462

상세보기
46. Ying D. Liu J. Rymer W.Z. EMG burst presence probability: A joint time-frequency representation of muscle activity and its application to onset detection J. Biomech. 2015 48 1193 1197 25748222

상세보기
47. Cheng J. Chen X. Lu Z. Key-press gestures recognition and interaction based on sEMG signals Proceedings of the 1st International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010 Beijing, China 8–12 November 2010
48. Chen X. Wang Z.J. Pattern recognition of number gestures based on a wireless surface EMG system Biomed. Signal Proc. Control 2013 8 184 192 10.1016/j.bspc.2012.08.005

상세보기
49. Oskoei M.A. Hu H. Myoelectric control systems: A survey Biomed. Signal Proc. Control 2007 2 275 294 10.1016/j.bspc.2007.07.009

상세보기
50. Liu C. Wechsler H. Robust coding schemes for indexing and retrieval from large face databases IEEE Trans. Image Proc. Publ. IEEE Signal Proc. Soc. 2000 9 132 137
51. Xu P. Sun Q. Liu R. Jiang L. Dynamic-time-warping-based measurement data alignment model for condition-based railroad track maintenance IEEE Trans. Intell. Transp. Syst. 2015 16 799 812 10.1109/TITS.2014.2342235

상세보기
52. Deng J.J. Leung C.H.C. Dynamic time warping for music retrieval using time series modeling of musical emotions IEEE Trans. Affect. Comput. 2015 6 137 151 10.1109/TAFFC.2015.2404352

상세보기
53. Salvador S. Chan P. Toward accurate dynamic time warping in linear time and space Intell. Data Anal. 2007 11 561 580
54. Coelho A.L.V. Lima C.A.M. Assessing fractal dimension methods as feature extractors for EMG signal classification Eng. Appl. Artif. Intell. 2014 36 81 98 10.1016/j.engappai.2014.07.009

상세보기
55. Chen J.J. Shiavi R. Temporal feature extraction and clustering analysis of electromyographic linear envelopes in gait studies IEEE Trans. Bio-Med. Eng. 1990 37 295 302 10.1109/10.52330 2184121

상세보기
56. Parker P.A. Englehart K. Hudgins B. Stevenson M. Classification of the myoelectric signal using time-frequency based representations Med. Eng. Phys. 1999 21 431 438 10624739
57. Kuzborskij I. Elsig S. Mittaz H.A.G. Deriaz O. Castellini C. Muller H. Atzori M. Gijsberts A. Caputo B. Characterization of a benchmark database for myoelectric movement classification IEEE Trans. Neural Syst. Rehabil. Eng. 2015 23 73 83 25486646

상세보기
58. Rabiner L. A tutorial on hidden Markov models and selected applications in speech recognition Proc. IEEE 1989 77 257 286 10.1109/5.18626

상세보기
59. Lee K.S. EMG-based speech recognition using hidden Markov models with global control variables IEEE Trans. Biomed. Eng. 2008 55 930 940 10.1109/TBME.2008.915658 18334384

상세보기
60. Chan A.D.C. Englehart K.B. Continuous myoelectric control for powered prostheses using hidden Markov models IEEE Trans. Biomed. Eng. 2005 52 121 124 10.1109/TBME.2004.836492 15651571

상세보기
61. Wang Y. Zhang X. Zhao Z. A hybrid speech recognition training method for HMM based on genetic algorithm and Baum Welch algorithm Proceedings of the Second International Conference on Innovative Computing, Information and Control Kumamoto, Japan 5–7 September 2007 572
62. Wickramasuriya J. Liu J. Zhong L. Vasudevan V. uWave: Accelerometer-based personalized gesture recognition and its applications Pervasive Mob. Comput. 2009 5 657 675

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors 원문보기

Abstract ▼ AI-Helper

주제어

참고문헌 (62)

이 논문을 인용한 문헌

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트