[논문]모바일 플랫폼 교육 콘텐츠 지원을 위한 손 글씨 기반 텍스트 인터페이스 설계

조윤식; 조세홍; 김진모

doi:10.15701/kcgs.2021.27.5.81

모바일 플랫폼 교육 콘텐츠 지원을 위한 손 글씨 기반 텍스트 인터페이스 설계
Design of Handwriting-based Text Interface for Support of Mobile Platform Education Contents 원문보기

컴퓨터그래픽스학회논문지 = Journal of the Korea Computer Graphics Society, v.27 no.5, 2021년, pp.81 - 89

조윤식 (한성대학교 일반대학원 컴퓨터공학과) , 조세홍 (한성대학교 컴퓨터공학부) , 김진모 (한성대학교 컴퓨터공학부)

초록
AI-Helper

본 연구는 모바일 플랫폼 환경에서 언어 기반의 교육 콘텐츠 지원을 위한 텍스트 인터페이스를 제안한다. 이는 손 글씨를 통해 단어를 작성하는 입력 구조로 딥 러닝을 활용한다. 모바일 플랫폼 콘텐츠의 버튼, 메뉴 등을 활용한 GUI (Graphical User Interface)와 화면 터치, 클릭, 드래그 등의 입력 방식을 기반으로 손 글씨를 사용자로부터 직접 입력하여 처리할 수 있는 텍스트 인터페이스를 설계한다. 이는 EMNIST (Extended Modified National Institute of Standards and Technology database) 데이터 셋과 훈련된 CNN (Convolutional Neural Network)을 사용하여 알파벳 텍스트를 분류하고 조합하여 단어를 완성한다. 최종적으로 영어 단어 교육 콘텐츠를 직접 제작하여 제안하는 인터페이스의 학습 지원 효과를 분석하고 만족도를 비교하기 위한 실험을 진행한다. 동일한 교육 환경에서 기존의 키 패드 방식의 인터페이스와 제안하는 손 글씨 기반 텍스트 인터페이스를 서로 체험한 사용자들이 제시하는 영어 단어를 학습하는 능력을 비교하고, 인터페이스를 조작하여 단어를 작성하는 과정에서의 전체적인 만족도를 분석, 확인하도록 한다.

Abstract ▼ AI-Helper

This study proposes a text interface for support of language-based educational contents in a mobile platform environment. The proposed interface utilizes deep learning as an input structure to write words through handwriting. Based on GUI (Graphical User Interface) using buttons and menus of mobile platform contents and input methods such as screen touch, click, and drag, we design a text interface that can directly input and process handwriting from the user. It uses the EMNIST (Extended Modified National Institute of Standards and Technology database) dataset and a trained CNN (Convolutional Neural Network) to classify and combine alphabetic texts to complete words. Finally, we conduct experiments to analyze the learning support effect of the interface proposed by directly producing English word education contents and to compare satisfaction. We compared the ability to learn English words presented by users who have experienced the existing keypad-type interface and the proposed handwriting-based text interface in the same educational environment, and we analyzed the overall satisfaction in the process of writing words by manipulating the interface.

주제어

참고문헌 (34)

T. Sweeney, "Foundational principles & technologies for the metaverse," in ACM SIGGRAPH 2019 Talks, ser. SIGGRAPH '19. New York, NY, USA: Association for Computing Machinery, 28 July-1 August 2019. [Online]. Available: https://doi.org/10.1145/3306307.3339844
A. van Dam, "Post-wimp user interfaces," Commun. ACM, vol. 40, no. 2, p. 63-67, feb 1997. [Online]. Available: https://doi.org/10.1145/253671.253708

상세보기
H.-Y. Huang, C.-W. Ning, P.-Y. Wang, J.-H. Cheng, and L.-P. Cheng, "Haptic-go-round: A surrounding platform for encounter-type haptics in virtual reality experiences," in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ser. CHI '20. New York, NY, USA: Association for Computing Machinery, 25-30 April 2020, p. 1-10. [Online]. Available: https://doi.org/10.1145/3313831.3376476
Shneiderman, "Direct manipulation: A step beyond programming languages," Computer, vol. 16, no. 8, pp. 57-69, 1983.

상세보기
J.-H. Sung and T. Y. Kim, "Leap motion-based playing yut through hand gesture interaction," The Journal of Korean Institute of Next Generation Computing, vol. 13, no. 1, pp. 74-81, 2017.
B. D. Song, H.-J. Kim, H.-J. Jeong, and Y. J. Choi, "Development of interactive media player for kiosk with user motion detection," The Journal of the Korea Contents Association, vol. 19, no. 11, pp. 270-277, 2019.

원문보기 상세보기
J.-H. Kim, "Interaction technique in smoke simulations using mouth-wind on mobile devices," Journal of the Korea Computer Graphics Society, vol. 24, no. 4, pp. 21-27, 2018.

원문보기 상세보기
S. Hong, G. Na, Y. Cho, and J. Kim, "A study on movement interface in mobile virtual reality," Journal of the Korea Computer Graphics Society, vol. 27, no. 3, pp. 55-63, 2021.
Y. Cho, J. Kang, J. Jeon, J. Park, M. Kim, and J. Kim, "X-person asymmetric interaction in virtual and augmented realities," Computer Animation and Virtual Worlds, vol. 32, no. 5, p. e1985, 2021. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/cav.1985

상세보기
Y. Cho and J. Kim, "Production of mobile english language teaching application based on text interface using deep learning," Electronics, vol. 10, no. 15, 2021. [Online]. Available: https://www.mdpi.com/2079-9292/10/15/1809
T. Kang, M. Chae, E. Seo, M. Kim, and J. Kim, "Deep-handsvr: Hand interface using deep learning in immersive virtual reality," Electronics, vol. 9, no. 11, 2020. [Online]. Available: https://www.mdpi.com/2079-9292/9/11/1863
R. Jayakanthan, "Application of computer games in the field of education," Electronic Library, vol. 20, no. 2, pp. 98-102, 2002. [Online]. Available: https://www.learntechlib.org/p/95865

상세보기
V. S. Zirawaga, A. I. Olusanya, and T. Maduku, "Gaming in education: Using games as a support tool to teach history," Journal of Education and Practice, vol. 8, no. 15, pp. 55-64, 2017.
남현숙, "2020년 국외 디지털콘텐츠 시장조사 및 동향 심층분석," 소프트웨어정책연구소 연구보고서, vol. RE-107, pp. 1-378, 2021.
Y. Chen, D. Zhou, Y. Wang, and J. Yu, "Application of augmented reality for early childhood english teaching," in 2017 International Symposium on Educational Technology (ISET), 2017, pp. 111-115.
M. Fan, A. N. Antle, and J. L. Warren, "Augmented reality for early language learning: A systematic review of augmented reality application design, instructional strategies, and evaluation outcomes," Journal of Educational Computing Research, vol. 58, no. 6, pp. 1059-1100, 2020. [Online]. Available: https://doi.org/10.1177/0735633120927489

상세보기
M. Lu, "Effectiveness of vocabulary learning via mobile phone," Journal of Computer Assisted Learning, vol. 24, no. 6, pp. 515-525, 2008. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-2729.2008.00289.x

상세보기
J. Jun, J. Lim, and S. Kim, "Investigating the educational effectiveness of an online educational game," The journal of Educational Studies, vol. 39, no. 1, pp. 85-105, 2008.
D. Aminatun and L. Oktaviani, "Memrise: Promoting students' autonomous learning skill through language learning application," Metathesis: Journal of English Language, Literature, and Teaching, vol. 3, no. 2, pp. 214-223, 2019.
S. E. Lindley, J. Le Couteur, and N. L. Berthouze, "Stirring up experience through movement in game play: Effects on engagement and social behaviour," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ser. CHI '08. New York, NY, USA: Association for Computing Machinery, 2008, p. 511-514. [Online]. Available: https://doi.org/10.1145/1357054.1357136
K. V. Nesbitt and I. Hoskens, "Multi-sensory game interface improves player satisfaction but not performance," in Proceedings of the Ninth Conference on Australasian User Interface - Volume 76, ser. AUIC '08. AUS: Australian Computer Society, Inc., 2008, p. 13-18.
N. Sidorakis, G. A. Koulieris, and K. Mania, "Binocular eye-tracking for the control of a 3d immersive multimedia user interface," in 2015 IEEE 1st Workshop on Everyday Virtual Reality (WEVR), 23 March 2015, pp. 15-18.
M. Kim, J. Lee, C. Jeon, and J. Kim, "A study on interaction of gaze-based user interface in mobile virtual reality environment," Journal of the Korea Computer Graphics Society, vol. 23, no. 3, pp. 39-46, 2017. [Online]. Available: http://dx.doi.org/10.15701/kcgs.2017.23.3.39

원문보기 상세보기
I. Choi, E. Ofek, H. Benko, M. Sinclair, and C. Holz, "Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ser. CHI '18. New York, NY, USA: ACM, 21-26 April 2018, pp. 654:1-654:13. [Online]. Available: http://doi.acm.org/10.1145/3173574.3174228
J. Kim, "A study on comparative experiment of hand-based interface in immersive virtua reality," Journal of the Korea Computer Graphics Society, vol. 25, no. 2, pp. 1-9, 2019.
M. Kim, J. Kim, K. Jeong, and C. Kim, "Grasping vr: Presence of pseudo-haptic interface based portable hand grip system in immersive virtual reality," International Journal of Human-Computer Interaction, vol. 36, no. 7, pp. 685-698, 2020.

상세보기
J. Kim, "Vivr: Presence of immersive interaction for visual impairment virtual reality," IEEE Access, vol. 8, pp. 196 151-196 159, 2020.

상세보기
S. Yang and M. Zhang, "Application of brain neural network in personalized english education system," International Journal of Emerging Technologies in Learning (iJET), vol. 13, no. 10, pp. 15-22, 2018. [Online]. Available: https://online-journals.org/index.php/i-jet/article/view/9488
G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, "Emnist: Extending mnist to handwritten letters," in 2017 International Joint Conference on Neural Networks (IJCNN), 14-19 May 2017, pp. 2921-2926.
A. Baldominos, Y. Saez, and P. Isasi, "A survey of handwritten character recognition with mnist and emnist," Applied Sciences, vol. 9, no. 15, 2019. [Online]. Available: https://www.mdpi.com/2076-3417/9/15/3169
S. S. Mor, S. Solanki, S. Gupta, S. Dhingra, M. Jain, and R. Saxena, "Handwritten text recognition: With deep learning and android," International Journal of Engineering and Advanced Technology (IJEAT), vol. 8, no. 3S, pp. 819-825, 2019. [Online]. Available: https://www.ijeat.org/wpcontent/uploads/papers/v8i3S/C11730283S19.pdf

상세보기
UnityTechnologies, "Unity engine," Unity Technologies, 2019. [Online]. Available: https://unity3d.com/
GoogleBrain, "Tensorflow," Google Brain, 2021. [Online]. Available: https://www.tensorflow.org/
A. Lund, "Measuring usability with the use questionnaire," Usability Interface, vol. 8, no. 2, pp. 3-6, 01 2001.

저자의 다른 논문 :

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증