[논문]단어그룹 확장 기법을 활용한 순환신경망 알고리즘 성능개선 연구

박대승; 성열우; 김정길

doi:10.22678/jic.2022.20.4.023

단어그룹 확장 기법을 활용한 순환신경망 알고리즘 성능개선 연구
A Study on Performance Improvement of Recurrent Neural Networks Algorithm using Word Group Expansion Technique 원문보기

산업융합연구 = Journal of industrial convergence, v.20 no.4, 2022년, pp.23 - 30

박대승 (남서울대학교 컴퓨터소프트웨어학과) , 성열우 (남서울대학교 컴퓨터소프트웨어학과) , 김정길 (남서울대학교 컴퓨터소프트웨어학과)

초록
AI-Helper

최근 인공지능(AI)과 딥러닝 발전으로 대화형 인공지능 챗봇의 중요성이 부각되고 있으며 다양한 분야에서 연구가 진행되고 있다. 챗봇을 만들기 위해서 직접 개발해 사용하기도 하지만 개발의 용이성을 위해 오픈소스 플랫폼이나 상업용 플랫폼을 활용하여 개발한다. 이러한 챗봇 플랫폼은 주로 RNN (Recurrent Neural Network)과 응용 알고리즘을 사용하며, 빠른 학습속도와 모니터링 및 검증의 용이성 그리고 좋은 추론 성능의 장점을 가지고 있다. 본 논문에서는 RNN과 응용 알고리즘의 추론 성능 향상방법을 연구하였다. 제안 방법은 RNN과 응용 알고리즘 적용 시 각 문장에 대한 핵심단어의 단어그룹에 대해 확장학습을 통해 데이터에 내재된 의미를 넓히는 기법을 사용하였다. 본 연구의 결과는 순환 구조를 갖는 RNN, GRU (Gated Recurrent Unit), LSTM (Long-short Term Memory) 세 알고리즘에서 최소 0.37%에서 최대 1.25% 추론 성능향상을 달성하였다. 본 연구를 통해 얻은 연구결과는 관련 산업에서 인공지능 챗봇 도입을 가속하고 다양한 RNN 응용 알고리즘을 활용하도록 하는데 기여할 수 있다. 향후 연구에서는 다양한 활성 함수들이 인공신경망 알고리즘의 성능 향상에 미치는 영향에 관한 연구가 필요할 것이다.

Abstract ▼ AI-Helper

Recently, with the development of artificial intelligence (AI) and deep learning, the importance of conversational artificial intelligence chatbots is being highlighted. In addition, chatbot research is being conducted in various fields. To build a chatbot, it is developed using an open source platform or a commercial platform for ease of development. These chatbot platforms mainly use RNN and application algorithms. The RNN algorithm has the advantages of fast learning speed, ease of monitoring and verification, and good inference performance. In this paper, a method for improving the inference performance of RNNs and applied algorithms was studied. The proposed method used the word group expansion learning technique of key words for each sentence when RNN and applied algorithm were applied. As a result of this study, the RNN, GRU, and LSTM three algorithms with a cyclic structure achieved a minimum of 0.37% and a maximum of 1.25% inference performance improvement. The research results obtained through this study can accelerate the adoption of artificial intelligence chatbots in related industries. In addition, it can contribute to utilizing various RNN application algorithms. In future research, it will be necessary to study the effect of various activation functions on the performance improvement of artificial neural network algorithms.

주제어

참고문헌 (31)

Rahman, A. M., Al Mamun, A. & Islam, A. (2018). Programming challenges of chatbot: Current and future prospective. 5th IEEE Region 10 Humanitarian Technology Conference 2017, R10-HTC 2017, 2018-Janua, 75-78. DOI : 10.1109/R10-HTC.2017.8288910
Hristidis, V. (2019). Chatbot Technologies and Challenges. 2018 First International Conference on Artificial Intelligence for Industries (AI4I), 126-126. DOI : 10.1109/ai4i.2018.8665692
Chen, L., Chen, P. & Lin, Z. (2020). Artificial Intelligence in Education: A Review. IEEE Access, 8, 75264-75278. DOI : 10.1109/ACCESS.2020.2988510

상세보기
Daniel, G. & Cabot, J. (2021). The Software Challenges of Building Smart Chatbots. Proceedings - International Conference on Software Engineering, 324-325. DOI : 10.1109/ICSE-COMPANION52605.2021.00138
Nguyen, P. A. H. (2021). Evaluating AI chatbot platforms by a fuzzy AHP approach. Proceedings of 2021 International Conference on System Science and Engineering, ICSSE 2021, 206-209. DOI : 10.1109/ICSSE52999.2021.9538447
Prasetyo, A. & Santoso, H. A. (2021). Intents Categorization for Chatbot Development Using Recurrent Neural Network (RNN) Learning. 2021 7th International Conference on Advanced Computing and Communication Systems, ICACCS 2021, 551-556. DOI : 10.1109/ICACCS51430.2021.9441947
Majid, R. & Santoso, H. A. (2021). Conversations Sentiment and Intent Categorization Using Context RNN for Emotion Recognition. 2021 7th International Conference on Advanced Computing and Communication Systems, ICACCS 2021, 46-50. DOI : 10.1109/ICACCS51430.2021.9441740
Anki, P., Bustamam, A., Al-Ash, H. S. & Sarwinda, D. (2020). High Accuracy Conversational AI Chatbot Using Deep Recurrent Neural Networks Based on BiLSTM Model. 2020 3rd International Conference on Information and Communications Technology, ICOIACT 2020, 382-387. DOI : 10.1109/ICOIACT50329.2020.9332074
Mou, L., Ghamisi, P. & Zhu, X. X. (2017). Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 3639-3655. DOI : 10.1109/TGRS.2016.2636241

상세보기
Li, S., Li, W., Cook, C., Zhu, C. & Gao, Y. (2018). Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 5457-5466. DOI : 10.1109/CVPR.2018.00572
Williams, R. J. & Zipser, D. (1989). A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation, 1(2), 270-280. DOI : 10.1162/neco.1989.1.2.270

상세보기
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T. & Saenko, K. (2015). Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07-12-June, 2625-2634. DOI : 10.1109/CVPR.2015.7298878
Lebeuf, C., Storey, M. A. & Zagalsky, A. (2017). Software Bots. IEEE Software, 35(1), 18-23. DOI : 10.1109/MS.2017.4541027

상세보기
Daniel, G., Cabot, J., Deruelle, L. & Derras, M. (2020). Xatkit: a Multimodal Low-Code Chatbot Development Framework. IEEE Access, 8, 15332-15346. DOI : 10.1109/aCCESS.2020.2966919

상세보기
Lokman, A. S. & Ameedeen, M. A. (2019). Modern chatbot systems: A technical review. Advances in Intelligent Systems and Computing, 881, 1012- 1023. DOI : 10.1007/978-3-030-02683-7_75
Hussain, S., Ameri Sianaki, O. & Ababneh, N. (2019). A Survey on Conversational Agents/Chatbots Classification and Design Techniques. Advances in Intelligent Systems and Computing, 927, 946-956. DOI : 10.1007/978-3-030-15035-8_93
Mellado-Silva, R., Faundez-Ugalde, A. & Lobos, M. B. (2020). Learning tax regulations through rules-based chatbots using decision trees: A case study at the time of COVID-19. Proceedings - International Conference of the Chilean Computer Science Society, SCCC, 2020-Novem. DOI : 10.1109/SCCC51225.2020.9281267
Shukla, V. K. & Verma, A. (2019). Enhancing LMS Experience through AIML Base and Retrieval Base Chatbot using R Language. 2019 International Conference on Automation, Computational and Technology Management, ICACTM 2019, 561-567. DOI : 10.1109/ICACTM.2019.8776684
Dutt, V., Sasubilli, S. M. & Yerrapati, A. E. (2020). Dynamic Information Retrieval with Chatbots: A Review of Artificial Intelligence Methodology. Proceedings of the 4th International Conference on Electronics, Communication and Aerospace Technology, ICECA 2020, 1299-1303. DOI : 10.1109/ICECA49313.2020.9297533
Varghese, E. & Pillai, M. T. R. (2018). A Standalone Generative Conversational Interface Using Deep Learning. Proceedings of the International Conference on Inventive Communication and Computational Technologies, ICICCT 2018, 1915-1920. DOI : 10.1109/ICICCT.2018.8473211
Nigam, A., Sahare, P. & Pandya, K. (2019). Intent Detection and Slots Prompt in a Closed-Domain Chatbot. Proceedings - 13th IEEE International Conference on Semantic Computing, ICSC 2019, 340-343. DOI : 10.1109/ICOSC.2019.8665635
Roller, S. et al. (2021). Recipes for building an open-domain chatbot. EACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference, 300-325. DOI : 10.18653/v1/2021.eacl-main.24
Young, T., Hazarika, D., Poria, S. & Cambria, E. (2018). Recent trends in deep learning based natural language processing [Review Article]. In IEEE Computational Intelligence Magazine (Vol. 13, Issue 3, pp. 55-75). Institute of Electrical and Electronics Engineers Inc. DOI : 10.1109/MCI.2018.2840738

상세보기
Ruineihart, D. E., Hinton, G. E. & Williams, R. J. (1986). Learning Internal Representations Error Propagation. In Cognitive Science (Vol. 1, Issue V). Retrieved from : https://apps.dtic.mil/docs/citations/ADA164453
Hochreiter, S. & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780. DOI : 10.1162/neco.1997.9.8.1735

상세보기
Chung, J., Gulcehre, C., Cho, K. & Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. Presented in NIPS 2014 Deep Learning and Representation Learning Workshop. DOI : 10.48550/arxiv.1412.3555
Shen, Y., Zhang, Q., Zhang, J., Huang, J., Lu, Y. & Lei, K. (2019). Improving medical short text classification with semantic expansion using word-cluster embedding. Lecture Notes in Electrical Engineering, 514, 401-411. DOI : 10.1007/978-981-13-1056-0_41
Cao, W., Song, A. & Hu, J. (2017). Stacked residual recurrent neural network with word weight for text classification. IAENG International Journal of Computer Science, 44(3), 277-284.

상세보기
Kim, S., Na, I., Shin, J. & Society, K. M. (2019). A Method on Associated Document Recommendation with Word Correlation Weights. Journal of Korea Multimedia Society, 22(2), 250-259. DOI : 10.9717/kmms.2019.22.2.250

원문보기 상세보기
Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowlege-Based Systems, 6(2), 107-116. DOI : 10.1142/S0218488598000094

상세보기
Talathi, S. S. & Vartak, A. (2015). Improving performance of recurrent neural network with relu nonlinearity. arXiv preprint arXiv:1511.03771. DOI : 10.48550/arxiv.1511.03771

저자의 다른 논문 :

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증