[논문]한국어 언어 모델을 활용한 보이스피싱 탐지 기능 개선

박동주

doi:10.3745/ktsde.2022.11.10.437

한국어 언어 모델을 활용한 보이스피싱 탐지 기능 개선
Exploiting Korean Language Model to Improve Korean Voice Phishing Detection 원문보기

정보처리학회논문지. KIPS transactions on software and data engineering. 소프트웨어 및 데이터 공학, v.11 no.10, 2022년, pp.437 - 446

초록
AI-Helper

보이스피싱 통화 내용을 탐지하고 분류하는데 핵심 엔진으로 최신 머신러닝(ML) 및 딥러닝(DL) 알고리즘과 결합된 자연어 처리(NLP)의 텍스트 분류 작업이 널리 사용된다. 비대면 금융거래의 증가와 더불어 보이스피싱 통화 내용 분류에 대한 많은 연구가 진행되고 양호한 성과를 보이고 있지만, 최신 NLP 기술을 활용한 성능 개선의 필요성이 여전히 존재한다. 본 논문은 KorCCVi라는 레이블이 지정된 한국 보이스 피싱 데이터의 텍스트 분류를 기반으로 여러 다른 최신 알고리즘과 비교하여 사전 훈련된 한국어 모델 KoBERT의 한국 보이스 피싱 탐지 성능을 벤치마킹한다. 실험 결과에 따르면 KoBERT 모델의 테스트 집합에서 분류 정확도가 99.60%로 다른 모든 모델의 성능을 능가한다.

Abstract ▼ AI-Helper

Text classification task from Natural Language Processing (NLP) combined with state-of-the-art (SOTA) Machine Learning (ML) and Deep Learning (DL) algorithms as the core engine is widely used to detect and classify voice phishing call transcripts. While numerous studies on the classification of voice phishing call transcripts are being conducted and demonstrated good performances, with the increase of non-face-to-face financial transactions, there is still the need for improvement using the latest NLP technologies. This paper conducts a benchmarking of Korean voice phishing detection performances of the pre-trained Korean language model KoBERT, against multiple other SOTA algorithms based on the classification of related transcripts from the labeled Korean voice phishing dataset called KorCCVi. The results of the experiments reveal that the classification accuracy on a test set of the KoBERT model outperforms the performances of all other models with an accuracy score of 99.60%.

주제어

표/그림 (13)

그림 Fig. 1. Text Classification Workflow
그림 Fig. 2. Workbench Methodology
표 Table 1. Summary of Content Removed from the Dataset During Data Preprocessing
표 Table 2. Summary of the Features Engineering Process
표 Table 3. Sampling of the Dataset in Training and Test Set
표 Table 4. Extract of KorCCVi Dataset with English Translation
그림 Fig. 3. Fine-tuning KoBERT Classification Model for Voice Phishing Detection
그림 Fig. 4. Pseudocode of the Algorithm for Fine-tuning the KoBERT Model
표 Table 5. Benchmarking Results of the ML and DL models' performances
그림 Fig. 5. Modeling Time Comparison
그림 Fig. 6. Precision Score Comparison
그림 Fig. 7. Recall Score Comparison
표 Table 6. Comparison of the ML and DL models' Complexity

참고문헌 (25)

K. Kowsari, D. E. Brown, M. Heidarysafa, K. J. Meimandi, M. S. Gerber, and L. E. Barnes, "HDLTex: Hierarchical deep learning for text classification," 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp.364-371, 2017. doi: 10.1109/ICMLA.2017.0-134.
C. C. Aggarwal and C. X. Zhai, "A survey of text classification algorithms," Mining Text Data, pp.163-222, Aug. 2012. doi: 10.1007/978-1-4614-3223-4_6.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.4171-4186, 2018, Accessed: Dec. 27, 2021. [Online]. Available: https://arxiv.org/abs/1810.04805v2.
SKTBrain/KoBERT: Korean BERT pre-trained cased (KoBERT) [Internet], https://github.com/SKTBrain/KoBERT (accessed Dec. 27, 2021).
M. K. M. Boussougou, S. Jin, D. Chang, and D.-J. Park, "Korean voice phishing text classification performance analysis using machine learning techniques," Proceedings of the Korea Information Processing Society Conference, pp. 297-299, 2021. doi: 10.3745/PKIPS.Y2021M11A.297.
K. Kowsari, K. J. Meimandi, M. Heidarysafa, S. Mendu, L. E. Barnes, and D. E. Brown, "Text classification algorithms: A survey," Information (Switzerland), Vol.10, No.4, Apr. 2019, doi: 10.3390/info10040150.

상세보기
M. K. M. Boussougou and D.-J. Park, "A real-time efficient detection technique of voice phishing with AI," in Proceedings of Korea Software Congress 2021, pp.768-770, 2021, Accessed: Oct. 13, 2021. [Online]. Available: https://www.dbpia.co.kr/Journal/articleDetail?nodeIdNODE10583070.
Catboost, "CatBoost - open-source gradient boosting library," 2021. [Internet], https://catboost.ai/ (accessed Mar. 11, 2022).
XBGoost, "XGBoost Documentation - Introduction to Boosted Trees," 2020. [Internet], https://xgboost.readthedocs.io/en/latest/tutorials/model.html (accessed Mar. 11, 2022).
T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system," Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.785794, 2016. doi: 10.1145/2939672.2939785.
G. Ke et al., "LightGBM: A highly efficient gradient boosting decision tree," Advances in Neural Information Processing Systems, pp.3147-3155, 2017, Accessed: Mar. 11, 2022. [Online]. Available: https://github.com/Microsoft/LightGBM.
"Welcome to LightGBM's documentation! - LightGBM 3.3.2.99 documentation." [Internet], https://lightgbm.readthedocs.io/en/latest/index.html (accessed Mar. 11, 2022).
Z. C. Lipton, J. Berkowitz, and C. Elkan, "A critical review of recurrent neural networks for sequence learning," 2015, Accessed: Mar. 11, 2022. [Online]. Available: http://arxiv.org/abs/1506.00019.
A. Sherstinsky, "Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network," Physica D: Nonlinear Phenomena, Vol.404, 2020, doi: 10.1016/j.physd.2019.132306.

상세보기
M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," IEEE Transactions on Signal Processing, Vol.45, No.11, pp.2673-2681, 1997. doi: 10.1109/78.650093.

상세보기
A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Networks, Vol.18, No.5-6, pp.602-610, 2005, doi: 10.1016/j.neunet.2005.06.042.

상세보기
K. Cho et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," EMNLP 2014 - 2014 In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1724-1734, 2014. doi: 10.3115/v1/d14-1179.
H. Cho, H. Im, Y. Yi, and J. Cha, "Comparison of Korean classification models' Korean essay score range prediction performance," KIPS Transactions on Software and Data Engineering, Vol.11, No.3, pp.133-140, 2022. doi: 10.3745/KTSDE.2022.11.3.133.

원문보기 상세보기
S. Choi, M.-K. Park, and E. Kim, "How are Korean neural language models 'surprised' layerwisely?," Journal of Language Sciences, Vol.28, No.4, pp.301-317, 2021. doi: 10.14384/kals.2021.28.4.301.

상세보기
K. Yang, "Transformer-based Korean pretrained language models: A survey on three years of progress," 2021, Accessed: Mar. 11, 2022. [Online]. Available: http://arxiv.org/abs/2112.03014.
K. Yang, W. Jang, and W. I. Cho, "APEACH: Attacking pejorative expressions with analysis on crowd-generated hate speech evaluation datasets," 2022, Accessed: Mar. 13, 2022. [Online]. Available: http://arxiv.org/abs/2202.12459.
S. Park, H. Yang, M. Choe, M. Ha, K. Chung, and M. Koo, "Sentimental Analysis of YouTube Korean Comments Using KoBERT," in Proceedings of Korea Software Congress 2020, pp.1385-1387, 2020. [Online]. Available: http://www.dbpia.co.kr/journal/articleDetail?nodeIdNODE10529995
K.-H. Park, Y.-S Jeong, "Korean daily conversation topics classification using KoBERT," in Proceedings of Korea Software Congress 2021, pp.1735-1737, 2021. Accessed: Dec. 27, 2021. [Online]. Available: https://www.dbpia.co.kr/journal/articleDetail?nodeIdNODE10583420.
A. K. Uysal and S. Gunal, "The impact of preprocessing on text classification," Information Processing & Management, Vol.50, No.1, pp.104-112, 2014. doi: 10.1016/J.IPM.2013.08.006.

상세보기
E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov, "Learning word vectors for 157 languages," 2019. Accessed: Nov. 15, 2020. [Online]. Available: https://fasttext.cc/

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증