[논문]설명 가능한 정기예금 가입 여부 예측을 위한 앙상블 학습 기반 분류 모델들의 비교 분석

신지안; 문지훈; 노승민

doi:10.7838/jsebs.2021.26.3.097

[국내논문] 설명 가능한 정기예금 가입 여부 예측을 위한 앙상블 학습 기반 분류 모델들의 비교 분석
A Comparative Analysis of Ensemble Learning-Based Classification Models for Explainable Term Deposit Subscription Forecasting 원문보기

한국전자거래학회지 = The Journal of Society for e-Business Studies, v.26 no.3, 2021년, pp.97 - 117

신지안 (Department of Security Convergence Science, Chung-Ang University) , 문지훈 (Chung-Ang University) , 노승민 (Department of Industrial Security, Chung-Ang University)

초록
AI-Helper

정기예금 가입 여부 예측은 은행의 대표적인 금융 마케팅 중 하나로, 은행은 다양한 고객 정보를 활용하여 예측 모델을 구성할 수 있다. 정기예금 가입 여부의 분류 정확도를 향상하기 위해, 많은 연구에서 기계학습 기법들을 이용하여 분류 모델들을 개발하였다. 하지만, 이러한 모델들이 만족스러운 성능을 보일지라도 모델의 의사결정 과정에 대한 근거가 적절하게 설명되지 않는다면 산업에서 활용하기가 쉽지 않다. 이러한 문제점을 해결하기 위해, 본 논문은 설명 가능한 정기예금 가입 여부 예측 기법을 제안한다. 먼저, 테이블 형식에서 우수한 성능을 도출하는 의사결정 나무 기반 앙상블 학습 기법인 랜덤 포레스트, GBM, XGBoost, LightGBM을 이용하여 분류 모델들을 개발하고, 10겹 교차검증을 통해 모델들의 분류 성능을 심층 분석한다. 다음으로, 가장 우수한 성능을 도출하는 모델에 설명 가능한 인공지능 기법인 SHAP을 적용하여 고객 정보의 영향도와 의사결정 과정 등을 해석할 수 있는 근거를 제공한다. 제안한 기법의 실용성과 타당성을 입증하기 위해, Kaggle에서 제공한 은행 마케팅 데이터 셋을 대상으로 모의실험을 진행하였으며, 데이터 셋 구성에 따라 GBM과 LightGBM 모델에 SHAP을 각기 적용하여 설명 가능한 정기예금 가입 여부를 위한 분석 및 시각화를 수행하였다.

Abstract ▼ AI-Helper

Predicting term deposit subscriptions is one of representative financial marketing in banks, and banks can build a prediction model using various customer information. In order to improve the classification accuracy for term deposit subscriptions, many studies have been conducted based on machine learning techniques. However, even if these models can achieve satisfactory performance, utilizing them is not an easy task in the industry when their decision-making process is not adequately explained. To address this issue, this paper proposes an explainable scheme for term deposit subscription forecasting. For this, we first construct several classification models using decision tree-based ensemble learning methods, which yield excellent performance in tabular data, such as random forest, gradient boosting machine (GBM), extreme gradient boosting (XGB), and light gradient boosting machine (LightGBM). We then analyze their classification performance in depth through 10-fold cross-validation. After that, we provide the rationale for interpreting the influence of customer information and the decision-making process by applying Shapley additive explanation (SHAP), an explainable artificial intelligence technique, to the best classification model. To verify the practicality and validity of our scheme, experiments were conducted with the bank marketing dataset provided by Kaggle; we applied the SHAP to the GBM and LightGBM models, respectively, according to different dataset configurations and then performed their analysis and visualization for explainable term deposit subscriptions.

Keyword

표/그림 (18)

표 Summary of Previous Works Based on XAI with SHAP
그림 System Architecture
표 List of Independent and Dependent Variables and Their Data Type
표 Advantage and Disadvantage of Tree-based Ensemble Learning Methods
그림 Flowchart of Bagging Process
그림 Illustration Demonstrating the Difference between Leaf-Wise Growth (above) and Level-Wise (below)
그림 Example of 10-fold Cross-Validation
표 Confusion Matrix
표 Guidelines of Landis and Koch
표 Selected Hyperparameters for Each Tree-based Ensemble Learning Model
표 Performance Comparison (the values in the bold font indicate the best values for respective metrics)
그림 Summary Plot of Selected SHAP Values
표 Performance Comparison on the Dataset Excluding Duration (the values in the bold font indicate the best values for respective metrics)
그림 SHAP Dependent Plot Shows that Effect of Duration on Model Output with Respect to Balance
그림 SHAP Dependent Plot Shows that Effect of Age on Model Output with Respect to Housing
그림 Summary Plot of Selected SHAP Values on the Dataset Excluding Duration
그림 SHAP Decision Plot
그림 SHAP Decision Plot on the Dataset Excluding Duration

참고문헌 (37)

Adadi, A. and Berrada, M., "Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)," IEEE Access, Vol. 6, pp. 52138-52160, 2018.

상세보기
Ahmadi, A., Nabipour, M., MohammadiIvatloo, B., Amani, A. M., Rho, S., and Piran, M. J., "Long-Term Wind Power Forecasting Using Tree-Based Learning Algorithms," IEEE Access, Vol. 8, pp. 151511-151522, 2020.

상세보기
Altman, N. and Krzywinski, M., "Ensemble methods: bagging and random forests," Nature Methods, Vol. 14, No. 10, pp. 933-935, 2017.

상세보기
Belgiu, M. and Dragut, L., "Random forest in remote sensing: A review of applications and future directions," ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 114, pp. 24-31, 2016.

상세보기
Chen, T. and Guestrin, C., "XGBoost: A Scalable Tree Boosting System," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785-794, 2016.
Chun, Y. E., Kim, S. B., Lee, J. Y., and Woo, J. H., "Study on credit rating model using explainable AI," Journal of the Korean Data and Information Science Society, Vol. 32, No. 2, pp. 283-295, 2021.

상세보기
Chun, Y. E., Park, Y., Sung, N., and Choi, J., "Model analysis using estimation of shapley value on classification of sentences explaining causes of changes in stock prices," KIISE Transactions on Computing Practices, Vol. 26, No. 4, pp. 195-201, 2020.

상세보기
Jung, C. and Lee, H., "A comparative study of explainable AI techniques for process analysis," Journal of the Institute of Electronics and Information Engineers, Vol. 57, No. 8, pp. 51-59, 2020.

상세보기
Hung, P. D., Hanh, T. D., and Tung, T. D., "Term deposit subscription prediction using spark MLlib and ML packages," in Proceedings of the 2019 5th International Conference on E-Business and Applications, pp. 88-93, 2019.
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y., "LightGBM: A highly efficient gradient boosting decision tree," in Proceedings of the 31st International Conference on Neural Information Processing Systems, Vol. 30, pp. 3146-3154, 2017.
Kim, S., Kim, W., Jang, Y., and Kim, H., "Development of Explainable AI-Based Learning Support System," The Journal of Korean Association of Computer Education, Vol. 24, No. 1, pp. 107-115, 2021.
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., and Choo, J., "RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records," IEEE Transactions on Visualization and Computer Graphics, Vol. 25, No. 1, pp. 299-309, 2018.

상세보기
Landis, J. R. and Koch, G. G., "An Application of Hierarchical Kappa-type Statistics in the Assessment of Majority Agreement among Multiple Observers," Biometrics, pp. 363-374, 1977.
Lee, D. Y. and Hwang, B. S., "Performance comparison of algorithm for the prediction of time deposit," in Proceedings of the Korea Computer Congress, pp. 2074-2076, 2018.
Lee, Y.-G., Oh, J.-Y., and Kim, G., "Interpretation of load forecasting using explainable artificial intelligence techniques," The Transactions of the Korean Institute of Electrical Engineers, Vol. 69, No. 3, pp. 480-485, 2020.

상세보기
Lim, M. and Jang, H., "A Study on the Risk Reduction Plan of Cryptocurrency Exchange," Journal of Platform Technology, Vol. 8, No. 4, pp. 29-37, 2020.
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., and Lee, S.-I., "From local explanations to global understanding with explainable AI for trees," Nature Machine Intelligence, Vol. 2, No. 1, pp. 56-67, 2020.

상세보기
Mangalathu, S., Hwang, S. H., and Jeon, J. S., "Failure mode and effects analysis of RC members based on machine-learning-based SHapley Additive exPlanations (SHAP) approach," Engineering Structures, Vol. 219, p. 110927, 2020.

상세보기
Moon, J., Jung, S., Rew, J., Rho, S., and Hwang, E., "Combination of short-term load forecasting models based on a stacking ensemble approach," Energy and Buildings, Vol. 216, p. 109921, 2020.

상세보기
Moon, J., Kim, J., Kang, P., and Hwang, E., "Solving the Cold-Start Problem in Short-Term Load Forecasting Using Tree-Based Methods," Energies, Vol. 13, No. 4, p. 886, 2020.

상세보기
Moon, J., Kim, Y., Son, M., and Hwang, E., "Hybrid short-term load forecasting scheme using random forest and multilayer perceptron," Energies, Vol. 11, No. 12, p. 3283, 2018.

상세보기
Moro, S., Cortez, P., and Rita, P., "A data-driven approach to predict the success of bank telemarketing," Decision Support Systems, Vol. 62, pp. 22-31, 2014.

상세보기
Natekin, A. and Knoll, A., "Gradient boosting machines, a tutorial," Frontiers in Neurorobotics, Vol. 7, p. 21, 2013.

상세보기
Oh, H. R., Son, A. L., and Lee, Z., "Occupational accident prediction modeling and analysis using SHAP," Journal of Digital Contents Society, Vol. 22, No. 7, pp. 1115-1123, 2021.

상세보기
Oshiro, T. M., Perez, P. S., and Baranauskas, J. A., "How Many Trees in a Random Forest?," Proceedings of the International Workshop on Machine Learning and Data Mining in Pattern Recognition, pp. 154-168, 2012.
Park, J., Moon, J., Jung, S., and Hwang, E., "Multistep-ahead solar radiation forecasting scheme based on the light gradient boosting machine: A case study of Jeju Island," Remote Sensing, Vol. 12, No. 14, p. 2271, 2020.

상세보기
Park, S. H., Lee, J. H., Jung, Y. W., and Won, Y. J., "Performance comparison of periodic deposit prediction using machine learning," Proceedings of the Korea Software Congress, pp. 2139-2141, 2018.
Park, S., Moon, J., Jung, S., Jung, S., and Hwang, E., "SHAP-based Explainable Influenza Occurrence Forecasting using LightGBM," Proceedings of the Korea Software Congress, pp. 666-668, 2020.
Park, S., Moon, J., and Hwang, E., "Explainable anomaly detection for district heating based on shapley additive explanations," Proceedings of the 2020 International Conference on Data Mining Workshops (ICDMW), pp. 762-765, 2020.
Park, S., Moon, J., Jung, S., Rho, S., and Baik, S. W., Hwang, E., "A two-stage industrial load forecasting scheme for day-ahead combined cooling, heating and power scheduling," Energies, Vol. 13, No. 2, p. 443, 2020.

상세보기
Park, W. and Jang, H., "A study on implementing a priority tasks for invigoration of cloud in financial sector," Journal of Platform Technology, Vol. 8, No. 1, pp. 10-15, 2020.
Parlar, T., "Using Data Mining Techniques for detecting the important features of the bank direct marketing data," International Journal of Economics and Financial Issues, Vol. 7, No. 2, p. 692, 2017.
Rew, J., Cho, Y., Moon, J., and Hwang, E., "Habitat suitability estimation using a two-stage ensemble approach," Remote Sensing, Vol. 12, No. 9, p. 1475, 2020.

상세보기
Rew, J., Kim, H., and Hwang, E., "Hybrid segmentation scheme for skin feature extraction using dermoscopy images," Computers, Materials & Continua, Vol. 69, No. 1, pp. 801-817, 2021.

상세보기
Ribeiro, M. H. D. M., and dos Santos Coelho, L., "Ensemble approach based on bagging, boosting and stacking for short-term prediction in agribusiness time series," Applied Soft Computing, Vol. 86, p. 105837, 2020.

상세보기
Rodriguez, J. D., Perez, A., and Lozano, J. A., "Sensitivity analysis of k-fold cross validation in prediction error estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 3, pp. 569-575, 2009.

상세보기
Sun, J. C. and Kim, I. S., "Improvement of selective consent method in the collection process of personal information of financial institutions," The Journal of Society for e-Business Studies, Vol. 25, No. 1, pp. 123-134, 2020.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증