[논문]홍수량 예측 인공신경망 모형의 활성화 함수에 따른 영향 분석

김지혜; 전상민; 황순호; 김학관; 허재민; 강문성

doi:10.5389/ksae.2021.63.1.011

[국내논문] 홍수량 예측 인공신경망 모형의 활성화 함수에 따른 영향 분석
Impact of Activation Functions on Flood Forecasting Model Based on Artificial Neural Networks 원문보기

한국농공학회논문집 = Journal of the Korean Society of Agricultural Engineers, v.63 no.1, 2021년, pp.11 - 25

김지혜 (Department of Rural Systems Engineering, Seoul National University) , 전상민 (Department of Rural Systems Engineering, Seoul National University) , 황순호 (Research Institute of Agriculture and Life Sciences, Seoul National University) , 김학관 (Graduate School of International Agricultural Technology, Institutes of Green Bio Science and Technology, Seoul National University) , 허재민 (Department of Rural Systems Engineering, Seoul National University) , 강문성 (Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Institutes of Green Bio Science and Technology, Seoul National University)

Abstract ▼ AI-Helper

The objective of this study was to analyze the impact of activation functions on flood forecasting model based on Artificial neural networks (ANNs). The traditional activation functions, the sigmoid and tanh functions, were compared with the functions which have been recently recommended for deep neural networks; the ReLU, leaky ReLU, and ELU functions. The flood forecasting model based on ANNs was designed to predict real-time runoff for 1 to 6-h lead time using the rainfall and runoff data of the past nine hours. The statistical measures such as R2, Nash-Sutcliffe Efficiency (NSE), Root Mean Squared Error (RMSE), the error of peak time (ETp), and the error of peak discharge (EQp) were used to evaluate the model accuracy. The tanh and ELU functions were most accurate with R2=0.97 and RMSE=30.1 (㎥/s) for 1-h lead time and R2=0.56 and RMSE=124.6~124.8 (㎥/s) for 6-h lead time. We also evaluated the learning speed by using the number of epochs that minimizes errors. The sigmoid function had the slowest learning speed due to the 'vanishing gradient problem' and the limited direction of weight update. The learning speed of the ELU function was 1.2 times faster than the tanh function. As a result, the ELU function most effectively improved the accuracy and speed of the ANNs model, so it was determined to be the best activation function for ANNs-based flood forecasting.

Keyword

표/그림 (13)

그림 Fig. 1 Structure of artificial neural networks
그림 Fig. 2 Summary of the activation functions
그림 Fig. 3 Structure of the flood forecasting model based on Artificial neural networks
그림 Fig. 4 Location of Heukcheon basin and the weather and gauging stations
표 Table 1 Performance evaluation criteria for watershed-scale modeling (Moriasi et al., 2015)*
그림 Fig. 5 Training and test data set for k-fold cross validation (ex. k=4) (Chollet, 2017)
그림 Fig. 6 Statistical values for training and test data set (minimum, average, and maximum values for 1 to 5 hidden layers)
그림 Fig. 7 ETp and EQp for training and test data set (minimum, average, and maximum values for 1 to 5 hidden layers)
그림 Fig. 8 Observed and simulated hydrographs of the ANNs model (5 hidden layers) with sigmoid and ELU activation functions at the test stage
그림 Fig. 9 Relative learning speed depending on the activation functions
표 Table 2 Number of epochs to converge depending on the activation functions
그림 Fig. 10 Changes of the loss function and coefficient of determination (R²) along the training epoch of ANNs model (3 hidden layers)
그림 Fig. 11 Comparison of the amount of weight updates at every training step (7^th node of 2^nd hidden layer of ANNs model with 3 hidden layers)

참고문헌 (28)

Adamowski, J., and S. O. Prasher, 2012. Comparison of machine learning methods for runoff forecasting in mountainous watersheds with limited data. Journal of Water and Land Development 17(1): 89-97. doi:10.2478/v10025-012-0038-4.

상세보기
Aggarwal, C. C., 2018. Neural networks and deep learning: Springer.
Chollet, F., 2015. Keras. https://keras.io. Accessed 20Jul. 2020.
Chollet, F., 2017. Deep Learning with Python: Manning.
Clevert, D. A., T. Unterthiner, and S. Hochreiter, 2016. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv preprint. arXiv:1511.07289.
Dawson, C. W., and R. Wilby, 1998. An artificial neural network approach to rainfall-runoff modeling. Hydrological Sciences 43(1): 47-66. doi:10.1080/02626669809492102.

상세보기
Donate, J. P., C. Paulo, G. S. German, and S. M. Araceli, 2013. Time series forecasting using a weighted crossvalidation evolutionary artificial neural network ensemble. Neurocompution 109: 27-32. doi:10.1016/j.neucom.2012.02.053.

상세보기
Glorot, X., and Y. Bengio, 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. Thirteenth International Conference on Artificial Intelligence and Statistics: 249-256.
Goodfellow, I., Y. Bengio, and A. Courville, 2016. Deep Learning: MIT press.
Hsu, K. N., H. V. Gupta, and S. sorooshian, 1995. Artificial neural network modeling of the rainfall-runoff process. Water Resources Research 31(10): 2517-2530. doi:10.1029/95WR01955.

상세보기
Hu, C., Q. Wu, H. Li, S. Jian, N. Li, and Z. Lou, 2018. Deep learning with a long short-term memory networks approach for rainfall-runoff simulation. Water 10(11): 1543. doi:10.3390/w10111543.

상세보기
Jeung, M., S. Baek, J. Beom, K. H. Cho, Y. Her, and K. Yoon, 2019. Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments. Journal of Hydrology 575: 1099-1110. doi:10.1016/j.jhydrol.2019.05.079.

상세보기
Kang, M. S., and S. W. Park, 2003, Short-term flood forecasting using artificial neural networks. Journal of The Korean Society of Agricultural Engineers 45(2): 45-57 (in Korean).
Kang, M. S., M. G. Kang, S. W. Park, J. J. Lee, and R. H. Yoo, 2006. Application of grey model and artificial neural networks to flood forecasting. Journal of the American Water Resources Association 42(2): 473-486.

상세보기
Kim, J. H., 1993. A study on hydrologic forecasting of stream flows based on artificial neural network. Ph.D. diss., Incheon, Republic of Korea: Inha University (in Korean).
Kingma, D. P., and J. Ba, 2014. Adam: A method for stochastic optimization. arXiv preprint. arXiv:1412.6980.
Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2017. Imagenet classification with deep convolutional neural networks. Communications of the ACM 60(6): 84-90. doi:10.1145/3065386.

상세보기
Lau, M. M., and K. H. Lim, 2017. Investigation of activation functions in deep belief network. 2017 2nd International Conference on Control and Robotics Engineering (ICCRE): 201-206. doi:10.1109/ICCRE.2017.7935070.
Lee, G. S., S. C. Park, H. M. Lee, and Y. H. Jin, 2000. The prediction of runoff using artificial neural network in the Young-san River. Journal of Korea Water Resources Association 33(1): 251-256 (in Korean).
Marina, C., P. Andreussi, and A. Soldati, 1999. River flood forecasting with a neural network model. Water Resources Research 35(4): 1191-1197. doi:10.1029/1998WR900086.

상세보기
Ministry of the Interior and Safety, 2018. Statistical yearbook of natural disaster (in Korean).
Moriasi, D. N., M. W. Gitau, N. Pai, and P. Daggupati, 2015. Hydrologic and water quality models: performance measures and evaluation criteria. Transactions of the ASABE 58(6): 1763-1785. doi:10.13031/trans.58.10715.

상세보기
Sarkar, A., and R. Kumar, 2012. Artificial neural networks for event based rainfall-runoff modeling. Journal of Water Resource and Protection 4(10): 891. doi:10.4236/jwarp.2012.410105.

상세보기
Solomatine, D., L. M. See, and R. J. Abrahart, 2009. Data-driven modelling: concepts, approaches and experiences. In Practical Hydroinformatics: 17-30.
Song, J. H., Y. Her, K. Suh, M. S. Kang, and H. Kim, 2019. Regionalization of a Rainfall-Runoff Model: Limitations and Potentials. Water 11(11): 2257. doi:10.3390/w11112257.

상세보기
Xu, B., N. Wang, T. Chen, and M. Li, 2015. Empirical evaluation of rectified activations in convolutional network. arXiv preprint. arXiv:1505.00853.
Young, C. C., W. C. Liu, and M. C. Wu, 2017. A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events. Applied Soft Computing 53: 205-216. doi:10.1016/j.asoc.2016.12.052.

상세보기
Zealnad, C. M., D. H. Burn, and S. P. Simonovic, 1999. Short term stream flow forecasting using artificial neural networks. Journal of Hydrology 214: 32-48. doi:10.1016/S0022-1694(98)00242-X.

상세보기

저자의 다른 논문 :

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증