[논문]제주도 표선유역 중산간지역의 최적 지하수위 예측을 위한 인공신경망의 활성화함수 비교분석

신문주; 김진우; 문덕철; 이정한; 강경구

doi:10.3741/jkwra.2021.54.s-1.1143

제주도 표선유역 중산간지역의 최적 지하수위 예측을 위한 인공신경망의 활성화함수 비교분석
Comparative analysis of activation functions of artificial neural network for prediction of optimal groundwater level in the middle mountainous area of Pyoseon watershed in Jeju Island 원문보기

Journal of Korea Water Resources Association = 한국수자원학회논문집, v.54 no.12 suppl., 2021년, pp.1143 - 1154

신문주 (제주특별자치도개발공사 수자원연구팀) , 김진우 (제주특별자치도개발공사 수자원연구팀) , 문덕철 (제주특별자치도개발공사 수자원연구팀) , 이정한 (제주특별자치도개발공사 수자원연구팀) , 강경구 (제주특별자치도개발공사 R&D 혁신센터)

초록
AI-Helper

활성화함수의 선택은 인공신경망(Artificial Neural Network, ANN) 모델의 지하수위 예측성능에 큰 영향을 미친다. 특히 제주도의 중산간 지역과 같이 지하수위의 변동폭이 크고 변동양상이 복잡한 경우 적절한 지하수위 예측을 위해서는 다양한 활성화함수의 비교분석을 통한 최적의 활성화함수 선택이 반드시 필요하다. 본 연구에서는 지하수위의 변동폭이 크고 변동양상이 복잡한 제주도 표선유역 중산간지역 2개 지하수위 관측정을 대상으로 5개의 활성화함수(sigmoid, hyperbolic tangent (tanh), Rectified Linear Unit (ReLU), Leaky Rectified Linear Unit (Leaky ReLU), Exponential Linear Unit (ELU))를 ANN 모델에 적용하여 지하수위 예측결과를 비교 및 분석하고 최적 활성화함수를 도출하였다. 그리고 최근 널리 사용되고 있는 순환신경망 모델인 Long Short-Term Memory (LSTM) 모델의 결과와 비교분석하였다. 분석결과 지하수위 변동폭이 상대적으로 큰 관측정과 상대적으로 작은 관측정에 대한 지하수위 예측에 대해서는 각각 ELU와 Leaky ReLU 함수가 최적의 활성화함수로 도출되었다. 반면 sigmoid 함수는 학습기간에 대해 5개 활성화함수 중 예측성능이 가장 낮았으며 첨두 및 최저 지하수위 예측에서 적절하지 못한 결과를 도출하였다. 따라서 ANN-sigmoid 모델은 가뭄기간의 지하수위 예측을 통한 지하수자원 관리목적으로 사용할 경우 주의가 필요하다. ANN-ELU와 ANN-Leaky ReLU 모델은 LSTM 모델과 대등한 지하수위 예측성능을 보여 활용가능성이 충분히 있으며 LSTM 모델은 ANN 모델들 보다 예측성능이 높아 인공지능 모델의 예측성능 비교분석 시 참고 모델로 활용될 수 있다. 마지막으로 학습기간의 정보량에 따라 학습기간의 지하수위 예측성능이 검증 및 테스트 기간의 예측성능보다 낮을 수 있다는 것을 확인하였으며, 관측지하수위의 변동폭이 크고 변동양상이 복잡할수록 인공지능 모델별 지하수위 예측능력의 차이는 커졌다. 본 연구에서 제시한 5개의 활성화함수를 적용한 연구방법 및 비교분석 결과는 지하수위 예측뿐만 아니라 일단위 하천유출량 및 시간단위 홍수량 등 지표수 예측을 포함한 다양한 연구에 유용하게 사용될 수 있다.

Abstract ▼ AI-Helper

The selection of activation function has a great influence on the groundwater level prediction performance of artificial neural network (ANN) model. In this study, five activation functions were applied to ANN model for two groundwater level observation wells in the middle mountainous area of the Pyoseon watershed in Jeju Island. The results of the prediction of the groundwater level were compared and analyzed, and the optimal activation function was derived. In addition, the results of LSTM model, which is a widely used recurrent neural network model, were compared and analyzed with the results of the ANN models with each activation function. As a result, ELU and Leaky ReLU functions were derived as the optimal activation functions for the prediction of the groundwater level for observation well with relatively large fluctuations in groundwater level and for observation well with relatively small fluctuations, respectively. On the other hand, sigmoid function had the lowest predictive performance among the five activation functions for training period, and produced inappropriate results in peak and lowest groundwater level prediction. The ANN-ELU and ANN-Leaky ReLU models showed groundwater level prediction performance comparable to that of the LSTM model, and thus had sufficient potential for application. The methods and results of this study can be usefully used in other studies.

주제어

표/그림 (16)

표 Table 1. Groundwater level analysis study applying various activation functions to ANN model
그림 Fig. 1. The structure of ANN model
그림 Fig. 2. The structure of LSTM model. Reproduced from Chollet and Allaire (2018)
그림 Fig. 3. Groundwater level modeling process of artificial intelligence model. Reproduced from Chollet and Allaire (2018)
그림 Fig. 4. Graph of activation functions: (a) sigmoid function, (b) differentiation of sigmoid function, (c) tanh function, (d) differentiation of tanh function, (e) ReLU function, (f) differentiation of ReLU function, (g) Leaky ReLU function (α = 0.2), (h) differentiation of Leaky ReLU function (α = 0.2), (i) ELU function (α = 0.2), (j) differentiation of ELU function (α = 0.2)
표 Table 2. Equations of activation function and differentiation of the activation function
표 Table 3. Data period of rainfall station, groundwater withdrawal well and groundwater monitoring well
그림 Fig. 5. Schematic diagram of the location of rainfall stations, groundwater withdrawal wells and groundwater monitoring wells
그림 Fig. 6. Comparison of precipitation data from Seongpanak and Gyorae rainfall stations
그림 Fig. 7. Variation range of the groundwater level in the groundwater monitoring wells
표 Table 4. Duration of training, validation and testing of artificial intelligence models^a
표 Table 5. Hyper-parameters of artificial intelligence models
표 Table 6. Results of activation functions for groundwater monitoring well A
표 Table 7. Results of activation functions for groundwater monitoring well B
그림 Fig. 8. Comparison of observed groundwater level and simulated groundwater level of ANN-ELU and ANN-sigmoid for the training period of monitoring well A
그림 Fig. 9. Comparison of observed groundwater level and simulated groundwater level of ANN-leaky ReLU and ANN-sigmoid for the training period of monitoring well B

참고문헌 (54)

Afzaal, H., Farooque, A.A., Abbas, F., Acharya, B., and Esau, T. (2020). "Groundwater estimation from major physical hydrology components using artificial neural networks and deep learning." Water, Vol. 12, No. 1, p. 5.

상세보기
Barthel, R., and Banzhaf, S. (2016). "Groundwater and surface water interaction at the regional-scale - a review with focus on regional integrated models." Water Resources Management, Vol. 30, No. 1, pp. 1-32.

상세보기
Bengio, Y., Simard, P., and Frasconi, P. (1994). "Learning long-term dependencies with gradient descent is difficult." IEEE Transactions on Neural Networks, Vol. 5, No. 2, pp. 157-166.

상세보기
Chang, J., Wang, G., and Mao, T. (2015). "Simulation and prediction of suprapermafrost groundwater level variation in response to climate change using a neural network model." Journal of Hydrology, Vol. 529, pp. 1211-1220.

상세보기
Chollet, F. Allaire, J.J. (2018). Deep learning with R, Manning Publications, Shelter Island, NY, U.S., p. 360.
Clevert, D.A., Unterthiner, T., and Hochreiter, S. (2016). "Fast and accurate deep network learning by exponential linear units (ELUs)." arXiv preprint arXiv:1511.07289.
Coulibaly, P., Anctil, F., Aravena, R., and Bobee, B. (2001). "Artificial neural network modeling of water table depth fluctuations." Water Resources Research, Vol. 37, No. 4, pp. 885-896.

상세보기
Emamgholizadeh, S., Moslemi, K., and Karami, G. (2014). "Prediction the groundwater level of bastam plain (Iran) by artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS)." Water Resources Management, Vol. 28, No. 15, pp. 5433-5446.

상세보기
Falbel, D., Allaire, J.J., Chollet, F., Tang, Y., Van Der Bijl, W., Studer, M., Keydana, S. (2019). R interface to 'Keras'. R package version 2.2.4.1, accessed on 5 April 2019, .
Hahnloser, R.H., Sarpeshkar, R., Mahowald, M.A., Douglas, R.J., and Seung, H.S. (2000). "Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit." Nature, Vol. 405, No. 6789, pp. 947-951.

상세보기
Haykin, S. (2009). Neural networks and learning machines, Pearson Prentice Hall, Upper Saddle River, NJ, U.S.
Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780.

상세보기
Hosseini, Z., Gharechelou, S., Nakhaei, M., and Gharechelou, S. (2016). "Optimal design of BP algorithm by ACOR model for groundwater-level forecasting: A case study on Shabestar plain, Iran." Arabian Journal of Geosciences, Vol. 9, No. 6, p. 436.

상세보기
Jeju Special Self-Governing Province (JSSGP) (2018). Comprehensive water resources management plan in Jeju Island. pp. 1-328.
Jeong, J., and Park, E. (2019). "Comparative applications of data-driven models representing water table fluctuations." Journal of Hydrology, Vol. 572, pp. 261-273.

상세보기
Jha, M.K., and Sahoo, S. (2014). "Efficacy of neural network and genetic algorithm techniques in simulating spatio-temporal fluctuations of groundwater." Hydrological Processes, Vol. 29, No. 5, pp. 671-691.

상세보기
Kim, G.B., and Oh, D.H. (2018). "Determination of the groundwater yield of horizontal wells using an artificial neural network model incorporating riverside groundwater level data." The Journal of Engineering Geology, Vol. 28, No. 4, pp. 583-592.

원문보기 상세보기
Kim, J., Jun, S.M., Hwang, S., Kim, H.K., Heo, J., and Kang, M.S. (2021a). "Impact of activation functions on flood forecasting model based on artificial neural networks." Journal of The Korean Society of Agricultural Engineers, Vol. 63, No. 1, pp. 11-25.

원문보기 상세보기
Kim, M., Choi, J.Y., Bang, J., Yoon, P., and Kim, K. (2021b). "Comparison of artificial neural network model capability for runoff estimation about activation functions." Journal of The Korean Society of Agricultural Engineers, Vol. 63, No. 1, pp. 103-116.

원문보기 상세보기
Kim, T.W., and Valdes, J.B. (2003). "Nonlinear model for drought forecasting based on a conjunction of wavelet transforms and neural networks." Journal of Hydrologic Engineering, Vol. 8, No. 6, pp. 319-328.

상세보기
Kingma, D.P., and Ba, J. (2014). "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980.
Klemes, V. (1986). "Operational testing of hydrological simulation models." Hydrological Sciences Journal, Vol. 31, No. 1, pp. 13-24.

상세보기
Krishna, B., Satyaji Rao, Y.R., and Vijaya, T. (2008). "Modelling groundwater levels in an urban coastal aquifer using artificial neural networks." Hydrological Processes, Vol. 22, No. 8, pp. 1180-1188.

상세보기
Le, X.H., Ho, H.V., Lee, G., and Jung, S. (2019). "Application of long short-term memory (LSTM) neural network for flood forecasting." Water, Vol. 11, No. 7, p. 1387.

상세보기
Lee, S., Lee, K.K., and Yoon, H. (2019). "Using artificial neural network models for groundwater level forecasting and assessment of the relative impacts of influencing factors." Hydrogeology Journal, Vol. 27, No. 2, pp. 567-579.

상세보기
Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013). "Rectifier nonlinearities improve neural network acoustic models." Proceedings of the 30 th International Conference on Machine Learning, Atlanta, GA, U.S., Vol. 30, No. 1, p. 3.
Maier, H.R., and Dandy, G.C. (2000). "Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications." Environmental Modelling & Software, Vol. 15, No. 1, pp. 101-124.

상세보기
Maxwell, R.M., Condon, L.E., and Kollet, S.J. (2015). "A high-resolution simulation of groundwater and surface water over most of the continental US with the integrated hydrologic model ParFlow v3." Geoscientific Model Development, Vol. 8, No. 3, pp. 923-937.

상세보기
McDonald, M.G., and Harbaugh, A.W. (1988). A modular three-dimensional finite-difference ground-water flow model. Vol. 6. US Geological Survey, Reston, VA, U.S.
Mohanty, S., Jha, M.K., Kumar, A., and Panda, D.K. (2013). "Comparative evaluation of numerical model and artificial neural network for simulating groundwater flow in Kathajodi - Surua Inter-basin of Odisha, India." Journal of Hydrology, Vol. 495, pp. 38-51.

상세보기
Moriasi, D.N., Arnold, J.G., Van Liew, M.W., Bingner, R.L., Harmel, R.D., and Veith, T.L. (2007). "Model evaluation guidelines for systematic quantification of accuracy in watershed simulations." Transactions of the ASABE, Vol. 50, No. 3, pp. 885-900.

상세보기
Muller, J., Park, J., Sahu, R., Varadharajan, C., Arora, B., Faybishenko, B., and Agarwal, D. (2021). "Surrogate optimization of deep neural networks for groundwater predictions." Journal of Global Optimization, Vol. 81, No. 1, pp. 203-231.

상세보기
Nash, J.E., and Sutcliffe, J.V. (1970). "River flow forecasting through conceptual models part I - A discussion of principles." Journal of Hydrology, Vol. 10, No. 3, pp. 282-290.

상세보기
Nayak, P.C., Rao, Y.S., and Sudheer, K.P. (2006). "Groundwater level forecasting in a shallow aquifer using artificial neural network approach." Water Resources Management, Vol. 20, No. 1, pp. 77-90.

상세보기
Prechelt, L. (2012) "Early stopping - But when?." Neural networks: Tricks of the trade, Edited by Montavon G., Orr G.B., and Muller KR., Springer, Berlin Heidelberg, pp. 53-67.
Rajaee, T., Ebrahimi, H., and Nourani, V. (2019). "A review of the artificial intelligence methods in groundwater level modeling." Journal of Hydrology, Vol. 572, pp. 336-351.

상세보기
Rakhshandehroo, G.R., Vaghefi, M., and Aghbolaghi, M.A. (2012). "Forecasting groundwater level in Shiraz plain using artificial neural networks." Arabian Journal for Science and Engineering, Vol. 37, No. 7, pp. 1871-1883.

상세보기
Sahoo, S., and Jha, M.K. (2013). "Groundwater-level prediction using multiple linear regression and artificial neural network techniques: a comparative assessment." Hydrogeology Journal, Vol. 21, No. 8, pp. 1865-1887.

상세보기
Sahoo, S., Russo, T.A., Elliott, J., and Foster, I. (2017). "Machine learning algorithms for modeling groundwater level changes in agricultural regions of the US." Water Resources Research, Vol. 53, No. 5, pp. 3878-3895.

상세보기
Shin, M.J., Moon, S.H., Kang, K.G., Moon, D.C., and Koh, H.J. (2020). "Analysis of groundwater level variations caused by the changes in groundwater withdrawals using long short-term memory network." Hydrology, Vol. 7, No. 3, p. 64.

상세보기
Sit, M., Demiray, B.Z., Xiang, Z., Ewing, G.J., Sermet, Y., and Demir, I. (2020). "A comprehensive review of deep learning applications in hydrology and water resources." Water Science and Technology, Vol. 82, No. 12, pp. 2635-2670.

상세보기
Sun, Y., Wendi, D., Kim, D.E., and Liong, S.Y. (2016). "Application of artificial neural networks in groundwater table forecasting-a case study in a Singapore swamp forest." Hydrology and Earth System Sciences, Vol. 20, No. 4. pp. 1405-1412.

상세보기
Taormina, R., Chau, K.W., and Sethi, R. (2012). "Artificial neural network simulation of hourly groundwater levels in a coastal aquifer system of the Venice lagoon." Engineering Applications of Artificial Intelligence, Vol. 25, No. 8, pp. 1670-1676.

상세보기
Todd, D.K. and Larry, W.M. (2004). Groundwater hydrology, Third edition. John Wiley & Sons Inc., Hoboken, NJ, U.S., pp. 1-656.
Ukkonen, P., and Makela, A. (2019). "Evaluation of machine learning classifiers for predicting deep convection." Journal of Advances in Modeling Earth Systems, Vol. 11, No. 6, pp. 1784-1802.

상세보기
Vu, M.T., Jardani, A., Massei, N., and Fournier, M. (2021). "Reconstruction of missing groundwater level data by using Long Short-Term Memory (LSTM) deep neural network." Journal of Hydrology, Vol. 597, p. 125776.

상세보기
Wen, X., Feng, Q., Deo, R.C., Wu, M., and Si, J. (2017). "Wavelet analysis - artificial neural network conjunction models for multi-scale monthly groundwater level predicting in an arid inland river basin, northwestern China." Hydrology Research, Vol. 48, No. 6, pp. 1710-1729.

상세보기
White, J.T., Doherty, J.E., and Hughes, J.D. (2014). "Quantifying the predictive consequences of model error with linear subspace analysis." Water Resources Research, Vol. 50, No. 2, pp. 1152-1173.

상세보기
White, J.T., Knowling, M.J., and Moore, C.R. (2020). "Consequences of groundwater-Model vertical discretization in risk-Based decision-Making." Groundwater, Vol. 58, No. 5, pp. 695-709.
Worland, S.C., Steinschneider, S., Asquith, W., Knight, R., and Wieczorek, M. (2019). "Prediction and inference of flow duration curves using multioutput neural networks." Water Resources Research, Vol. 55, No. 8, pp. 6850-6868.

상세보기
Xu, B., Wang, N., Chen, T., and Li, M. (2015). "Empirical evaluation of rectified activations in convolutional network." arXiv preprint arXiv:1505.00853.
Yoon, H., Hyun, Y., Ha, K., Lee, K.K., and Kim, G.B. (2016). "A method to improve the stability and accuracy of ANN- and SVM-based time series models for long-term groundwater level predictions." Computers & Geosciences, Vol. 90, pp. 144-155.

상세보기
Yoon, H., Jun, S.C., Hyun, Y., Bae, G.O., and Lee, K.K. (2011). "A comparative study of artificial neural networks and support vector machines for predicting groundwater levels in a coastal aquifer." Journal of Hydrology, Vol. 396, No. 1-2, pp. 128-138.

상세보기
Yu, H., Wen, X., Feng, Q., Deo, R.C., Si, J., and Wu, M. (2018). "Comparative study of hybrid-wavelet artificial intelligence models for monthly groundwater depth forecasting in extreme arid regions, Northwest China." Water Resources Management, Vol. 32, No. 1, pp. 301-323.

상세보기

저자의 다른 논문 :

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format