[논문]임계치 모형과 인공신경망 모형을 이용한 실시간 저수지 수위자료의 이상치 탐지

김마가; 최진용; 방재홍; 이재주

doi:10.5389/ksae.2019.61.1.107

임계치 모형과 인공신경망 모형을 이용한 실시간 저수지 수위자료의 이상치 탐지
Outlier Detection of Real-Time Reservoir Water Level Data Using Threshold Model and Artificial Neural Network Model 원문보기

한국농공학회논문집 = Journal of the Korean Society of Agricultural Engineers, v.61 no.1, 2019년, pp.107 - 120

김마가 (Department of Rural Systems Engineering, Seoul National University) , 최진용 (Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Seoul National University) , 방재홍 (Department of Rural Systems Engineering, Seoul National University) , 이재주 (Rural Research Institute, Korea Rural Community Corporation)

Abstract ▼ AI-Helper

Reservoir water level data identify the current water storage of the reservoir, and they are utilized as primary data for management and research of agricultural water. For the reservoir storage management, Korea Rural Community Corporation (KRC) installed water level stations at around 1,600 agricultural reservoirs and has been collecting the water level data every 10 minutes. However, various kinds of outliers due to noise and erroneous problems are frequently appearing because of environmental and physical causes. Therefore, it is necessary to detect outlier and improve the quality of reservoir water level data to utilize the water level data in purpose. This study was conducted to detect and classify outlier and normal data using two different models including the threshold model and the artificial neural network (ANN) model. The results were compared to evaluate the performance of the models. The threshold model identifies the outlier by setting the upper/lower bound of water level data and variation data and by setting bandwidth of water level data as a threshold of regarding erroneous water level. The ANN model was trained with prepared training dataset as normal data (T) and outlier (F), and the ANN model operated for identifying the outlier. The models are evaluated with reference data which were collected reservoir water level data in daily by KRC. The outlier detection performance of the threshold model was better than the ANN model, but ANN model showed better detection performance for not classifying normal data as outlier.

주제어

표/그림 (15)

그림 Fig. 1 Flow chart of the study
그림 Fig. 2 10-minute interval raw water level data of the Gaeun reservoir
그림 Fig. 3 General structure of multi-layer perceptron
표 Table 1 Properties of the Gaeun reservoir
표 Table 2 Properties of water level data
그림 Fig. 4 T dataset after data pre-process
그림 Fig. 5 The structure of the artificial neural network model for outlier detection
표 Table 3 Input data of the artificial neural network model for outlier detection
그림 Fig. 6 The results of the threshold model according to value of α
그림 Fig. 7 Daily mean value of (a) raw data, (b) threshold data, (c) reference data after threshold model application
표 Table 4 The errors by input data in application of the artificial neural network model for outlier detection
그림 Fig. 8 Daily mean value of (a) raw data, (b) target data, (c) ANN data, (d) reference data after ANN model application
표 Table 5 The statistical parameters (R², MAE, RMSE) compared to reference data
그림 Fig. 9 The scatter plot of (a) raw data, (b) threshold data, (c) target data, (d) ANN data compared with reference data
그림 Fig. 10 Daily mean value of (a) raw data, (b) threshold data, (c) ANN data, (d) reference data (2016. 01. 01.~2016. 12. 31.)

질의응답

핵심어	질문	논문에서 추출한 답변
	인공신경망 모형은 어떻게 구성되나요?	인공신경망 모형은 뉴런과 시냅스로 이루어져 있는 인간의 뇌 구조를 단순화하여 만든 연산모델로 단순한 연산자들의 결합으로 구성되어 있다. 인공신경망 모형은 문제를 해결하기 위한 직접적인 지식이나 방법을 설정하지 않아도 주어진 자료를 통해 문제를 해결할 수 있으며 (Yeo et al.
	수문 자료가 무엇인가요?	수문 자료는 수자원을 이용하고 관리하기 위해 필요한 분석의 중요한 기초자료이며, 댐의 운영이나 홍수조절, 관개용수 관리 등 다양한 분야에 밀접하게 관련되어 있다. 특히 저수지 수위자료는 저수지 용량 곡선을 이용하여 저수용량을 산정하는 기준으로 사용되고 있으며 (Jeong and Kim, 2007), 저수지 운영 모형의 개발 (Shim et al.
	저수지 수위계측 방식 두 가지는 무엇이며, 문제점은 무엇인가요?	한국에서는 10만 톤이상의 저수용량을 가진 농업용 저수지에 대해 전국 약 1,600여개의 저수지 수위계측 장비가 설치되어 있다. 수위계측은 10분 단위 간격으로 이루어지며, 압력식 센서와 초음파식 센서를 사용하는 두 가지 방식이 있다. 그러나 압력식 센서의 경우 사통 내부로 토사물이 유입되거나 센서 부근에 퇴적된 토사 등으로 인해 이상치가 발생할 수 있으며, 초음파 센서의 경우 기온, 습도 등 환경 변화나 파랑, 센서 부근의 수초 등 장애물과 같은 물리적인 이유로 인한 이상치가 발생할 수 있다 (Bang et al., 2017).

참고문헌 (34)

Ahn, J. H., M. S. Kang, I. H. Song, K. D. Lee, J. H. Song, and J. R. Jang, 2012. Estimation of surface runoff from paddy plots using an artificial neural network. Journal of the Korean Society of Agricultural Engineers 54(4): 66-71 (in Korean). doi:10.5389/KSAE.2012.54.4.065.
Ahn, S. J., I. S. Yeon, and K. I. Kim, 2000a. Rainfall forecasting Using neural network. Journal of The Korean Society of Civil Engineers 20(5B): 711-722 (in Korean).
Ahn, S. J., K. W. Jun, and K. I. Kim, 2000b. Forecasting of runoff hydrograph using neural network algorithms. Journal of Korean Water Resources Association 33(4): 505-515 (in Korean).
Bang, J. H., Y. H. Lee, S. Y. Jeong, and J. Y. Choi, 2017. A study of outlier detection on time series of water-level in agricultural reservoir. In Proceedings of the Korean Society of Agricultural Engimeers Conefrence 60 (in Korean).
Choi, J. K., and M. S. Kang, 2000. Application of neural network to water resources. Journal of Korean National Comittee on Irrigation and Drainage 7(2): 248-258 (in Korean).
Choi, J. Y., 2018. Development of quality control methods for water level data and irrigation water supply estimation, 4-10. Korea Rural Community Corporation.
Feng, S., Q. Hu, and W. Qian, 2004. Quality control of daily meteorological data in China, 195-2000: a new dataset. International Journal of Climatology 24: 853-870. doi:10.1002/joc.1047.

상세보기
Gonzalez-Rouco, J. F., J. L. Jimenez, V. Quesada, and F. Valero, 1999. Quality control and homogeneity of precipitation data in the southwest of Europe. Journal of Climate 14: 964-978. doi:10.1175/1520-0442(2001)014 2.0.CO;2.
Gunther, F., and S. Fritsch, 2010. Neuralnet: training of neural networks. The R Journal 2(1): 30-38.

상세보기
Harrison, D. L., S. J. Driscoll, and M. Kitchen, 2000. Improving precipitation estimates from weather radar using quality control and correction techniques. Meteorological Applications 7(2): 135-144. doi:10.1017/S1350482700001468.

상세보기
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment, amd I. Zaslavsky, 2008. A relational model for environmental and water resources data. Water Resources Research 44(5). doi:10.1029/2007WR006392.
Jang, S. H., J. Y. Yoon, S. D. Kim, and Y. N. Yoon, 2007. An establishment of operation and management system for flood control and conservation in reservoir with date: I. establishment of real-time inflow prediction model using recorded rainfall data. Journal of The Korean Society of Civil Engineers 27(2B): 133-140 (in Korean).
Jeong, G. H., and T. W. Kim, 2007. Comparing water distribution model with reservoir and water system management. Water and Future 40(10): 38-43 (in Korean).
Kang, B. S., and B. K. Lee, 2008. Predicting probability of precipitation using artificial neural network and mesoscale numerical weather prediction. Journal of The Korean Society of Civil Engineers 28(5B): 485-493 (in Korean).
Kang, M. G., H. S. Jeong, and J. T. Kim, 2010. Efficient management of agricultural canal systems through quality management of water level and water quantity data. Rural Resources 52(2): 87-96 (in Korean).
Kilonsky, B. J., and P. Caldwell, 1991. In the pursuit of high-quality sea level data. In Oceans 91 Proceedings (2): 669-675. doi:10.1109/OCEANS.1991.627921.
Kim, C. H., J. A. Ryu, D. G. Kim, and G. B. Kim, 2016. Analysis of the effects of drainage systems in wetlands based on changes in groundwater level, soil moisture content, and water quality. The Journal of Engineering Geology 26(2): 251-260 (in Korean). doi:/10.9720/kseg.2016.2.251.

원문보기 상세보기
Kim, H. G., M. I. Kim, M. S. Lee, Y. S. Park, and J. H. Kwak, 2017. Correlation of deep landside occurrence and variation of groundwater level. Journal of The Korea Society of Forest Engineering and technology 15(1): 1-12 (in Korean).
Kim, H. K., S. M. Kim, and S. W. Park, 2006. Development of hydrologic data management system based on relational database. Journal of Korean Water Resources Association 39(10): 855-866 (in Korean).

원문보기 상세보기
Kim, S. J., Y. S. Kwon, K. H. Lee, and H. S. Kim, 2010. Radar rainfall adjustment by artificial neural network and runoff analysis. Journal of The Korean Society of Civil Engineers 30(2B): 159-167 (in Korean).
Kim, S. W., 2000. A study on the forecasting of daily streamflow using the multilayer neural networks model. Journal of Korean Water Resources Association 33(5): 537-550 (in Korean).
Kim, S. W., and J. D. Salas, 2000. The flood water stage prediction based on neural networks method in stream gauge station. Journal of Korean Water Resources Association 33(2): 247-262 (in Korean).
Lee, C. W., Y. S. Mang, and Y. S. Kim, 2014. Behavior of fill dam subjected to continuous water level change and overflow. Journal of the Korean Geo-Environmental Society 15(6): 41-48 (in Korean). doi:10.14481/jkges.2014.15.6.41.
Mounce, S. R., R. B. Mounce, and J. B. Boxall, 2011. Novelty detection for time series data analysis in water distribution systems using support vector machines. Journal of Hydroinformatics 13(4): 672-686. doi:10.2166/hydro.2010.144.

상세보기
Mourad, M., and J. L. Bertrand-Krajewski, 2002. A method for automatic validation of long time series of data in urban hydrology. Water Science & Technology 45(4-5): 263-270. doi:10.2166/wst.2002.0601.
Oh, C. R., S. C. Park, H. M. Lee, and Y. P. Pyo, 2002. A forecasting of water quality in the Youngsan river using neural network. Journal of The Korean Society of Civil Engineers 22(3B): 372-382 (in Korean).
Oh, J. W., J. H. Park, and Y. K. Kim, 2008. Missing hydrological data estimation using neural network and real time data reconciliation. Journal of Korean Water Resources Association (10): 1059-1065 (in Korean). doi:10.3741/JKWRA.2008.41.10.1059.
Park, J. H., M. S. Kang, J. H. Song, and S. M. Jun, 2015. Design and implementation of IoT-based intelligent platform for water level monitoring. Journal of the Korean Society of Rural Planning 21(4): 177-186 (in Korean). doi:10.7851/ksrp.2015.21.4.177.
Schneider, U., A. Becker, P. Finger, A. Meyer-Christoffer, M. Ziese, and B. Rudolf, 2014. GPCC's new land surface precipitation climatology based on quality-controlled in situ data its role in quantifying the global water cycle. Theoretical and Applied Climatology 115(1-2): 15-40. doi:10.1007/s00704-013-0860-x.

상세보기
Seo, Y. M., E. H. Choi, and W. K. Yeo, 2017. Reservoir water level forecasting using machine learning models. Journal of the Korean Society of Agricultural Engineers 59(3): 97-110 (in Korean). doi:10.5389/KSAE.2017.59.3.097.
Shim, S. B., S. K. Kim, R. H. Park, and D. K. Hoh, 1997. Decision support system for real-time reservoir operation during flood period. Journal of Korean Water Resources Association 30(5): 431-439 (in Korean).
Steiner, M., J. A. Smith, S. J. Burges, C. V. Alonso, and R. W. Darden, 1999. Effect of bias adjustment and rain gauge data quality control on radar rainfall estimation. Water Resources Research 35(8): 2487-2503. doi:10.1029/1999WR900142.

상세보기
Yeo, W. K., Y. M. Seo, S. Y. Lee, and H. K. Ji, 2010. Study on water stage prediction using hybrid model of artificial neural network and genetic algorithm. Journal of Korean Water Resources Association 43(8): 721-731 (in Korean). doi:10.3741/JKWRA.2010.43.8.721.

원문보기 상세보기
Yoon, K. H., B. C. Seo, and H. S. Shin, 2004. Dam inflow forecasting for short term flood based on neural networks in Nakdong river basin. Journal of Korean Water Resources Association 37(1): 67-75 (in Korean).

원문보기 상세보기

저자의 다른 논문 :

LOADING...

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증