[논문]비주석 재귀신경망 앙상블 모델을 기반으로 한 조위관측소 해수위의 준실시간 이상값 탐지

이은주; 김영택; 김송학; 주호정; 박재훈

doi:10.7850/jkso.2021.26.4.307

[국내논문] 비주석 재귀신경망 앙상블 모델을 기반으로 한 조위관측소 해수위의 준실시간 이상값 탐지
A Non-annotated Recurrent Neural Network Ensemble-based Model for Near-real Time Detection of Erroneous Sea Level Anomaly in Coastal Tide Gauge Observation 원문보기

바다 : 한국해양학회지 = The sea : the journal of the Korean society of oceanography, v.26 no.4, 2021년, pp.307 - 326

이은주 (인하대학교 해양과학과) , 김영택 (국립해양조사원 해양예보과) , 김송학 (인하대학교 해양과학과) , 주호정 (인하대학교 해양과학과) , 박재훈 (인하대학교 해양과학과)

초록
AI-Helper

상시 관측되는 조위관측소 해수위 자료는 결측값과 오측값을 포함하고 있으며, 그 중 오측 값은 이상값으로 분류되는 전처리 대상이다. 이러한 오측을 제거하기 위해 대표적으로 3𝜎 (three standard deviations) 규칙이 적용되어왔으나, 기상이변 등에 의한 극값이 존재하거나 3𝜎 범위 안에서도 오측이 존재하는 해수위 자료에는 그 적용이 어렵다. 본 연구에서 설계된 모델은 오측에 대한 사전 정보가 필요하지 않은 비주석 학습으로 구성되며, 재귀신경망과 앙상블 기법을 이용함으로써 실시간으로 수집되는 해수위 자료가 오측일 가능성을 발생한지 20분 이내로 제시한다. 검증이 완료된 모델은 평시 및 기상이변시의 정상값과 오측값을 잘 분리하며, 학습이 이뤄지지 않은 연도의 해수위 자료에서도 이상값 탐지가 가능함을 확인하였다. 본 연구의 관측 이상치 탐지 알고리즘은 조위관측소 해수위에 국한되지 않고 다양한 해양 및 대기자료의 이상치 탐지 인공신경망 모델에 확장 적용할 수 있다.

Abstract ▼ AI-Helper

Real-time sea level observations from tide gauges include missing and erroneous values. Classification as abnormal values can be done for the latter by the quality control procedure. Although the 3𝜎 (three standard deviations) rule has been applied in general to eliminate them, it is difficult to apply it to the sea-level data where extreme values can exist due to weather events, etc., or where erroneous values can exist even within the 3𝜎 range. An artificial intelligence model set designed in this study consists of non-annotated recurrent neural networks and ensemble techniques that do not require pre-labeling of the abnormal values. The developed model can identify an erroneous value less than 20 minutes of tide gauge recording an abnormal sea level. The validated model well separates normal and abnormal values during normal times and weather events. It was also confirmed that abnormal values can be detected even in the period of years when the sea level data have not been used for training. The artificial neural network algorithm utilized in this study is not limited to the coastal sea level, and hence it can be extended to the detection model of erroneous values in various oceanic and atmospheric data.

주제어

표/그림 (17)

그림 Fig. 1. Bathymetry of the study area and location of five tide gauge stations (BNP: Busan new port, BS: Busan, GJ: Geoje-do, TY: Tongyeong, US: Ulsan).
표 Table 1. Summary of the characteristics of sea level components (BNP: Busan new port, BS: Busan, GJ: Geoje-do, TY: Tongyeong, US: Ulsan)
표 Table 2. Length of dataset at the five tide gauge observatories during 2018－2020 (BNP: Busan new port; BS: Busan; GJ: Geoje-do; TY: Tongyeong; US: Ulsan. Unit: ea.)
그림 Fig. 2. Structure of LSTM model1). The three-dimensional structure consists of batch size, sequence, and feature, while the two-dimensional structure consists of batch size and feature. Question mark indicates flexible batch sizes depending on data length.
표 Table 3. Summary of model verification results at the five tide gauge observatories (BNP: Busan new port; BS: Busan; GJ: Geoje-do; TY: Tongyeong; US: Ulsan)
그림 Fig. 3. Scatter plot of model verification results at the five observatories. The results of the voting are colored, and observations that receive more than 70 percent of the votes are QC targets.
그림 Fig. 4. Time series of validated model results in normal times. The gray and navy solid lines mean the observations and the average value of the model results (filled with blue: 99% confidence interval), respectively. For decision making, fewer (1 to 5) voting and non-voting are represented by green and blue scissors, respectively. Depending on the voting rate with the model, bad, probably bad, and probably good are marked with red, orange, and yellow dots, respectively.
그림 Fig. 5. Time series of validated model results in 2020 oceanic events. Legend is the same as Fig. 4.
그림 Fig. 6. Time series of test data from 2018 to 2019 applied to the validated model trained with 2020 data (in normal times). Legend is the same as Fig. 4.
그림 Fig. 7. Time series of test data from 2018 to 2019 applied to the validated model trained with 2020 data (during the oceanic events). Legend is the same as Fig. 4.
그림 Fig. 8. Test for overfitting cases depending on the year of training data. The horizontal axis represents the training year and the vertical axis represents the test year, and hence the overfitting can be seen in the diagonal components (red boxes; the same year of training and test). Legend is the same as Fig. 4.
그림 Fig. 9. Test for underfitting cases depending on the year of training data. The x-axis represents the training year and the y-axis represents the test year. At very few oceanic events, underfitting may occur, especially in untrained years (black boxes). Legend is the same as Fig. 4.
표 Table 4. Configurations of the case studies for the impact of time step change
표 Table 5. Summary of quality control results for the three model cases
그림 Fig. 10. Time series of Case1 (FC-ANN, time step: 1) model results. Legend is the same as Fig. 4.
그림 Fig. 11. Time series of Case2 (LSTM, time step: ±1 hour) model results. Legend is the same as Fig. 4.
그림 Fig. 12. Quality control results from the three QC methods. The first row is the result of this study, the second row is an example of a 5sigma rule with a combination of the residuals and the time variations of the residuals, and the third row is an example of a 3sigma rule with a 31-minute window. The solid black line represents the quality-treated sea level value. The solid yellow line means the time variation of the residuals, and the dashed gray and yellow line means the 5σ of the residuals and the residual variation, respectively. The red dot is the target of QC, and the green and blue dot is the target of QC by the 5σ limit of residual and residual variations, respectively.

참고문헌 (24)

국립해양조사원(KHOA), 2021. "바다누리 해양정보 서비스". http://www.khoa.go.kr/oceangrid/gis/category/reference/distribution.
김마가, 최진용, 방재홍, 이재주, 2019. 임계치 모형과 인공신경망 모형을 이용한 실시간 저수위 수위자료의 이상치 탐지. 한국농공학회, 61(1): 107-120.
Bell, C., J.M. Vassie and P.L. Woodworth, 1999. POL/PSMSL Tidal Analysis Software Kit 2000 (TASK-2000). edited by 2000UKCCMS Proudman Oceanographic Laboratory Permanent Service for Mean Sea Level.
Castelao, G.P., 2021. A machine learning approach to quality control oceanographic data. Computers & Geosciences, 155.

상세보기
Fourrier, M., L. Coppola, H. Claustre, F. D'Ortenzio, R. Sauzede and J.-P. Gattuso, 2020. A Regional Neural Network Approach to Estimate Water-Column Nutrient Concentrations and Carbonate System Variables in the Mediterranean Sea: CANYON-MED. Frontiers in Marine Science 7.
Goyal, S., T. Jain and Dr. N. Tyagi, 2018. Artificial Neural Network: A Review and its Application in Managing Water Quality Control. International Journal of Scientific Development and Research, 3(6).
Guan, Y. and T. Plotz, 2017. Ensembles of Deep LSTM Learners for Activity Recognition Using Wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2): 1-28.
Hochreiter, S. and J. Schmidhuber, 1997. Long Short-Term Memory. Neural Computation, 9(8): 1735-1780.

상세보기
Just, A.C., M.M. De Carli, A. Shtein, M. Dorman, A. Lyapustin and I. Kloog, 2018. Correcting Measurement Error in Satellite Aerosol Optical Depth with Machine Learning for Modeling PM2.5 in the Northeastern USA. Remote Sens (Basel), 10(5).
Kim, H.J., S.M. Park, B.J. Choi, S.H. Moon and Y.H. Kim, 2020. Spatiotemporal Approaches for Quality Control and Error Correction of Atmospheric Data through Machine Learning. Comput Intell Neurosci, 2020: 7980434.
Linares-Rodriguez, A., J.A. Ruiz-Arias, D. Pozo-Vazquez and J. Tovar-Pescador, 2013. An artificial neural network ensemble model for estimating global solar radiation from Meteosat satellite images. Energy, 61: 636-645.
Liu, J., J. Gu, H. Li and K.H. Carlson, 2020. Machine learning and transport simulations for groundwater anomaly detection. Journal of Computational and Applied Mathematics, 380.

상세보기
Lopez-Lineros, M., J. Estevez, J.V. Giraldez and A. Madueno, 2014. A new quality control procedure based on non-linear autoregressive neural network for validating raw river stage data. Journal of Hydrology, 510: 103-109.
Miau, S. and W.-H. Hung, 2020. River Flooding Forecasting and Anomaly Detection Based on Deep Learning. IEEE Access 8:198384-198402.

상세보기
Murray, M.T., 1964. A General Method for the Analysis of Hourly Heights of Tide. International Hydrographic Review, 41(2): 91-101.
Omar, S., A. Ngadi and H.H. Jebur, 2013. Machine Learning Techniques for Anomaly Detection: An Overview. International Journal of Computer Applications, 79(2).
Opitz, D. and R. Maclin, 1999. Popular Ensemble Learning: An Empirical Study. Journal of Artificial Intelligence Research, 11: 169-98.
Pastore, V.P., T.G. Zimmerman, S.K. Biswas and S. Bianco, 2020. Annotation-free learning of plankton for classification and anomaly detection. Sci Rep, 10(1): 12142.
Russell, S. and P. Norvig, 2010. Artificial Intelligence: A Modern Approach, Third Edition. Upper Saddle River, New Jersey: Prentice Hall, c2010.
Saini, S. and Preeti, 2015. Advance Applications of Artificial Neural Network. Journal of Basic and Applied Engineering Research, 2(11): 1001-1005.
Sciuto, G., B. Bonaccorso, A. Cancelliere and G. Rossi, 2009. Quality control of daily rainfall data with neural networks. Journal of Hydrology, 364(1-2): 13-22.
UNESCO (United Nations Educational, Scientific and Cultural Organization). 2013. Ocean Data Standards, Vol.3: Recommendation for a Quality Flag Scheme for the Exchange of Oceanographic and Marine Meteorological Data. (IOC Manuals and Guides, 54, Vol. 3.).
Wang, Y., L. Han, W. Liu, S. Yang and Y. Gao, 2019. Study on wavelet neural network based anomaly detection in ocean observing data series. Ocean Engineering, 186.

상세보기
Yu, J.B. and L.F. Xi, 2009. A neural network ensemble-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes. Expert Systems with Applications, 36(1): 909-921.

상세보기

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format