[논문]Anomaly pattern detection for streaming data

Kim, Taegong; Park, Cheong Hee

doi:10.1016/j.eswa.2020.113252

[해외논문] Anomaly pattern detection for streaming data

Expert systems with applications, v.149, 2020년, pp.113252 -

Kim, Taegong (Corresponding author.) , Park, Cheong Hee

Abstract ▼ AI-Helper

Abstract Outlier detection aims to find a data sample that is different from most other data samples. While outlier detection is performed at an individual instance level, anomaly pattern detection on a data stream means detecting a time point where a pattern to generate data is unusual and significantly different from normal behavior. Beyond predicting the outlierness of individual data samples in a data stream, it can be very useful to detect the occurrence of anomalous patterns in real time. In this paper, we propose a method for anomaly pattern detection in a data stream based on binary classification for outliers and statistical tests on a data stream of binary labels of normal or an outlier. In the first step, by applying the clustering-based outlier detection method, we transform a data stream into a stream of binary values where 0 stands for the prediction as normal data and 1 for outlier prediction. In the second step, anomaly pattern detection is performed on a stream of binary values by two approaches: testing the equality of parameters in the binomial distributions of a reference window and a detection window, and using control charts for the fraction defective. The proposed method obtained the average true positive detection rate of 94% in simulated experiments using real and artificial data. The experimental results also show that anomaly pattern occurrence can be detected reliably even when outlier detection performance is relatively low. Highlights We propose an anomaly pattern detection method for streaming data. It detects a point where the pattern is unusually different from normal behavior. A data stream is transformed into a stream of binary values indicating outliers. An anomaly pattern on a stream of binary values is detected by two approaches. The proposed method is not sensitive to the performance of outlier detection method.

Keyword

참고문헌 (22)

Aggarwal 2017 Outlier analysis
Ahmad, S., & Purdy, S. (2016). Real-time anomaly detection for streaming analytics. Retrieved from https://arxiv.org/pdf/1607.02480.pdf.
Journal of Machine Learning Research Bife 11 1601 2010 Moa: Massive online analysis

상세보기
Das 2008 Proceedings of KDD Anomaly pattern detection in categorical datasets
Pattern Recognition Domingues 74 406 2018 10.1016/j.patcog.2017.09.037 A comparative evaluation of outlier detection algorithms: Experiments and analyses

상세보기
Haque 2016 Proceedings of the AAAI Sand: Semi-supervised adaptive novel class detection and classification over data stream
Hawkins 1980 Identification of outliers
Hawkins 2002 Proceedings of the international conference on data warehousing and knowledge discovery Outlier detection using replicator neural networks
IEICE Transactions on Information and systems Kim E100-D(10) 2537 2017 10.1587/transinf.2017EDP7091 An efficient concept drift detection method for streaming data under limited labeling

상세보기
Lindstrom 2011 Proceedings of the IEEE 11th int. conf. data mining workshops Drift detection using uncertainty distribution divergence
Liu 2008 Proceedings of the 8th international conference on data mining Isolation forest
Biometrika Mosteller 48(3-4) 433 1961 10.1093/biomet/48.3-4.433 Tables of the Freeman-Tukey transformations for the binomial and poisson distributions
Navidi 2006 Statistics for engineers and scientists
Padilla 2013 Proceedings of IEEE international conference on computational intelligence and cybernetics Performance of a hierarchical temporal memory network in noisy sequence learning
Journal of Supercomputing Park 75(9) 6118 2019 10.1007/s11227-018-2674-1 Outlier and anomaly pattern detection on data streams

상세보기
KIPS Transactions Software and Data Engineering Park 7(11) 435 2018 Outlier detection by clustering-based ensemble model construction
Sethi 2015 Proceedings of the INNS conference on big data Don’t pay for validation: Detecting drifts from unlabeled data using margin density
Sheldon 2004 Introduction to probability and statistics for engineers and scientists
Spinosa 2007 Proceedings of the SAC Olindda: A cluster-based approach for detecting novelty and concept drift in data streams
Spinosa 2008 Proceedings of the SAC Cluster-based novel concept detection in data streams applied to intrusion detection in computer networks
Wong 2002 Proceedings of the 18th international conference on artificial intelligence Rule-based anomaly pattern detection for detecting disease outbreaks
Zliobaite 2010 Proceedings of the IEEE international conference on data mining workshops Change with delayed labeling: When is it detectable?

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

원문 URL 링크

DOI : 10.1016/j.eswa.2020.113252
Elsevier : 저널 > 논문

*원문 PDF 파일 및 링크정보가 존재하지 않을 경우 KISTI DDS 시스템에서 제공하는 원문복사서비스를 사용할 수 있습니다.

저작권 관리 안내

내보내기 메뉴

내보내기 구분

파일저장
인쇄
메일전송

구성항목

기본정보
상세정보

관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관

저장형식

Text(ASCII format)
Excel format
RefWorks Direct Export
RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley

메일정보

받는사람 (필수): @
보내는사람 (선택): @
제목
내용: KISTI 검색결과 이메일 서비스

안내

총 건의 자료가 검색되었습니다.

다운받으실 자료의 인덱스를 입력하세요. (1-10,000)

검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다.

데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요)

다운로드 파일은 UTF-8 형태로 저장됩니다.
파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오.

Text(ASCII format)
Excel format

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

AI-Helper ※ AI-Helper는 을 사용합니다.

AI-Helper

안녕하세요, AI-Helper입니다. 좌측 "선택된 텍스트"에서 텍스트를 선택하여 요약, 번역, 용어설명을 실행하세요.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.

연합인증