[논문]Abnormal State Detection using Memory-augmented Autoencoder technique in Frequency-Time Domain

Haoyi Zhong; Yongjiang Zhao; Chang Gyoon Lim

doi:10.3837/tiis.2024.02.005

Abnormal State Detection using Memory-augmented Autoencoder technique in Frequency-Time Domain

KSII Transactions on internet and information systems : TIIS, v.18 no.2, 2024년, pp.348 - 369

Haoyi Zhong (Department of Computer Engineering, Chonnam National University) , Yongjiang Zhao (Department of Computer Engineering, Chonnam National University) , Chang Gyoon Lim (Department of Computer Engineering, Chonnam National University)

Abstract ▼ AI-Helper

With the advancement of Industry 4.0 and Industrial Internet of Things (IIoT), manufacturing increasingly seeks automation and intelligence. Temperature and vibration monitoring are essential for machinery health. Traditional abnormal state detection methodologies often overlook the intricate frequency characteristics inherent in vibration time series and are susceptible to erroneously reconstructing temperature abnormalities due to the highly similar waveforms. To address these limitations, we introduce synergistic, end-to-end, unsupervised Frequency-Time Domain Memory-Enhanced Autoencoders (FTD-MAE) capable of identifying abnormalities in both temperature and vibration datasets. This model is adept at accommodating time series with variable frequency complexities and mitigates the risk of overgeneralization. Initially, the frequency domain encoder processes the spectrogram generated through Short-Time Fourier Transform (STFT), while the time domain encoder interprets the raw time series. This results in two disparate sets of latent representations. Subsequently, these are subjected to a memory mechanism and a limiting function, which numerically constrain each memory term. These processed terms are then amalgamated to create two unified, novel representations that the decoder leverages to produce reconstructed samples. Furthermore, the model employs Spectral Entropy to dynamically assess the frequency complexity of the time series, which, in turn, calibrates the weightage attributed to the loss functions of the individual branches, thereby generating definitive abnormal scores. Through extensive experiments, FTD-MAE achieved an average ACC and F1 of 0.9826 and 0.9808 on the CMHS and CWRU datasets, respectively. Compared to the best representative model, the ACC increased by 0.2114 and the F1 by 0.1876.

주제어

표/그림 (13)

그림 Fig. 1. Comparison of abnormal and normal samples in CMHS dataset.
그림 Fig. 2. Comparison of normal and abnormal vibration data and their spectrum plot after Fast Fourier Transform in the CWRU dataset.
그림 Fig. 3. An overview of the FTD-MAE.
그림 Fig. 4. STFT workflow implementation with Hamming Window.
표 Table 1. Description of the dataset
표 Table 2. Performance of FTD-MAE and six representative models
표 Table 3. Various experimental results applying FTD-MAE with the datasets
그림 Fig. 5. Comparison of reconstructed abnormal samples in FTD-MAE time domain branch with and without memory module.
그림 Fig. 6. Comparison of reconstructed abnormal samples in FTD-MAE frequency domain branch with and without memory module.
표 Table 4. Temperature data preprocessing comparison
표 Table 5. Performance of FTD-MAE frequency domain branch on two datasets with different time-frequency map transformation methods
그림 Fig. 7. Performance of each branch of FTD-MAE on CMHS and CWRU datasets with different memory sizes.
그림 Fig. 8. Performance of FTD-MAE on CMHS and CWRU datasets with different weighting factor sizes.

AI 본문요약
AI-Helper

가설 설정

In this study, we choose data where the ‘stability flag’ is 1 and aside from the cooler, all other components are largely normal, to represent abnormal states, and we consider data where the cooler is functioning normally and the ‘stability flag’ is 0 as representing normal states

제안 방법

The results emphasize the benefits of using a memory module and a dual-branch architecture. Additionally, ablation studies and sensitivity tests for hyperparameters were conducted, further validating the effectiveness and robustness of the proposed method.
In this experiment, we compare FTD-MAE with six other representative models, including AE, VAE, Isolation Forest, PCA (Principal Component Analysis) [40], OC-SVM [41] and LSTM (Long Short-Term Memory) [42]. The encoders and decoders of AE, VAE and the time domain branch of FTD-MAE are the same.
In this study, we used Accuracy and F1-Score as metrics to evaluate the performance of the model. Accuracy is the proportion of the number of samples that are correctly classified to the total number of samples.
In the course of training, the frequency domain and time domain branches are individually trained, with the training set comprising only normal samples. The objective of training is to have the reconstructed samples closely approximate the normal samples; therefore, we employ mean squared error as the loss function. The formulae are shown in (18) and (19).

성능/효과

(1) In the context of industrial equipment's temperature data, the variation patterns between normal and abnormal samples within each sampling period are highly alike
(2) AE used for abnormalstate detection is generally crafted to reconstruct input data within the time domain. Yet, the majority of vibration data are not stationary and have features like a broad dynamic scope and rapid frequency shifts.
As shown in Fig. 3, the input first enters the time domain branch (green line) where the encoder captures its latent representation. Utilizing this latent representation as a query, the memory module learns the distance between memories via attention mechanisms and then passes it on to the decoder for reconstruction.
As shown in Fig. 4, when the time series is input into the frequency domain branch of the model, it first undergoes STFT to produce a two-dimensional image containing time and corresponding frequencies. The time series is then decomposed into M sub-sequences with overlapping sections through a sliding Hamming window.
A memory module between the branch encoder and decoder enables FTD-MAE to learn normal data patterns, minimizing over-generalization on abnormal samples. Additionally, the weights of the branches can be dynamically adjusted using Spectral Entropy, thereby adapting to time series with different frequency complexities and reducing computational costs. After rigorous testing, FTD-MAE demonstrated remarkable performance on the CMHS and CWRU datasets, with an average ACC of 0.
Additionally, the weights of the branches can be dynamically adjusted using Spectral Entropy, thereby adapting to time series with different frequency complexities and reducing computational costs. After rigorous testing, FTD-MAE demonstrated remarkable performance on the CMHS and CWRU datasets, with an average ACC of 0.9826 and an average F1 score of 0.9808, respectively. When compared to the top-performing model, there was an enhancement of 0.
Although FTD-MAE has achieved good performance, its performance improvement is more dependent on each branch itself. The association of two branches is only in the inference part of the model, which balances the decisions of the branches through the weight factor.
In terms of vibration data, Case Western Reserve University utilized a 2-horsepower Reliance electric motor for collecting acceleration data at different positions relative to the bearings of the motor. Artificial faults were induced on the inner raceway, rolling elements, and outer raceway with diameters ranging from 0.007 to 0.040 inches. The faulty bearings were then reinstalled into the testing motor and vibration data were logged when the motor was under loads ranging from 0 to 3 horsepower and with rotational speeds from 1797 to 1720 RPM [39].
FTD-MAE's overall performance suffers when it lacks a memory module, because AE's generalization is strong
This is because when frequency complexity is low, abnormalities in the data are more likely to occur in short-term fluctuations or sudden changes, rather than distributed across the entire frequency range. FTD-MAE-T excels in capturing these short-term abnormal shifts since it analyzes data directly within its time domain, allowing for more accurate pinpointing and identification of local abnormal. Conversely, FTD-MAE-F focuses mainly on features in the frequency domain, making it more suitable for handling data with high-frequency complexity or obvious periodicity.
The main reasons are twofold. First, AE has strong generalization capabilities, which makes it easy for abnormalities in waveform-similar data like temperature data to be reconstructed, thereby affecting the performance of abnormal state detection. Second, for non-stationary data like vibration data, AE tends to overlook frequency characteristics.
The selection of the memory size is highly correlated with the data complexity. For temperature time series with low complexity, only a small memory is needed to achieve better reconstruction of normal samples and suppression of abnormal samples (Fig. 7 a).
From the table, it's apparent that on complex frequency vibration dataset, STFT, wavelet transform, and HHT perform similarly. However, for temperature dataset with trends, wavelet transform and HHT show significantly lower ACC than STFT. This is likely because, despite their variable time-frequency resolution, wavelet transform and HHT may struggle to maintain a consistent time-frequency resolution like STFT in signals with long-term trends, impacting result accuracy.
In comparison to the differencing method, moving averages, wavelet transformations and HHT, STFT offers an intuitive bi-dimensional representation of time and frequency, which enables clearer capturing of the signal's time and spectral attributes
Conversely, FTD-MAE-F focuses mainly on features in the frequency domain, making it more suitable for handling data with high-frequency complexity or obvious periodicity. In such data, abnormalities are likely to manifest as sudden changes in frequency components, therefore, FTD-MAE-F performs better in CWRU data with high-frequency complexity. Without a sparsity threshold, the performance of both FTD-MAE-F and FTD-MAE-T declines, with FTD-MAE-T experiencing the most significant drop in performance on the CWRU dataset.
In summary, FTD-MAE-T excels at capturing time domain features, while FTD-MAE-F specializes in capturing frequency characteristics. Together, they make the model robust to different types of data.
ACC and F1-Score results of FTD-MAE as compared to other representative models are displayed in Table 2. It can be seen that the ACC and F1-Score of FTD-MAE on the CMHS dataset are 0.9734 and 0.9668, respectively, which are 2.66% and 3.75% higher than the best representative model. On the CWRU dataset, FTD-MAE records an ACC and F1-Score of 0.
75% higher than the best representative model. On the CWRU dataset, FTD-MAE records an ACC and F1-Score of 0.9917 and 0.9948, surpassing the best representative model by 39.61% and 33.76% respectively.
7 a). Relative to the more complex vibrational time series, the optimal memory size reaches 500, far exceeding the temperature time series (Fig. 7 b).
First, AE has strong generalization capabilities, which makes it easy for abnormalities in waveform-similar data like temperature data to be reconstructed, thereby affecting the performance of abnormal state detection. Second, for non-stationary data like vibration data, AE tends to overlook frequency characteristics.
Since the trends of individual temperature time series are closely aligned, the model could potentially fit the data trends at large time scales, neglecting variations at smaller scales. Therefore, we apply trend attenuation in processing these temperature time series.
The frequency domain branch doesn't attenuate the trend, as its window function lowers time resolution and makes it challenging to detect short-term fluctuations
We compared the two preprocessing methods on the CMHS temperature dataset, and the results are shown in Table 4. The frequency domain branch had a higher score when data was processed using MinMax-Scale, while the time domain branch scored higher when data was processed using Detrend.
However, for temperature dataset with trends, wavelet transform and HHT show significantly lower ACC than STFT. This is likely because, despite their variable time-frequency resolution, wavelet transform and HHT may struggle to maintain a consistent time-frequency resolution like STFT in signals with long-term trends, impacting result accuracy.
7 b). Unlike time series, image data after STFT is more complex, so the optimal memory size is 2000 (Fig. 7 c and d).
9808, respectively. When compared to the top-performing model, there was an enhancement of 0.2114 in average ACC and 0.1876 in average F1 score.
In such data, abnormalities are likely to manifest as sudden changes in frequency components, therefore, FTD-MAE-F performs better in CWRU data with high-frequency complexity. Without a sparsity threshold, the performance of both FTD-MAE-F and FTD-MAE-T declines, with FTD-MAE-T experiencing the most significant drop in performance on the CWRU dataset. This is because too many memory items can better reconstruct abnormal samples.

후속연구

For signals with a fast change trend (temperature), a shorter window can be used to better capture changes in time. For signals that trend more slowly (vibration), a longer window can be used for more accurate frequency analysis.
The size of the memory module affects the reconstruction quality. If the memory module is too small, the model will not be effective in reconstructing any samples; if it is too large, it may result in better reconstruction of abnormal samples. Hence, we evaluated the size of each memory module, as shown in Fig.
Signals with complex frequency components have characteristics such as multiple frequency components, susceptibility to noise interference, and nonlinear properties. Preprocessing these types of signals is a common problem in signal processing and time series analysis, the primary aim of which is to make the signal or data smoother for easier subsequent analysis or processing. Common methods include differencing, moving averages [29], wavelet transform [30], Hilbert-Huang transform (HHT) [31] and STFT.
It's important to note that setting static weights is impractical since the features of unseen data cannot be predicted. Therefore, it is necessary to dynamically adjust these weights by continually evaluating the frequency complexity of both the original and unseen data. There are multiple ways to evaluate the frequency complexity of data, including statistical approaches [32], Renyi Entropy [33] and Spectral Entropy [15].
In comparison to the differencing method, moving averages, wavelet transformations and HHT, STFT offers an intuitive bi-dimensional representation of time and frequency, which enables clearer capturing of the signal's time and spectral attributes. Therefore, we opt to use STFT for the analysis of the signal.

참고문헌 (42)

P. K. Malik, R. Sharma, R. Singh, A. Gehlot, S. C. Satapathy, W. S. Alnumay, D. Pelusi, U. Ghosh, and J. Nayak, "Industrial Internet of Things and its Applications in Industry 4.0: State of The Art," Computer Communications, vol. 166, pp. 125-139, Jan. 2021.

상세보기
G. Toh and J. Park, "Review of vibration-based structural health monitoring using deep learning," Applied Sciences, vol. 10(5), p. 1680, Mar. 2020.
M. Alarcon, F. M. Martinez-Garcia, and F. C. Gomez de Leon Hijes, "Energy and maintenance management systems in the context of industry 4.0. Implementation in a real case," Renewable and Sustainable Energy Reviews, vol. 142, p. 110841, May 2021.

상세보기
H. Dhiman, D. Deb, S. M. Muyeen, and I. Kamwa, "Wind Turbine Gearbox Anomaly Detection Based on Adaptive Threshold and Twin Support Vector Machines," IEEE Transactions on Energy Conversion, vol. 36, no. 4, pp. 3462-3469, Dec. 2021.

상세보기
A. K. Pathak, S. Saguna, K. Mitra, and C. Ahlund, "Anomaly Detection using Machine Learning to Discover Sensor Tampering in IoT Systems," in Proc. of ICC 2021 - IEEE International Conference on Communications, Montreal, QC, Canada, 2021.
J. Gao, X. Song, Q. Wen, P. Wang, L. Sun, and H. Xu, "RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks," Cornell University - arXiv, Cornell University - arXiv, Feb. 2020.
M. Thill, W. Konen, H. Wang, and T. Back, "Temporal convolutional autoencoder for unsupervised anomaly detection in time series," Applied Soft Computing, vol. 122, p. 107751, Nov. 2021.
S. Ramaswamy, R. Rastogi, and K. Shim, "Efficient algorithms for mining outliers from large data sets," in Proc. of ACM SIGMOD Record, pp. 427-438, Jun. 2000.
F. T. Liu, K. M. Ting, and Z.-H. Zhou, "Isolation Forest," in Proc. of 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, 2008.
A. Toshniwal, K. Mahesh, and J. R., "Overview of Anomaly Detection techniques in Machine Learning," in Proc. of 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, India, 2020.
O. I. Provotar, Y. M. Linder, and M. M. Veres, "Unsupervised Anomaly Detection in Time Series Using LSTM-Based Autoencoders," in Proc. of 2019 IEEE International Conference on Advanced Trends in Information Theory (ATIT), Kyiv, Ukraine, 2019.
G. Spigler, "Denoising Autoencoders for Overgeneralization in Neural Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 4, pp. 998-1004, Apr. 2020.

상세보기
J. W. Cooley and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series," Mathematics of Computation, vol. 19, pp. 297-301, 1965.

상세보기
F. Jurado and J. R. Saenz, "Comparison between discrete STFT and wavelets for the analysis of power quality events," Electric Power Systems Research, vol. 62, no. 3, pp. 183-190, Jul. 2002.

상세보기
Z. Wang, J. Zhou, J. Wang, W. Du, J. Wang, X. Han, and G. He, "A Novel Fault Diagnosis Method of Gearbox Based on Maximum Kurtosis Spectral Entropy Deconvolution," IEEE Access, vol. 7, pp. 29520-29532, Jan. 2019.

상세보기
A. A. Cook, G. Misirli, and Z. Fan, "Anomaly Detection for IoT Time-Series Data: A Survey," IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6481-6494, Jul. 2020.

상세보기
X. Xia, X. Pan, N. Li, X. He, L. Ma, X. Zhang, and N. Ding, "GAN-based Anomaly Detection: A Review," Neurocomputing, vol. 493, pp. 497-535, Jul. 2022.

상세보기
M. Hassan, M. Rehmani, and J. Chen, "Anomaly Detection in Blockchain Networks: A Comprehensive Survey," IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 289-318, 2023.

상세보기
W. A. Chaovalitwongse, Y.-J. Fan, and R. C. Sachdeo, "On the Time Series K-Nearest Neighbor Classification of Abnormal Brain Activity," IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, vol. 37, no. 6, pp. 1005-1016, Nov. 2007.
M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander, "LOF: identifying density-based local outliers," in Proc. of ACM SIGMOD Record, pp. 93-104, Jun. 2000.
J. Baihong, C. Yuxin, L. Dan, P. Kameshwar, and S.-V. Alberto, "A One-Class Support Vector Machine Calibration Method for Time Series Change Point Detection," in Proc. of IEEE Conference Proceedings, IEEE Conference Proceedings, Jan. 2019.
S. Zhong, S. Fu, L. Lin, X. Fu, Z. Cui, and R. Wang, "A novel unsupervised anomaly detection for gas turbine using Isolation Forest," in Proc. of 2019 IEEE International Conference on Prognostics and Health Management (ICPHM), San Francisco, CA, USA, 2019.
V. Aggarwal, V. Gupta, P. Singh, K. Sharma, and N. Sharma, "Detection of Spatial Outlier by Using Improved Z-Score Test," in Proc. of 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, 2019.
F. E. Grubbs, "Sample Criteria for Testing Outlying Observations," The Annals of Mathematical Statistics, vol. 21, no. 1, pp. 27-58, 1950.

상세보기
K. Choi, J. Yi, C. Park, and S. Yoon, "Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines," IEEE Access, vol. 9, pp. 120043-120065, Jan. 2021.

상세보기
P. Matias, D. Folgado, H. Gamboa, and A. Carreiro, "Robust Anomaly Detection in Time Series through Variational AutoEncoders and a Local Similarity Score," in Proc. of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies, Online Streaming, --- Select a Country ---, 2021.
H. Liu, J. Zhou, Y. Zheng, W. Jiang, and Y. Zhang, "Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders," ISA Transactions, vol. 77, pp. 167-178, Jun. 2018.

상세보기
B. Du, X. Sun, J. Ye, K. Cheng, J. Wang, and L. Sun, "GAN-Based Anomaly Detection for Multivariate Time Series Using Polluted Training Set," IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 12, pp. 12208-12219, 2023.

상세보기
D. J. Bartholomew, G. E. P. Box, and G. M. Jenkins, "Time Series Analysis Forecasting and Control," Operational Research Quarterly (1970-1977), vol. 22, no. 2, p. 199, Jun. 1971.
C. Chakrabarti, M. Vishwanath, and R. M. Owens, "Architectures for wavelet transforms: A survey," Journal of VLSI signal processing systems for signal, image and video technology, vol. 14, no. 2, pp. 171-192, Nov. 1996.

상세보기
N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng, N.-C. Yen, C. C. Tung, and H. H. Liu, "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis," in Proc. of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, vol. 454, pp. 903-995, Mar. 1998.
A. Gupta, P. Mishra, C. Pandey, U. Singh, C. Sahu, and A. Keshri, "Descriptive statistics and normality tests for statistical data," Annals of Cardiac Anaesthesia, vol. 22, no. 1, pp. 67-72, Jan. 2019.

상세보기
R. G. Baraniuk, P. Flandrin, A. J. E. M. Janssen, and O. J. J. Michel, "Measuring time-frequency information content using the Renyi entropies," IEEE Transactions on Information Theory, vol. 47, no. 4, pp. 1391-1409, May 2001.

상세보기
J. Weston, S. Chopra, and A. Bordes, "Memory networks," in Proc. of inInternational Conference on Learning Representations (ICLR), 2015.
S. Sukhbaatar, a. szlam, J. Weston, and R. Fergus, "End-to-end memory networks," in Proc. of Neural Information Processing Systems, 2015.
A. Kumar, O. Irsoy, P. Ondruska, M. Iyyer, J. Bradbury, I. Gulrajani, V. Zhong, R. Paulus, and R. Socher, "Ask Me Anything: Dynamic Memory Networks for Natural Language Processing," arXiv: Computation and Language, arXiv: Computation and Language, Jun. 2015.
D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. Van Den Hengel, "Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection," in Proc. of 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019.
N. Helwig, E. Pignanelli, A. Schtze, "Condition monitoring of hydraulic systems," UCI Machine Learning Repository, 2018. [Online] . Available: https://archive.ics.uci.edu/dataset/447/condition+monitoring+of+hydraulic+systems
Case Western Reserve University Bearing Data Center, Dec. 2019. [Online] . Available: https://engineering.case.edu/bearingdatacenter
H. Ringberg, A. Soule, J. Rexford, and C. Diot, "Sensitivity of PCA for traffic anomaly detection," in Proc. of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, San Diego California USA, pp. 109-120, 2007.
B. Scholkopf, RobertC. Williamson, AlexanderJ. Smola, J. Shawe-Taylor, and J. Platt, "Support Vector Method for Novelty Detection," Neural Information Processing Systems, Neural Information Processing Systems, Nov. 1999.
S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, Nov. 1997.

상세보기

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증