[논문]Application of YOLOv5 Neural Network Based on Improved Attention Mechanism in Recognition of Thangka Image Defects

Fan, Yao; Li, Yubo; Shi, Yingnan; Wang, Shuaishuai

doi:10.3837/tiis.2022.01.014

Application of YOLOv5 Neural Network Based on Improved Attention Mechanism in Recognition of Thangka Image Defects 원문보기

KSII Transactions on internet and information systems : TIIS, v.16 no.1, 2022년, pp.245 - 265

Fan, Yao (School of Information Engineering, Xizang Minzu University) , Li, Yubo (School of Information Engineering, Xizang Minzu University) , Shi, Yingnan (School of Information Engineering, Xizang Minzu University) , Wang, Shuaishuai (School of Information Engineering, Xizang Minzu University)

Abstract ▼ AI-Helper

In response to problems such as insufficient extraction information, low detection accuracy, and frequent misdetection in the field of Thangka image defects, this paper proposes a YOLOv5 prediction algorithm fused with the attention mechanism. Firstly, the Backbone network is used for feature extraction, and the attention mechanism is fused to represent different features, so that the network can fully extract the texture and semantic features of the defect area. The extracted features are then weighted and fused, so as to reduce the loss of information. Next, the weighted fused features are transferred to the Neck network, the semantic features and texture features of different layers are fused by FPN, and the defect target is located more accurately by PAN. In the detection network, the CIOU loss function is used to replace the GIOU loss function to locate the image defect area quickly and accurately, generate the bounding box, and predict the defect category. The results show that compared with the original network, YOLOv5-SE and YOLOv5-CBAM achieve an improvement of 8.95% and 12.87% in detection accuracy respectively. The improved networks can identify the location and category of defects more accurately, and greatly improve the accuracy of defect detection of Thangka images.

주제어

표/그림 (24)

그림 Fig. 1. Structure diagram of YOLOv5-SE mechanism
그림 Fig. 2. Structure diagram of YOLOv5-CBAM mechanism
그림 Fig. 3. The relationship between the rectangular boxes of X, Y, and D
그림 Fig. 4. Picture size and channel number of each layer of YOLOv5-SE
그림 Fig. 5. Picture size and channel number of each layer of YOLOv5-CBAM
표 Table 1. Algorithm flow description
표 Table 2. Distribution of defect samples in the test set
그림 Fig. 6. Histogram of the effect of 3000 iterations
그림 Fig. 8. Histogram of the effect of 1000 iterations
그림 Fig. 7. The histogram of the effect of 6000 iterations
그림 Fig. 9. Histogram of the effect of 3000 iterations
표 Table 3. Time consumption of unexpanded data set training
그림 Fig. 10. Histogram of the effect of 6000 iterations
그림 Fig. 11. Histogram of the effect of 10000 iterations
표 Table 4. Time consumption and FPS of expanded data set training
표 Table 5. Detection data concerning Fade defect (Defect 0)
표 Table 6. Detection data concerning Crack defect (Defect 1)
표 Table 7. Detection data concerning Dent defect (Defect 2)
표 Table 8. Detection data concerning Damage defect (Defect 3)
표 Table 9. Detection data concerning Stain defect (Defect 4)
그림 Fig. 12. Comparison of images of different bbox loss functions
표 Table 10. Detection effect of GIoU and CIoU after 6000 iterations
그림 Fig. 13. Comparison of different algorithms for defective images
표 Table 11. Comparison of the model size when the epoch iterated 10000 times

AI 본문요약
AI-Helper

제안 방법

Aiming at resolving the problems of difficult feature extraction and low detection accuracy concerning defective Thangka images with complex background color, this paper has proposed an improved YOLOv5 model based on attention mechanism to detect the defect area in Thangka images. The model integrated SE and CBAM mechanisms respectively, and thus obtained two improved algorithms, YOLOv5-SE and YOLOv5-CBAM, which have effectively solved the problems encountered in Thangka image defect detection.
Due to the special data set of this experiment and the small number of initial data, the training effect was not very obvious. Therefore, the data set was expanded by the method of data enhancement, which increased not only the number of data sets, but also the diversity of the training data, thereby making the data training achieve better results.

대상 데이터

Consequently, image collection and processing has become an important part of this experiment. The data used in this paper came from the Thangka pictures taken in Tibet. Thangka images with defects were selected from the acquired data set of 5277 Thangka images.

이론/모형

This paper used the labeling tool of LabelImg to label the Thangka data sets and classify the defect area. There are five types of defect targets: fade, crack, dent, damage, and stain.

성능/효과

They have achieved improved detection accuracy and recall rate without greatly increasing the time cost, and their detection speed has been only slightly slower than that of YOLOv5s. However, compared with the speed of YOLOv5l and YOLOv3, the improved networks have had many advantages, and their precision has been far from inferior to that of other models. YOLOv5-SE has achieved an improvement of 8.
The improvement in accuracy of YOLOv5-CBAM has been more obvious than that of YOLOv5-SE. In general, the real-time performance of both the improved networks has been effectively enhanced, thereby capable of effectively solving the problems regarding defect extraction of Thangka images.
The experimental results have shown that the improved defect detection models can accurately extract the features of the defect area from images with complex background color and detect multiple defects simultaneously. They have achieved improved detection accuracy and recall rate without greatly increasing the time cost, and their detection speed has been only slightly slower than that of YOLOv5s.
The experimental results show that the in terms of the precision, recall rate, and @0.5, the two networks improved based on YOLOv5s greatly outperformed the original YOLOv5s model, thus capable of extracting features of defect Thangka images with complex background more accurately. However, the addition of SE and CBAM mechanisms increased the depth of the network models, which sacrificed a certain speed advantage to improve the effect of detecting defects.
87%. The improvement in accuracy of YOLOv5-CBAM has been more obvious than that of YOLOv5-SE. In general, the real-time performance of both the improved networks has been effectively enhanced, thereby capable of effectively solving the problems regarding defect extraction of Thangka images.
Aiming at resolving the problems of difficult feature extraction and low detection accuracy concerning defective Thangka images with complex background color, this paper has proposed an improved YOLOv5 model based on attention mechanism to detect the defect area in Thangka images. The model integrated SE and CBAM mechanisms respectively, and thus obtained two improved algorithms, YOLOv5-SE and YOLOv5-CBAM, which have effectively solved the problems encountered in Thangka image defect detection.
The experimental results have shown that the improved defect detection models can accurately extract the features of the defect area from images with complex background color and detect multiple defects simultaneously. They have achieved improved detection accuracy and recall rate without greatly increasing the time cost, and their detection speed has been only slightly slower than that of YOLOv5s. However, compared with the speed of YOLOv5l and YOLOv3, the improved networks have had many advantages, and their precision has been far from inferior to that of other models.
However, compared with the speed of YOLOv5l and YOLOv3, the improved networks have had many advantages, and their precision has been far from inferior to that of other models. YOLOv5-SE has achieved an improvement of 8.95% in detection accuracy, and YOLOv5-CBAM an improvement of 12.87%. The improvement in accuracy of YOLOv5-CBAM has been more obvious than that of YOLOv5-SE.

참고문헌 (23)

L. Li, "Tibetan Buddhist Thangka Art," Journal of Kangding Nationaly Teachers College, no. 01, pp. 13-16, 2008.
L. Jie, "Research and implementation of Thang-ga image inpainting system based on deep learning," Ningxia University, 2020.
Zhao Z., Li B., Dong R., Zhao P, "A Surface Defect Detection Method Based on Positive Samples," in Proc. of Pacific Rim International Conference on Artificial Intelligence (PRICAI 2018), Nanjing, China, vol. 11013, pp. 473-481, 2018.
M. Shuang, W. Yudan, and W. Guojun, "Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model," Journal of Sensors, Basel, Switzerland, vol. 18, no. 4, 2018.
Z. Xu and H. Dingjiang, "Defect detection on aluminum surfaces based on deep learning," Journal of East China Normal University (Natural Science Edition), no. 06, pp. 105-114, 2020.
C. Conghan, "Application of deep neural network based on YOLOv3 in aircraft surface defect recognition, " Civil Aviation Flight University of China, vol. 10, 2020.
X. Yingying, L. Dawei, X. Qian, W. Qiaoyun, and W. Jun, "Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN," Journal of Measurement, vol. 178, 2021.
W. Qingxia, Y. Ronghui, W. Chongjun, and L. Yao, "An effective defect detection method based on improved Generative Adversarial Networks (iGAN) for machined surfaces," Journal of Journal of Manufacturing Processes, vol. 65, no. 5, pp. 373-381, 2021.

상세보기
Y. J. Cha, W. Choi, G. Suh, S. Mahmoudkhani, and O. Buyukozturk, "Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types," Special Issue: Health Monitoring of Structures, vol. 33, no. 9, pp. 731-747, 2018.
D. Tabernik, S. Sela, J. Skvarc, and D. Skocaj, "Segmentation-based deep-learning approach for surface-defect detection," Journal of Journal of Intelligent Manufacturing, vol. 31, no. 3, pp. 759-776, 2020.

상세보기
K. He, G. Gkioxari, P. Dollar, R. Girshick, "Mask R-CNN," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386-397, 2020.

상세보기
Wang, T., Chen, Y., Qiao, M. et al., "A fast and robust convolutional neural network-based defect detection model in product quality control," Int J Adv Manuf Technol, vol. 94, pp. 3465-3471, 2018.

상세보기
J. Chen, Z. Liu, H. Wang, A. Nunez, Z. Han, "Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network," IEEE Transactions on Instrumentation and Measurement, vol. 67, no. 2, pp. 257-269, 2018.

상세보기
S. Mei, H. Yang, Z. Yin, "An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces," IEEE Transactions on Instrumentation and Measurement, vol. 67, no. 6, pp. 1266-1277, 2018.

상세보기
C. Feng, M. Y. Liu, C. C. Kao, and T. Y. Lee, "Deep Active Learning for Civil Infrastructure Defect Detection and Classification," in Proc. of ASCE International Workshop on Computing in Civil Engineering, 2017.
Lei et al., "Scale insensitive and focus driven mobile screen defect detection in industry," Neurocomputing, vol. 294, pp. 72-81, 2018.

상세보기
D. Tabernik, S. Ela, J. Skvar, and D. J. J. o. I. M. Skoaj, "Segmentation-based deep-learning approach for surface-defect detection," Journal of Intelligent Manufacturing, vol. 31, pp. 759-776, 2020.

상세보기
H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, and S. Savarese, "Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression," in Proc. of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, pp. 658-666, 2019.
D. Oro, C. Fernandez, X. Martorell, and J. Hernando, "Work-efficient parallel non-maximum suppression for embedded GPU architectures," in Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, pp. 1026-1030, 2016.
J. Hu, L. Shen, S. Albanie, G. Sun and E. Wu, "Squeeze-and-Excitation Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 8, pp. 2011-2023, 1 Aug. 2020.

상세보기
Woo S., Park J., Lee JY., Kweon I.S, "CBAM: Convolutional Block Attention Module," in Proc. of the European Conference On Computer Vision (ECCV 2018), Munich, Germany, vol. 11211, pp. 3-19, 2018.
Jiang, B., Luo, R., Mao, J., Xiao, T., & Jiang, Y, "Acquisition of localization confidence for accurate object detection," in Proc. of the European Conference on Computer Vision (ECCV 2018), Munich, Germany, vol. 11218, pp. 816-832, 2018.
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D, "Distance-IoU loss: Faster and better learning for bounding box regression," in Proc. of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12993-13000, 2020.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증