[논문]소량 데이터 딥러닝 기반 강판 표면 결함 검출 시스템 개발

게이뷸라예프 압둘라지즈; 이나현; 이기환; 김태형

doi:10.14372/iemek.2022.17.3.129

소량 데이터 딥러닝 기반 강판 표면 결함 검출 시스템 개발
Development of a Steel Plate Surface Defect Detection System Based on Small Data Deep Learning 원문보기

대한임베디드공학회논문지 = IEMEK Journal of embedded systems and applications, v.17 no.3, 2022년, pp.129 - 138

게이뷸라예프 압둘라지즈 (Kumoh Nat'l Institute of Technology) , 이나현 (Kumoh Nat'l Institute of Technology) , 이기환 (Kumoh Nat'l Institute of Technology) , 김태형 (Kumoh Nat'l Institute of Technology)

Abstract ▼ AI-Helper

Collecting and labeling sufficient training data, which is essential to deep learning-based visual inspection, is difficult for manufacturers to perform because it is very expensive. This paper presents a steel plate surface defect detection system with industrial-grade detection performance by training a small amount of steel plate surface images consisting of labeled and non-labeled data. To overcome the problem of lack of training data, we propose two data augmentation techniques: program-based augmentation, which generates defect images in a geometric way, and generative model-based augmentation, which learns the distribution of labeled data. We also propose a 4-step semi-supervised learning using pseudo labels and consistency training with fixed-size augmentation in order to utilize unlabeled data for training. The proposed technique obtained about 99% defect detection performance for four defect types by using 100 real images including labeled and unlabeled data.

주제어

표/그림 (15)

그림 그림 1. 제안하는 기법의 전체 구조 Fig. 1. Overall Structure of the Proposed Method
그림 그림 2. f-DAGAN 네트워크 구조 Fig. 2. The f-DAGAN Network Structure
그림 그림 3. 생성모델 기반 데이터 증강 과정 Fig. 3. Data Augmentation based on Generative Models
그림 그림 4. 제안하는 준지도 학습 절차 Fig. 4. The Proposed Semi-Supervised Learning Procedure
그림 그림 5. 라벨 데이터 일관성 손실 기반 학습 Fig. 5. Consistency Loss Based Training of Labeled Data
그림 그림 6. 고정 크기 증강 기법 예 Fig. 6. Sample Fixed-Size Augmentation Methods
표 표 1. 각 학습 단계에 사용된 이미지 수 Table 1. The Number of Images Used in Each Training Stage
그림 그림 7. 수집된 결함 이미지 샘플 (흑점, 백점, 오염, 긁힘) Fig. 7. Sample Defect Images Collected (Black Spot, White Spot, Stain, Scratch)
그림 그림 8. 프로그램 생성 결함 이미지 샘플 (흑점, 백점, 오염, 긁힘) Fig. 8. Sample Defect Images Generated by the Program (Black Spot, White Spot, Stain, Scratch)
그림 그림 9. 생성 모델 생성 결함 이미지 샘플 (흑점, 백점, 오염, 긁힘) Fig. 9. Sample Defect Images Generated by the Generative Model (Black Spot, White Spot, Stain, Scratch)
그림 그림 10. 학습 및 추론 관리 플랫폼 구조 Fig. 10. The Structure of the Training and Inference Management Platform
그림 그림 11. 학습 및 추론 관리 서비스 화면 Fig. 11. A Screenshot of the Training and Inference Management Service
표 표 2. 결함 추론 결과에 대한 판정 방법 Table 2. Evaluation Method of Inference Outcomes
표 표 3. 각 생성 데이터로 사전 학습된 모델의 성능 비교 (f1-score) Table 3. Performance Comparison Between Pretrained Models with Each Generated Data (f1-score)
표 표 4. 학습 모델의 단계별 결함 검출 성능 (f1-score) Table 4. Step-by-Step Defect Detection Performance of the Trained Models (f1-score)

참고문헌 (39)

C. Shorten, T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," J Big Data, Vol. 6, No. 1, pp. 1-48, 2019.
Y. Wang, Q. Yao, J. T. Kwok, L. M. Ni, "Generalizing from a Few Examples: A Survey on Few-shot Learning," ACM Comput. Surv. Vol. 53, No. 3, pp. 1-3, 2021.
K. Weiss, T. M. Khoshgoftaar, D. D. Wang, "A Survey of Transfer Learning," J Big Data Vol. 3, No. 1, pp. 1-40, 2016.
S. Bernhard, P. John, H. Thomas, "Analysis of Representations for Domain Adaptation," Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference , MIT Press, pp. 137-144, 2007.
M. Khanam, T. Mahboob, W. Imtiaz, H. A. Ghafoor, R. Sehar, "A Survey on Unsupervised Machine Learning Algorithms for Automation, Classification and Maintenance," International Journal of Computer Applications. Vol. 119, No.13, pp. 34-39, 2015.
A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, F. Makedon, "A Survey on Contrastive Self- Supervised Learning," Technologies, Vol. 9, No.1 pp. 2, 2021.

상세보기
J. E. Van Engelen, H. H. Hoos, "A Survey on Semi-supervised Learning," Mach Learn Vol. 109, No. 2, pp. 373-440, 2020.

상세보기
T. DeVries, G. W. Taylor, "Improved Regularization of Convolutional Neural Networks with Cutout," arXiv:1708.04552, 2017.
H. Zhang, M. Cisse, Y. N. Dauphin, David Lopez-Paz, "mixup: Beyond Empirical Risk Minimization," arXiv:1710.09412, 2017.
A. Bochkovskiy, C. Y. Wang, H. Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection", arXiv:2004.10934, 2020.
G. Ghiasi, Y. Cui, A. Srinivas, R. Qian, Tsung-Yi Lin, Ekin D. Cubuk, Quoc V. Le, Barret Zoph, "Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2918-2928, 2021.
B. Christopher, C. Liang, G. P. B. Ricardo, G. Roger, H. Alexander, A. D. David, V. H. Maria, W. Joanna, R. Daniel, "GAN Augmentation: Augmenting Training data Using Generative Adversarial Networks," arXiv preprint. 2018.
A. Antoniou, A. Storkey, H. Edwards. "Data Augmentation Generative Adversarial Networks," In Proceedings of the International Conference on Learning Representations Workshops (ICLR Workshops), 2018.
M. Arjovsky, S. Chintala, L. Bottou, "Wasserstein GAN," arXiv:1701.07875, 2017.
Y. Xian, T. Lorenz, B. Schiele, Z. Akata. "Feature Generating Networks for Zero-shot Learning," In CVPR, 2018.
Y. Wang, Q. Yao, J. Kwok, L. M. Ni, "Generalizing from a Few Examples: A Survey on Few-Shot Learning," ACM Computing Surveys, Vol. 53, No. 3, pp. 1-34, 2020.

상세보기
D. Kim, D. Cho, D. Yoo, I. S. Kweon, "Learning Image Representations by Completing Damaged Jigsaw Puzzles," 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 793-802, 2018.
S. Gidaris, P. Singh, N. Komodakis, "Unsupervised Representation Learning by Predicting Image Rotations," 6th International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018.
L. Songtao, Z. Li, J. Sun. "Self-EMD: Self-Supervised Object Detection Wthout ImageNet." ArXiv abs/2011.13677, 2020.
D. H. Lee, "Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks," ICML 2013 Workshop: Challenges in Representation Learning (WREPL), Vol. 3, No. 2, pp. 896, 2013
S. Laine, T. Aila, "Temporal Ensembling for Semi-Supervised Learning," arXiv:1610.02242, 2016.
J. Jeong, S. Lee, J. Kim, N. Kwak, "Consistency-based Semi-supervised Learning for Object Detection," NIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 10759-10768, 2019.
A. Tarvainen, H. Valpola, "Mean Teachers are Better Role Models: Weight-averaged Consistency Targets Improve Semi-supervised Deep Learning Results," NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1195-1204, 2017.
Q. Xie, Z. Dai, E. Hovy, T. Luong, Q. Le, "Unsupervised Data Augmentation for Consistency Training," Advances in Neural Information Processing Systems 33 (NeurIPS 2020), pp. 6256-6268, 2020.
D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, C. Raffel, "MixMatch: a Holistic Approach to Semi-supervised Learning," NIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 5049-5059, 2019.
Q. Guo, Y. Mu, J. Chen, T. Wang, Y. Yu, P. Luo, "Scale-Equivalent Distillation for Semi-Supervised Object Detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14522-14531, 2022.
F. Guo, J. Zhao, P. Jiang, "Surface Defects Detection of Steel Plate Based on Visual Attention Mechanism," 2017 Chinese Automation Congress (CAC), pp. 3174-3177, 2017.
K. He, X. Zhang, S. Ren, J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," Advances in Neural Information Processing Systems Vol. 28 (NIPS 2015), 2015.
S. Wang, X. Xia, L. Ye, B. Yang, "Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks." Metals, Vol. 11, No. 3, pp. 388, 2021.

상세보기
D. Amin, S. Akhter, "Deep Learning-Based Defect Detection System in Steel Sheet Surfaces," 2020 IEEE Region 10 Symposium (TENSYMP), pp. 444-448, 2020,
P. Damacharla, A. R. M. V., J. Ringenberg, A. Y. Javaid, "TLU-Net: A Deep Learning Approach for Automatic Steel Surface Defect Detection," 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pp. 1-6, 2021.
A. M. Deshpande, A. A. Minai, M. Kumar, "One-Shot Recognition of Manufacturing Defects in Steel Surfaces," Procedia Manufacturing, Vol. 48, pp. 1064-1071, 2020.

상세보기
J. Bozic, D. Tabernik, D. Skocaj, "Mixed Supervision for Surface-defect Detection: from Weakly to Fully Supervised Learning," Computers in Industry, Vol. 129, 103459, 2021.

상세보기
Wikipedia, "Jensen-Shannon Divergence", see https://en.wikipedia.org/wiki/Jensen-Shannon_divergence.
L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," Proceedings of the European Conference on Computer Vision (ECCV), pp. 801-818, 2018.
J. Hu, L. Shen, S. Albanie, G. Sun, E. Wu, "Squeeze-and-Excitation Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
Ultralytics, "YOLOv5 v5.0 YOLOv5-P6 models," see https://github.com/ultralytics/yolov5/releases, 2021.
T. Y. Lin, P.a Goyal, R. Girshick, K. He, P. Dollar, "Focal Loss for Dense Object Detection," Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988, 2017.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증