The automatic defect sorting function of machinery parts is being introduced to the automation of the manufacturing process. In the final stage of automation of the manufacturing process, it is necessary to apply computer vision rather than human visual judgment to determine whether there is a defec...
The automatic defect sorting function of machinery parts is being introduced to the automation of the manufacturing process. In the final stage of automation of the manufacturing process, it is necessary to apply computer vision rather than human visual judgment to determine whether there is a defect. In this paper, we introduce a deep learning method to improve the classification performance of typical mechanical parts, such as welding parts, galvanized round plugs, and electro galvanized nuts, based on the results of experiments. In the case of poor welding, the method to further increase the depth of layer of the basic deep learning model was effective, and in the case of a circular plug, the surrounding data outside the defective target area affected it, so it could be solved through an appropriate pre-processing technique. Finally, in the case of a nut plated with zinc, since it receives data from multiple cameras due to its three-dimensional structure, it is greatly affected by lighting and has a problem in that it also affects the background image. To solve this problem, methods such as two-dimensional connectivity were applied in the object segmentation preprocessing process. Although the experiments suggested that the proposed methods are effective, most of the provided good/defective images data sets are relatively small, which may cause a learning balance problem of the deep learning model, so we plan to secure more data in the future.
The automatic defect sorting function of machinery parts is being introduced to the automation of the manufacturing process. In the final stage of automation of the manufacturing process, it is necessary to apply computer vision rather than human visual judgment to determine whether there is a defect. In this paper, we introduce a deep learning method to improve the classification performance of typical mechanical parts, such as welding parts, galvanized round plugs, and electro galvanized nuts, based on the results of experiments. In the case of poor welding, the method to further increase the depth of layer of the basic deep learning model was effective, and in the case of a circular plug, the surrounding data outside the defective target area affected it, so it could be solved through an appropriate pre-processing technique. Finally, in the case of a nut plated with zinc, since it receives data from multiple cameras due to its three-dimensional structure, it is greatly affected by lighting and has a problem in that it also affects the background image. To solve this problem, methods such as two-dimensional connectivity were applied in the object segmentation preprocessing process. Although the experiments suggested that the proposed methods are effective, most of the provided good/defective images data sets are relatively small, which may cause a learning balance problem of the deep learning model, so we plan to secure more data in the future.
As is well known, the kernel size of VGG-16 is fixed to 3×3, but the depth of the layer may be insufficient to determine welding defects thus a comparative experiment was carried out by expanding to 7×7, 9×9, 11×11[13].
Based on these data, a comparison experiment was performed with a different kernel size affecting the defect detection performance targeting “VGG-16”, a representative deep learning model[12]
대상 데이터
The images used in the experiment are images of electroplated nuts used as parts of mechanical devices, and there are a total of 367 images, of which 180 are of normal conditions and 187 are of abnormal conditions. The ratios of images used for training, validation, and testing (inference) are shown in Table 1 as 60%, 20%, and 20%, respectively.
2 is sample photos of normal and abnormal conditions related to welding. The provided images were 53 normal models and 47 abnormal models. Since the amount of data is relatively small, these images were rotated by 90, 180, and 270 degrees respectively, and the results including the original image were symmetrical and 424 normal models expanded to 376 abnormal models and built a data set of 800 in total.
성능/효과
the performance of normal condition is lower than when pre-processing is applied, especially when all four cameras including the floor are used, the result is not decisive at all. And in the case of using all four images with a white background in the proposed method, the accuracy of both abnormal and normal condition identification is more than 90%, and the accuracy is similar to each other. And the best performance is when only the floor image excluding the alpha value is determined as a target, but since defects may occur on surfaces other than the floor surface, determining only the floor surface has a problem in that it is not realistically valid.
And in the case of using all four images with a white background in the proposed method, the accuracy of both abnormal and normal condition identification is more than 90%, and the accuracy is similar to each other. And the best performance is when only the floor image excluding the alpha value is determined as a target, but since defects may occur on surfaces other than the floor surface, determining only the floor surface has a problem in that it is not realistically valid.
We concluded that a strategy is needed to prevent unnecessary parts from being included as feature values in the convolutional network of the deep learning model. Finally, in the case of the hexagon nut, it was concluded that the pre-processing technique that removes the effect of lighting, etc. from images acquired from multiple cameras from multiple angles was effective due to the 3D shape.
However, among them, we introduced based on the contents of experiments conducted in conjunction with image-based and deep learning methods, focusing on poor welding between metal parts, poor plating of circular plugs in automobile steering system, and poor plating of hexagon nuts. Fine welding defects occurring in welding between metal parts showed better discrimination performance when the deep learning model enlarged the kernel size and gave the effect of composing a deeper layer. In the case of a circular plug, when the round border outside the minimum inner diameter is removed by a preprocessing method using a Circle Hough Transform and linked with a deep learning model, it shows better performance than when linked with a deep learning model while leaving the round border as it is.
In this paper, there are many types of product groups of machine parts and various types of defects appearing in these product groups. However, among them, we introduced based on the contents of experiments conducted in conjunction with image-based and deep learning methods, focusing on poor welding between metal parts, poor plating of circular plugs in automobile steering system, and poor plating of hexagon nuts. Fine welding defects occurring in welding between metal parts showed better discrimination performance when the deep learning model enlarged the kernel size and gave the effect of composing a deeper layer.
Fine welding defects occurring in welding between metal parts showed better discrimination performance when the deep learning model enlarged the kernel size and gave the effect of composing a deeper layer. In the case of a circular plug, when the round border outside the minimum inner diameter is removed by a preprocessing method using a Circle Hough Transform and linked with a deep learning model, it shows better performance than when linked with a deep learning model while leaving the round border as it is. We concluded that a strategy is needed to prevent unnecessary parts from being included as feature values in the convolutional network of the deep learning model.
Recently, automation of the manufacturing process is rapidly spreading centering on smart factories, and in the manufacturing field, the automatic selection of abnormal conditions, especially for mechanical parts, is included in the automation of the manufacturing process. Most of the mechanical parts have been classified as abnormal conditions by hand, but the rate of misclassification of normal conditions into abnormal conditions or vice versa was not low. In this paper, there are numerous types of product groups of mechanical parts and various types of defects appearing in these product groups, but among them, focusing on the examples directly implemented through experiments on the method of determining the metal parts welding defect, the circular plug plating defect in the automobile steering system, and the plating defect of hexagon nuts in connection with the image-based and deep learning methods will be introduced.
Since the amount of data is relatively small, these images were rotated by 90, 180, and 270 degrees respectively, and the results including the original image were symmetrical and 424 normal models expanded to 376 abnormal models and built a data set of 800 in total. Of these, 400 were used for deep learning training, 200 were used for verification, and the remaining 200 for testing, of the 200, 94 are normal and 106 are abnormal. Based on these data, a comparison experiment was performed with a different kernel size affecting the defect detection performance targeting “VGG-16”, a representative deep learning model[12].
The provided images were 53 normal models and 47 abnormal models. Since the amount of data is relatively small, these images were rotated by 90, 180, and 270 degrees respectively, and the results including the original image were symmetrical and 424 normal models expanded to 376 abnormal models and built a data set of 800 in total. Of these, 400 were used for deep learning training, 200 were used for verification, and the remaining 200 for testing, of the 200, 94 are normal and 106 are abnormal.
In the object segmentation preprocessing process, a nut is segmented and detected using the connectivity information of 2D pixels in the steps of converting a gray scale image to a binary image and in the step of extracting the object[6]. Since the separately detected nut is a divided nut object that does not include a background, performance can be improved by reflecting only on the nut region that does not include a background in the training and inference. It compares and analyzes performance by applying various deep learning to the segmented and detected objects.
The kernel size was 11×11, we found it to be the best in the determination of abnormal conditions, we also confirmed that there was little difference compared to the case in which the kernel size was small
The kernel size was 11×11, we found it to be the best in the determination of abnormal conditions, we also confirmed that there was little difference compared to the case in which the kernel size was small. The reason why the kernel size had an effect in the case of abnormal condition determination is that the expansion to a deeper layer through an increase in the kernel size has some effect in extracting the features of fine defects.
In the case of a circular plug, when the round border outside the minimum inner diameter is removed by a preprocessing method using a Circle Hough Transform and linked with a deep learning model, it shows better performance than when linked with a deep learning model while leaving the round border as it is. We concluded that a strategy is needed to prevent unnecessary parts from being included as feature values in the convolutional network of the deep learning model. Finally, in the case of the hexagon nut, it was concluded that the pre-processing technique that removes the effect of lighting, etc.
Where, TP (True Positive) is the number of images detected or recognized as True for the correct answer that is actually True, FN (False Negative) is the number of images detected or recognized as False for an actual True answer, FP (False Positive) is the number of images detected or recognized as True for a correct answer that is actually False, TN (True Negative) is the number of images detected or recognized as False for a correct answer that is actually False[14].
In Table 2, in the case where the preprocessing process was not applied and in the comparative experiment, the abnormal condition showed similar performance. the performance of normal condition is lower than when pre-processing is applied, especially when all four cameras including the floor are used, the result is not decisive at all. And in the case of using all four images with a white background in the proposed method, the accuracy of both abnormal and normal condition identification is more than 90%, and the accuracy is similar to each other.
후속연구
However, most of the images of normal or abnormal conditions provided by demanding companies are relatively small, which can cause problems with the learning balance of the deep learning model. Based on the sufficiently secured data in the future, we intend to conduct more reliable research and more objective performance evaluation.
Most of the mechanical parts have been classified as abnormal conditions by hand, but the rate of misclassification of normal conditions into abnormal conditions or vice versa was not low. In this paper, there are numerous types of product groups of mechanical parts and various types of defects appearing in these product groups, but among them, focusing on the examples directly implemented through experiments on the method of determining the metal parts welding defect, the circular plug plating defect in the automobile steering system, and the plating defect of hexagon nuts in connection with the image-based and deep learning methods will be introduced.
참고문헌 (14)
Hansen, E. B., and Bogh, Sg, "Artificial intelligence and internet of things in small and medium-sized enterprises: A survey," Journal of Manufacturing Systems, Vol. 58, No. 2, pp. 368-372, 2021.
Zhao, Yong Jie, et al., "Automatic and accurate measurement of microhardness profile based on image processing," IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-9, 2021.
Pang, Guansong, et al. "Deep learning for anomaly detection: A review". ACM Computing Surveys (CSUR), vol. 52, no. 2, pp. 1-38, 2021.
L. Liu, Ouyang, W. Wang, X. Fieguth, P. Chen, J. Liu, and M. Pietikainen, "Deep learning for generic object detection: A survey." International journal of computer vision, Vol.128, pp. 261-318, 2019.
M. Al-Garadi, A. Mohamed, A. Al-Ali, A. Du, X. Ali, and M. Guizani, "A survey of machine and deep learning methods for internet of things (IoT) security." IEEE Communications Surveys & Tutorials, 2020.
Cruttwell, Geoffrey SH, et al., "Categorical foundations of gradient-based learning," European Symposium on Programming, pp. 1-28, 2022.
D. Park and K. Jun, "CHT-based Automatic Go Recording System under Illumination Change and Stone Dislocation," Journal of the Korean Institute of Imformation Scientists and Engineers, vol. 41, no. 6, pp. 448-455, 2014.
Beyer, Lucas, et al., "Are we done with imagenet?.", arXiv preprint arXiv:2006.07159, 2020.
Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao., "Scaled-yolov4: Scaling cross stage partial network.". IProceedings of the IEEE/cvf conference on computer vision and pattern recognition. p.13029-13038, 2021.
Touvron, Hugo, et al. "Going deeper with image transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
H. J. Kang, "Efficient Fixed-Point Representation for ResNet-50 Convolutional Neural Network," Journal of the Korea Institute of Information and Communication Engineering, Korea, vol. 22, no. 1, pp. 1-8, May 2018.
Bai, X., Wang, X., Liu, X., Liu, Q., Song, J., Sebe, N., and Kim, B., "Explainable deep learning for efficient and robust pattern recognition: A survey of recent developments.", Pattern Recognition, vol. 120, 2021.
T. Jin, "Feature Extraction Using Convolutional Neural Networks for Random Translation," Journal of the Korean Society of Industry Convergence, vol. 23, no. 3, pp.515-521, Jun. 2020.
S.-Y. Choi, "A Study on the Automation of Cam Heat Treatment Process using Deep Learning," Journal of the Korean Society of Industry Convergence, vol. 23, no. 2_2, pp. 281-288, Apr. 2020.
AI-Helper
※ AI-Helper는 부적절한 답변을 할 수 있습니다.