[논문]건설 현장 CCTV 영상을 이용한 작업자와 중장비 추출 및 다중 객체 추적

조영운; 강경수; 손보식; 류한국

doi:10.5345/jkibc.2021.21.5.397

건설 현장 CCTV 영상을 이용한 작업자와 중장비 추출 및 다중 객체 추적
Extraction of Workers and Heavy Equipment and Muliti-Object Tracking using Surveillance System in Construction Sites 원문보기

한국건축시공학회지 = Journal of the Korea Institute of Building Construction, v.21 no.5, 2021년, pp.397 - 408

조영운 (Construction Engineering and Management Institute, Sahmyook University) , 강경수 (Construction Engineering and Management Institute, Sahmyook University) , 손보식 (Department of Architectural Engineering, Namseoul University) , 류한국 (Department of Architectural, Sahmyook University)

초록
AI-Helper

건설업은 업무상 재해 발생빈도와 사망자 수가 다른 산업군에 비해 높아 가장 위험한 산업군으로 불린다. 정부는 건설 현장에서 발생하는 산업 재해를 줄이고 예방하기 위해 CCTV 설치 의무화를 발표했다. 건설 현장의 안전 관리자는 CCTV 관제를 통해 현장의 잠재된 위험성을 찾아 제거하고 재해를 예방한다. 하지만 장시간 관제 업무는 피로도가 매우 높아 중요한 상황을 놓치는 경우가 많다. 따라서 본 연구는 딥러닝 기반 컴퓨터 비전 모형 중 개체 분할인 YOLACT와 다중 객체 추적 기법인 SORT을 적용하여 다중 클래스 다중 객체 추적 시스템을 개발하였다. 건설 현장에서 촬영한 영상으로 제안한 방법론의 성능을 MS COCO와 MOT 평가지표로 평가하였다. SORT는 YOLACT의 의존성이 높아서 작은 객체가 적은 데이터셋을 학습한 모형의 성능으로 먼 거리의 물체를 추적하는 성능이 떨어지지만, 크기가 큰 객체에서 뛰어난 성능을 나타냈다. 본 연구로 인해 딥러닝 기반 컴퓨터 비전 기법들의 안전 관제 업무에 보조 역할로 업무상 재해를 예방할 수 있을 것으로 판단된다.

Abstract ▼ AI-Helper

The construction industry has the highest occupational accidents/injuries and has experienced the most fatalities among entire industries. Korean government installed surveillance camera systems at construction sites to reduce occupational accident rates. Construction safety managers are monitoring potential hazards at the sites through surveillance system; however, the human capability of monitoring surveillance system with their own eyes has critical issues. A long-time monitoring surveillance system causes high physical fatigue and has limitations in grasping all accidents in real-time. Therefore, this study aims to build a deep learning-based safety monitoring system that can obtain information on the recognition, location, identification of workers and heavy equipment in the construction sites by applying multiple object tracking with instance segmentation. To evaluate the system's performance, we utilized the Microsoft common objects in context and the multiple object tracking challenge metrics. These results prove that it is optimal for efficiently automating monitoring surveillance system task at construction sites.

주제어

표/그림 (9)

그림 Figure 1. The Flow chart of our research
표 Table 1. Summary of construction dataset
표 Table 2. Summary of ground truth videos for evaluating multiple object tracking
그림 Figure 2. Flowchart of for multiple object tracking with instance segmentation at construction site
표 Table 3. Description of MOT challenge evaluation metrics
그림 Figure 3. Examples of YOLACT qualitative visual result (a) ProtoNet inference (b) failure results on test samples
그림 Figure 4. Multi-object tracking results at the first frame of the test videos (a) dump truck tracking result at wheel wash system (b) worker tracking at rebar workplace (c) Multi class multi object tracking results of concrete pouring work. The red colored object indicates tracking failure
표 Table 4. Average precision(AP) for each construction object category under different IoU threshold, and object sizes
표 Table 5. Result of multiple object tracking evaluation for each video clips derived from SORT

참고문헌 (39)

Kim D. Occupational accident/injury analysis 2009. Ulsan (Korea): Korea Occupational Safety and Health Agency; 2021 Jan;15-22. Grant No.: 118006 Supported by KOSTAT.
Kim H. Construction safety innovation plan: Reinforcement of management of vulnerable construction, etc [Internet]. Sejong (Korea): Ministry of Land, Infrastructure and Transport. 2020 Apr 24 [cited 2021 Apr 7]. Available from: http://www.molit.go.kr/USR/NEWS/m_71/dtl.jsp?id95083805
Heejung. Women who "watch the monitor" [Internet]. Seoul (Korea): Ildaro. 2019 Aug 30 [cited 2021 Apr 7]. Available from: https://ildaro.com/8536
Park Y. Only one person monitors 438 CCTVs [Internet]. Seoul (Korea): Munhwa Ilbo. 2017 Nov 28 [cited 2021 Apr 7]. Available from: http://www.munhwa.com/news/view.html?no2017112801031627109001
Choi M, Choi J. CCTV integrated control center operation status and improvement plan legislative policy report. Seoul, Korea: National Assembly Research Service, NARS; 2019. p. 1-33.
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD. Backpropagation applied to handwritten zip code recognition. Neural computation. 1989 Dec;1(4):541-51. https://doi.org/10.1162/neco.1989.1.4.541

상세보기
Lee YJ, Park MW. 3D tracking of multiple onsite workers based on stereo vision. Automation in Construction. 2019 Feb;98:146-59. https://doi.org/10.1016/j.autcon.2018.11.017

상세보기
Dalal N, Triggs B. Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 2005 Jun 20-25; San Diego, CA. NJ: Institute of Electrical and Electronics Engineers; 2005. p. 886-93. https://doi.org/10.1109/CVPR.2005.177
Park MW, Brilakis I. Continuous localization of construction workers via integration of detection and tracking. Automation in Construction. 2016 Dec;72:129-42. https://doi.org/10.1016/j.autcon.2016.08.039

상세보기
Zhang Z. Determining the epipolar geometry and its uncertainty: A review. International journal of computer vision. 1998 Mar;27(2):161-95. https://doi.org/10.1023/A:1007941100561
Zhao Y, Chen Q, Cao W, Yang J, Xiong J, Gui G. Deep learning for risk detection and trajectory tracking at construction sites. IEEE Access; 2019 Mar;7:30905-12. https://doi.org/10.1109/ACCESS.2019.2902658

상세보기
Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv:1804:02767 [Preprint]. 2018 [cited 2021 Apr 8]. Available from: https://arxiv.org/abs/1804.02767
Kalman RE. A new approach to linear filtering and prediction problems. 1960 Mar;82(1):35-45. https://doi.org/10.1115/1.3662552

상세보기
Kuhn HW. The Hungarian method for the assignment problem. Naval research logistics quarterly. 1955 Mar;2(1-2):83-97. https://doi.org/10.1002/nav.3800020109

상세보기
Ishioka H, Weng X, Man Y, Kitani K. Single camera worker detection, tracking and action recognition in construction site. Proceedings of the 37th International Symposium on Automation and Robotics in Construction (ISARC); 2020 Oct; Kitakyushu, Japan. FL: International Association for Automation and Robotics in Construction (IAARC); 2020. p. 653-60. https://doi.org/10.22260/ISARC2020/0092
Angah O, Chen AY. Tracking multiple construction workers through deep learning and the gradient based method with rematching based on multi-object tracking accuracy. Automation in Construction. 2020 Nov;119:103308. https://doi.org/10.1016/j.autcon.2020.103308

상세보기
He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV); 2017 Oct 22-29; Venice, Italy. NJ: Institute of Electrical and Electronics Engineers; 2017. p. 2961-9. https://doi.org/10.1109/ICCV.2017.322
Nath ND, Behzadan AH, Paal SG. Deep learning for site safety: Real-time detection of personal protective equipment. Automation in Construction. 2020 Apr;112:103085. https://doi.org/10.1016/j.autcon.2020.103085

상세보기
Son H, Choi H, Seong H, Kim C. Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks. Automation in Construction. 2019 Mar;99:27-38. https://doi.org/10.1016/j.autcon.2018.11.033

상세보기
Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence. 2017 Jun;39(6):1137-49. https://doi.org/10.1109/TPAMI.2016.2577031

상세보기
Guo Y, Xu Y, Li S. Dense construction vehicle detection based on orientation-aware feature fusion convolutional neural network. Automation in Construction. 2020 Apr;112:103124. https://doi.org/10.1016/j.autcon.2020.103124

상세보기
Li Z, Zhou F. FSSD: feature fusion single shot multibox detector. arXiv:1712.00960 [Preprint]. 2017 [cited 2021 Apr 12]. Available from: https://arxiv.org/abs/1712.00960
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015 Oct 5; Munich, Germany. MN: The Medical Image Computing and Computer Assisted Intervention Society; 2015. p. 234-41. https://doi.org/10.1007/978-3-319-24574-4_28
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017 Apr;39(4): 640-51. https://doi.org/10.1109/TPAMI.2016.2572683

상세보기
Truong T, Bhatt A, Queiroz L, Lai K, Yanushkevich S. Instance segmentation of personal protective equipment using a multi-stage transfer learning process. 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2020 Oct 11-14; Toronto, Canada. NJ: Institute of Electrical and Electronics Engineers; 2017. p.1181-6. https://doi.org/10.1109/SMC42975.2020.9283427
Yang Z, Yuan Y, Zhang M, Zhao X, Zhang Y, Tian B. Safety distance identification for crane drivers based on mask R-CNN. Sensors. 2019 Jan;19(12):2789. https://doi.org/10.3390/s19122789

상세보기
GitHub: Where the world builds software [Internet]. Image Polygonal Annotation with Python: GitHub, Inc. 2008 - [cited 2021 Apr 7]. Available from: https://github.com/wkentaro/labelme
Bolya D, Zhou C, Xiao F, Lee YJ. Yolact: Real-time instance segmentation. 2019 IEEE/CVF International Conference on Computer Vision(ICCV). 2019 Oct 27-Nov 2; Seoul, Korea. NJ: Institute of Electrical and Electronics Engineers; 2020. p.9157-66. https://doi.org/10.1109/ICCV.2019.00925
Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2017 Jul 21-26; Honolulu, HI. NJ: Institute of Electrical and Electronics Engineers; 2017. p. 2117-25. https://doi.org/10.1109/CVPR.2017.106
Voigtlaender P, Krause M, Osep A, Luiten J, Sekar BB, Geiger A, Leibe B. Mots: Multi-object tracking and segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2019 June 15-20; Long Beach, CA. NJ: Institute of Electrical and Electronics Engineers; 2020. p. 7942-51. https://doi.org/10.1109/CVPR.2019.00813
Luo W, Xing J, Milan A, Zhang X, Liu W, Kim TK. Multiple object tracking: A literature review. Artificial Intelligence. 2021 Apr 293:103448. https://doi.org/10.1016/j.artint.2020.103448

상세보기
Bewley A, Ge Z, Ott L, Ramos F, Upcroft B. Simple online and realtime tracking. 2016 IEEE international conference on image processing(ICIP). 2016 Sept 25-28; Phoenix, AZ. NJ: Institute of Electrical and Electronics Engineers; 2016. p. 3464-8. https://doi.org/10.1109/ICIP.2016.7533003
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A. The pascal visual object classes (voc) challenge. International journal of computer vision. 2010 Jun;88(2):303-38. https://doi.org/10.1007/s11263-009-0275-4

상세보기
Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL. Microsoft coco: Common objects in context. European conference on computer vision. 2014 Sep;8693:740-55. https://doi.org/10.1007/978-3-319-10602-1_48

상세보기
Wojke N, Bewley A, Paulus D. Simple online and realtime tracking with a deep association metric. 2017 IEEE international conference on image processing(ICIP). 2017 Sep 17-20; Beijing, China. NJ: Institute of Electrical and Electronics Engineers; 2018. p. 3645-9. https://doi.org/10.1109/ICIP.2017.8296962
Leal-Taixe L, Milan A, Reid I, Roth S, Schindler K. Motchallenge 2015: Towards a benchmark for multi-target tracking. arXiv:1504.01942 [Preprint]. 2015 [cited 2021 Apr 8]. Available from: https://arxiv.org/abs/1504.01942
Milan A, Leal-Taixe L, Reid I, Roth S, Schindler K. MOT16: A benchmark for multi-object tracking. arXiv:1603.00831 [Preprint]. 2016 [cited 2021 Apr 8]. Available from: https://arxiv.org/abs/1603.00831
GitHub: Where the world builds software [Internet]. Deep learning-based Computer Vision Models for PyTorch: GitHub, Inc. 2008 - [cited 2021 Apr 7]. Available from: https://github.com/unerue/boda
Xuehui A, Li Z, Zuguang L, Chengzhi W, Pengfei L, Zhiwei L. Dataset and benchmark for detecting moving objects in construction sites. Automation in Construction. 2021 Feb;122:103482. https://doi.org/10.1016/j.autcon.2020.103482

상세보기

저자의 다른 논문 :

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증