[논문]웨어러블 AR 기기를 이용한 객체인식 기반의 건설 현장 정보 시각화 구현

이용주; 박만우; 송은석

doi:10.13161/kibim.2021.11.3.045

[국내논문] 웨어러블 AR 기기를 이용한 객체인식 기반의 건설 현장 정보 시각화 구현
Augmented Reality Framework to Visualize Information about Construction Resources Based on Object Detection 원문보기

Journal of KIBIM = 한국BIM학회논문집, v.11 no.3, 2021년, pp.45 - 54

(명지대학교 토목환경공학과) , (명지대학교 토목환경공학과) , 이용주 (명지대학교 토목환경공학과) , 박만우 (명지대학교 토목환경공학과) , 송은석 (한국도로공사 스마트건설사업단)

Abstract ▼ AI-Helper

The augmented reality (AR) has recently became an attractive technology in construction industry, which can play a critical role in realizing smart construction concepts. The AR has a great potential to help construction workers access digitalized information about design and construction more flexibly and efficiently. Though several AR applications have been introduced for on-site made to enhance on-site and off-site tasks, few are utilized in actual construction fields. This paper proposes a new AR framework that provides on-site managers with an opportunity to easily access the information about construction resources such as workers and equipment. The framework records videos with the camera installed on a wearable AR device and streams the video in a server equipped with high-performance processors, which runs an object detection algorithm on the streamed video in real time. The detection results are sent back to the AR device so that menu buttons are visualized on the detected objects in the user's view. A user is allowed to access the information about a worker or equipment appeared in one's view, by touching the menu button visualized on the resource. This paper details implementing parts of the framework, which requires the data transmission between the AR device and the server. It also discusses thoroughly about accompanied issues and the feasibility of the proposed framework.

Keyword

표/그림 (9)

그림 Figure 1. AR framework for visualizing the information of on-site resources (materials, worker, and equipment)
그림 Figure 2. Data communication between Hololens 2 and PC
그림 Figure 3. Localization of a menu button in 3D
그림 Figure 4. Visualization of a button on each detected worker in the Hololens 2 user's view
표 Table 1. Time delay caused when using mixed reality streaming or WebRTC
표 Table 2. Performance of YOLOv3 and YOLOv4 trained with the ACID dataset
표 Table 3. Displacement of the indicator depending on the distance of a worker from the Hololens 2 user
표 Table 4. Displacement of the indicator depending on the walking speed of a worker
그림 Figure 5. Required processing time from capturing a frame to visualizing a button (500 ms total)

참고문헌 (26)

Asadi, K., Chen, P., Han, K., Wu, T., Lobaton, E. (2019). Real-time Scene Segmentation Using a Light Deep Neural Network Architecture for Autonomous Robot Navigation on Construction Sites, The 2019 ASCE International Conference on Computing in Civil Engineering, arXiv:1901.08630 [cs.RO], https://arxiv.org/abs/1901.08630.
Behringer, R., Klinker, G., Mizell, D. (1999). Augmented Reality: Placing Artificial Objects in Real Scenes, CRC Press.
Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y. M. (2020). YOLOv4: Optimal Speed and Accu racy of Object Detection, Computer Vision and Pattern Recognition, arXiv:2004.10934 [cs.CV], https://arxiv.org/abs/2004.10934.
Christine, B. (2019). Bentley's SYNCHRO XR and Microsoft HoloLens 2 bring the benefits of mixed reality to construction sites, https://www.bentley.com/ko/about-us/news/2019/february/24/synchro-xr-with-hololens-2-release (May. 21. 2021).
Elsbach, K. D., Kramer, R. M. (2015). Handbook of Qualitative Organizational Research: Innovative Pathways and Methods, Routledge.
Fang, Q., Li, H., Luo, X., Ding, L., Rose, T. M., An, W., Yu, Y. (2018). A deep learning-based method for detecting non-certified work on construction sites, Advanced Engineering Informatics, 35, pp. 56-68.

상세보기
Fang, W., Ding, L., Zhong, B., Love, P. E.D., Luo, H. (2018). Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach, Advanced Engineering Informatics, 37, pp. 139-149.

상세보기
Fang, Y., Ding, L., Luo, H., Love, P. E.D. (2018). Falls from heights: A computer vision-based approach for safety harness detection, Automation in Construction, 91, pp. 53-61.

상세보기
Girshick, R., Donahue, J., Darrell, T., Malik., J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 580-587.
Ippolito, B. (2018). simplejson - JSON encoder and decoder, https://simplejson.readthedocs.io/en/latest/ (May. 06. 2021).
Jiang, Z., Zhao, L., Li, S., Jia, Y. (2020). Real-time object detection method based on improved YOLOv4-tiny, Computer Vision and Pattern Recognition, arXiv:2011.04244 [cs.CV], https://arxiv.org/abs/2011.04244.
Jiao, Y., Zhang, S., Li, Y., Wang, Y., Yang, B. (2013). Towards cloud Augmented Reality for construction application by BIM and SNS integration, Automation in Construction, 33, pp. 37-47.

상세보기
Kim, H., Bang, S., Jeong, H., Ham, Y., Kim, H. (2018). Analyzing context and productivity of tunnel earthmoving processes using imaging and simulation, Automation in Construction, 92, pp. 188-198.

상세보기
Kim, J., Sung, J.-Y., Park, S. (2020). Comparison of Faster-RCNN, YOLO, and SSD for Real-Time Vehicle Type Recognition, 2020 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia), 2020, pp. 1-4.
Lee, Y.-J., Kim, J.-Y., Pham, H., Park, M.-W. (2020). Augmented Reality Framework for Efficient Access to Schedule Information on Construction Sites, Journal of KIBIM, 10(4), pp. 60-69.

원문보기 상세보기
Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., Dollar, P. (2014). Microsoft COCO: Common Objects in Context, ECCV 2014: Computer Vision - ECCV 2014, pp. 740-755.
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., Berg, A. C. (2016). SSD: Single Shot MultiBox Detector, Computer Vision - ECCV 2016, pp. 21-37.
Luo, H., Xiong, C., Fang, W., Love, P. E.D., Zhang, B., Ouyang, X. (2018). Convolutional neural networks: Computer vision-based workforce activity assessment in construction, Automation in Construction, 94, pp. 282-289.

상세보기
Microsoft. (2019). Introducing MRTK for Unity, https://docs.microsoft.com/ko-kr/windows/mixed-reality/develop/unity/mrtk-getting-started (May. 04. 2021).
Redmon, J., Divvala, S., Girshick, S., Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788.
Ren, S., He, K., Girshick, R., Sun, Jian. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), pp. 1137-1149.

상세보기
Sredojev, B., Samardzija, D., Posarac, D. (2015). WebRTC technology overview and signaling solution design and implementation, 38th International Convention on Information and Communication Technology Electronics and Microelectronics (MIPRO), 2015, pp. 1006-1009.
vGis. (2021). BIM and GIS Data In Augmented Reality, https://www.vgis.io/esri-augmented-reality-gis-ar-for-utilities-municipalities-locate-and-municipal-service-companies/ (May. 22. 2021).
Xiao, B., Kang, S.-C. (2021). Development of an Image Data Set of Construction Machines for Deep Learning Object Detection, Journal of Computing in Civil Engineering, 35(2), https://doi.org/10.1061/(ASCE)CP.1943-5487.0000945.

상세보기
Yang, Z., Yuan, Y., Zhang, M., Zhao, X., Zhang, Y., Tian, B. (2019). Safety Distance Identification for Crane Drivers Based on Mask R-CNN, Sensors, 19(12), https://doi.org/10.3390/s19122789.

상세보기
Zollmann, S., Hoppe, C., Kluckner, S., Poglitsch, C., Bischof, H., Reitmayr, G. (2014). Augmented Reality for Construction Site Monitoring and Documentation, Proceedings of the IEEE, 102(2), pp. 137-154.

상세보기

저자의 다른 논문 :

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증