[논문]다중 공간정보 데이터의 점진적 조합에 의한 의미적 분류 딥러닝 모델 학습 성능 분석

이대건; 신영하; 이동천

doi:10.7848/ksgpc.2022.40.2.91

[국내논문] 다중 공간정보 데이터의 점진적 조합에 의한 의미적 분류 딥러닝 모델 학습 성능 분석
Training Performance Analysis of Semantic Segmentation Deep Learning Model by Progressive Combining Multi-modal Spatial Information Datasets 원문보기

한국측량학회지 = Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, v.40 no.2, 2022년, pp.91 - 108

이대건 (AI Studio Lab, INFINIQ.) , 신영하 (Dept. of Geoinformation Engineering, Sejong University) , 이동천 (Dept. of Environment, Energy & Geoinformatics, Sejong University)

초록
AI-Helper

대부분의 경우 광학 RGB 영상을 딥러닝(DL: Deep learning)의 학습 데이터로 사용하여 객체탐지, 인식, 식별, 분류, 의미적 분할 및 객체 분할 등을 수행하지만, 실세계의 3차원 객체들을 2차원 영상으로 완전하게 파악하는 것은 한계가 있다. 그러므로 대표적인 3차원 지형 공간정보인 수치표면모델(DSM: Digital Surface Model)과 더불어 DSM에 내재된 특성정보를 이용하여 3차원 지형지물을 분석하는 것이 효과적이다. 건물과 같이 기하학적으로 정형화된 형태의 인공구조물은 3차원 공간데이터로부터 얻을 수 있는 기하학적 요소와 특성을 이용하여 객체의 분류와 형상 묘사가 가능하다. 이 연구는 고차원 시각정보(high-level visual information) 시스템에서 중요한 역할을 하는 내재된 고유의 특성정보(intrinsic information)를 기반으로 하며, 이를 위하여 객체의 기하학적 요소인 경사와 주향을 DSM으로부터 도출하고, 다방향에서 생성한 음영기복영상(SRI: Shaded Relief Image)과 함께 DL 모델의 학습 수행에 사용하였다. 실험은 ISPRS (International Society for Photogrammetry and Remote Sensing)에서 제공하는 데이터 셋 중에서 DSM과 레이블 데이터를 객체의 의미적 분류를 위해 개발된 합성곱 기반의 SegNet 학습에 사용하였다. 지형지물을 분류하고 분류 결과를 이용하여 건물을 추출하였다. 특히 DL 모델의 학습 성능 향상을 위해 학습 데이터의 여러 조합에 따른 시너지 효과를 분석하는 것에 핵심이다. 제안한 방법은 건물 분류와 추출에 효과적임을 보여주고 있다.

Abstract ▼ AI-Helper

In most cases, optical images have been used as training data of DL (Deep Learning) models for object detection, recognition, identification, classification, semantic segmentation, and instance segmentation. However, properties of 3D objects in the real-world could not be fully explored with 2D images. One of the major sources of the 3D geospatial information is DSM (Digital Surface Model). In this matter, characteristic information derived from DSM would be effective to analyze 3D terrain features. Especially, man-made objects such as buildings having geometrically unique shape could be described by geometric elements that are obtained from 3D geospatial data. The background and motivation of this paper were drawn from concept of the intrinsic image that is involved in high-level visual information processing. This paper aims to extract buildings after classifying terrain features by training DL model with DSM-derived information including slope, aspect, and SRI (Shaded Relief Image). The experiments were carried out using DSM and label dataset provided by ISPRS (International Society for Photogrammetry and Remote Sensing) for CNN-based SegNet model. In particular, experiments focus on combining multi-source information to improve training performance and synergistic effect of the DL model. The results demonstrate that buildings were effectively classified and extracted by the proposed approach.

Keyword

표/그림 (26)

그림 Fig. 1. Vaihingen datasets for DL
그림 Fig. 2. Conﬁguration of datasets with IR image, DSM, and label data of Vaihingen area
그림 Fig. 3. DSM of test areas
그림 Fig 4. Surface normal vectors (red arrows) of DSM (Part of Test Area 1)
그림 Fig. 5. Visualization of slope, aspect and combining slope and aspect of Test Area 1
그림 Fig. 6. Visualization of combining slope and aspect of test areas
그림 Fig. 7. Generation of multi-directional SRIs from DSM with different locations of light source and combined SRI
그림 Fig. 8. Combined multi-directional SRI of test areas
그림 Fig. 9. Characteristics of DSM, slope and aspect, and multi-directional SRI with different roof type
그림 Fig. 10. Dataset partition for DL model training
그림 Fig. 11. Beneﬁt of training data partition (Center, up, down, left, and right images are overlapped)
그림 Fig. 12. Workﬂow of proposed approach and training strategy
그림 Fig. 13. Implementation of transfer learning
그림 Fig. 14. Label data of test areas
그림 Fig. 15. Graphs of accuracy and loss for epoch
그림 Fig. 16. Classiﬁcation results using individual datasets
그림 Fig. 17. Classiﬁcation results using progressively combined datasets
그림 Fig. 18. Graphical representation of classiﬁcation accuracy of each category for test areas
표 Table 1. Classiﬁcation accuracy of each category for test areas
그림 Fig. 19. Overall and building accuracy using individual datasets
그림 Fig. 20. Overall and building accuracy using progressively combined datasets
그림 Fig. 21. Difference between label data and classiﬁcation with combining all datasets, and overall accuracy
그림 Fig. 22. Selected areas to evaluate building detection
그림 Fig. 23. Detail view of classiﬁcation results
그림 Fig. 24. Building extraction using semantic segmentation by DL
그림 Fig. 25. Detail view of building extraction

참고문헌 (39)

Audebert, N., Le Saux, B., and Lefevre, S. (2018), Beyond RGB: very high resolution urban remote sensing with multimodal deep networks, ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 140, pp. 20-32.

상세보기
Badrinarayanan, V., Kendall, A., and Cipolla, R. (2017), SegNet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions On Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495.

상세보기
Ballard, D. and Brown, C. (1982), Computer Vision, Prentice-Hall, Inc., Englewood Cliffs, NJ, 523p.
Bronshtein, A. (2017), Train/test split and cross validation in Python, https://towardsdatascience.com/train-test-split-andcross-validation-in-python-80b61beca4b6 (last date accessed: 30 August 2020).
Chen, K., Weinmann, M., Gao1, X., Yan, M., Hinz, S., Jutzi, B., and Weinmann, M. (2018), Residual shuffling convolutional neural networks for deep semantic image segmentation using multi-modal data, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4-7 June, Riva del Garda, Italy, pp. 65-72.
Cheng, W., Yang, W., Wang, M., Wang, G., and Chen, J. (2019), Context aggregation network for semantic labeling in aerial images, Remote Sensing, Vol. 11, No. 10, pp. 1-19.
Cho, E. and Lee, D.C. (2020), Building detection by convolutional neural network with infrared image, LiDAR data and characteristic information fusion, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 38, No. 6, pp. 635-644. (in Korean with English abstract)
Cramer, M. (2010), The DGPF test on digital aerial camera evaluation: Overview and test design. Photogrammetrie, Fernerkundung, Geoinformation, Vol. 2, pp. 73-82.
Goodfellow, I., Bengio, Y., and Courville, A. (2016), Deep Learning, The MIT Press, Cambridge, MA, 775p.
Horn, B.K. (1981), Hill shading and the reflectance map, Proceedings of the IEEE, Vol. 69, No. 1, pp. 14-47.

상세보기
Kim, J. and Bathe, K. (2013), The finite element method enriched by interpolation covers, Computers and Structures, Vol. 116, pp. 35-49.

상세보기
Krizhevsky, A., Sutskever, I., and Hinton, G. (2017), ImageNet classification with deep convolutional neural networks, Communications of the ACM, Vol. 60, No. 6, pp. 84-90.

상세보기
Laurini, R. and Thompson, D. (1998), Fundamentals of Spatial Information Systems, Academic Press, London, 680p.
Lee, D., Cho, E., and Lee, D.C. (2019), Semantic classification of DSM using convolutional neural network based deep learning, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 2-3 July, Korea, pp. 93-96. (in Korean with English abstract)
Lee, D., Shin, Y., and Lee, D.C. (2020), Land cover classification using SegNet with slope, aspect and multi-directional shaded relief images derived from digital surface model, Journal of Sensors, Vol. 2020, pp. 1-19.

상세보기
Lee, D.C., Lee, D.H., and Lee, D. (2019), Determination of building model key points using multidirectional shaded relief images generated from airborne LiDAR data, Journal of Sensors, Vol. 2019, pp. 1-19.

상세보기
Lemaire, C. (2008), Aspects of the DSM production with high resolution images, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 37, Part B4, pp. 1143-1146.
Lillesand, T., Kiefer, R., and Chipman, J. (2004), Remote Sensing and Image Interpretation - 5th edition, John Wiley & Sons, New York, NY, 763p.
Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L., F. Li, Yuille1, A., Huang, J., and Murphy, K. (2018), Progressive neural architecture search, Computer Vision - ECCV 2018, pp. 19-35.
Macko, V., Weill, C., Mazzawi, H., and Gonzalvo, J. (2019), Improving neural architecture search image classifiers via ensemble learning, arXiv:1903.06236v1.
Maltezos, E., Doulamis, A, Doulamis, N., and Ioannidis, C. (2019), Building extraction from LiDAR data applying deep convolutional neural networks, IEEE Geoscience and Remote Sensing Letters, Vol. 16, No. 1, pp. 155-159.

상세보기
Maune, D., Kopp, S., Crawford, C., and Zervas, C. (2007), Digital Elevation Model Technologies and Applications: The DEM Users Manual - 2nd edition, American Society for Photogrammetry and Remote Sensing, Bethesada, MD, 655p.
McCulloch, W. and Pitts, W. (1943), A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics, Vol. 7, pp. 115-133.
Meyer, F. and Beucher, S. (1990), Morphological segmentation, Journal of Visual Communication and Image Representation, Vol. 1, No. 1, pp. 21-46.

상세보기
Nahhas, F., Shafri, H., Sameen, M, Pradhan, B., and Mansor, S. (2018), Deep learning approach for building detection using LiDAR - orthophoto fusion, Journal of Sensors, Vol. 2020, No. 7, pp. 1-12.
Pibre, L., Chaumont, M., Subsol, G., Ienco, D., and Derras, M. (2017), How to deal with multi-source data for tree detection based on deep learning, IEEE Global Conference on Signal and Information Processing, pp. 1150-1154.
Prados, E. and Faugeras, O. (2006), Shape from Shading, In: Paragios, N., Chen, Y., and Faugeras, O. (eds.), Handbook of Mathematical Models in Computer Vision, Springer, New York, N.Y., pp. 375-403.
Rottensteiner, F., Sohn, G., Gerke, M., and Wegner, J. (2013), ISPRS test project on urban classification and 3D building reconstruction, ISPRS, http://www2.isprs.org/tl_files/isprs/wg34/docs/ComplexScenes_revision_v4.pdf (last date accessed: 30 July 2020).
Rusu, A., Rabinowitz, N., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsel, R. (2016), Progressive neural networks, arXiv:1606.04671v3.
Sander, R. (2020), Data Fusion and Class Imbalance Correction Techniques for Efficient Multi-Class Point Cloud Semantic Segmentation. https://www.researchgate.net/publication/339323048_Sparse_Data_Fusion_and_Class_Imbalance_Correction_Techniques_for_Efficient_Multi-Class_Point_Cloud_Semantic_Segmentation (last date accessed: 25 January 2022).
Sharada, P., Mohanty, S., Hughes, D., and Salathe, M. (2016), Using deep learning for image-based plant disease detection, Frontiers in Plant Science, doi: 10.3389/fpls.2016.01419, pp. 1-10.

상세보기
Shin, Y.H., Son, K.W., and Lee, D.C. (2022), Semantic segmentation and building extraction from airborne LiDAR data with multiple return using PointNet++, Applied Sciences, Vol. 12, No. 4, pp. 1-20.
Simonyan, K. and Zisserman, A. (2015), Very deep convolutional networks for large-scale image recognition, International Conference on Learning Representations, 7-9 May, San Diego, CA, USA, arXiv:1409.1556v6.
Speldekamp, T., Fries, C., Gevaert, C., and Gerke, M. (2015), Automatic semantic labelling of urban areas using a rulebased approach and realized with MeVisLab, https://www.researchgate.net/publication/275639040_Automatic_Semantic_Labelling_of_Urban_Areas_using_a_rule-based_approach_and_realized_with_MeVisLab (last date accessed: 13 August 2020).
Szeliski, R. (2011), Computer Vision: Algorithms and Applications, Springer-Verlag, London, U.K., 812p.
Tao, S. (2019), Deep neural network ensembles, arXiv:1904.05488v2.
Varney, N., Asari, V.K., and Graehling, Q. (2020), DALES: A large-scale aerial LiDAR data set for semantic segmentation, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14 April 2020.
Wang, X. and Wang, X. (2020), Spatiotemporal fusion of remote sensing image based on deep learning, Journal of Sensors, Vol. 2020, Article ID 8873079, pp. 1-11.

상세보기
Zhou, K., Ming, D., Lv, X., Fang, J., and Wang, M. (2019), CNN-based land cover classification combining stratified segmentation and fusion of point cloud and very high-spatial resolution remote sensing image data, Remote Sensing, Vol. 2019, No. 11, pp. 1-28.

저자의 다른 논문 :

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증