[논문]Semantic Segmentation of Heterogeneous Unmanned Aerial Vehicle Datasets Using Combined Segmentation Network

Ahram, Song

doi:10.7780/kjrs.2023.39.1.6

Semantic Segmentation of Heterogeneous Unmanned Aerial Vehicle Datasets Using Combined Segmentation Network 원문보기

대한원격탐사학회지 = Korean journal of remote sensing, v.39 no.1, 2023년, pp.87 - 97

Ahram, Song (Department of Location-based Information System, Kyungpook National University)

Abstract ▼ AI-Helper

Unmanned aerial vehicles (UAVs) can capture high-resolution imagery from a variety of viewing angles and altitudes; they are generally limited to collecting images of small scenes from larger regions. To improve the utility of UAV-appropriated datasetsfor use with deep learning applications, multiple datasets created from variousregions under different conditions are needed. To demonstrate a powerful new method for integrating heterogeneous UAV datasets, this paper applies a combined segmentation network (CSN) to share UAVid and semantic drone dataset encoding blocks to learn their general features, whereas its decoding blocks are trained separately on each dataset. Experimental results show that our CSN improves the accuracy of specific classes (e.g., cars), which currently comprise a low ratio in both datasets. From this result, it is expected that the range of UAV dataset utilization will increase.

주제어

표/그림 (9)

표 Table 1. Information on the two datasets
그림 Fig. 1. Examples of images and labels of (a) UAVid and (b) semantic drone dataset (SDD).
그림 Fig. 2. Architecture of the combined segmentation network.
그림 Fig. 3. Architecture of the combined segmentation network (CSN) with three cases.
그림 Fig. 4. Learning accuracy and loss of training and validation sets obtained from (a) UAVid and (b) semantic drone dataset (SDD).
그림 Fig. 5. Prediction Results of randomly selected test images from UAVid (a), (g) RGB image, (b),(h) reference images, and others are segmentation results of (c),(i) U-Net, (d),(j) CSN with case 1, (e),(k) CSN with case 2, (f),(l) CSN with case 3.
표 Table 2. Accuracy assessment of test images from UAVid using four models
표 Table 3. Accuracy assessment of test images from SDD using four models
그림 Fig. 6. Prediction results of randomly selected test images from UAVid (a), (g) RGB image, (b), (h) reference images, and others are segmentation results of (c), (i) U-Net, (d), (j) CSN with case 1, (e), (k) CSN with case 2, (f), (l) CSN with case 3.

AI 본문요약
AI-Helper

제안 방법

The model was trained for 50 epochs with the two heterogeneous inputs. Because the objective of this study was to confirm the efficacy of the combined network in a situation in which the amount of data was insufficient, only some of the imagesin the dataset were used for training the model. From UAVid, 200 images were taken and divided into 70 and 30% for training and validation, respectively.
In this study, the lengths of the encoding and decoding blocks are configured in three ways to confirm the effects of the CSN architecture. Case 1 uses six convolutional blocks in encoding phase and ten convolutional blocksin decoding phase.

성능/효과

Thus, the total losses of the basic U-Net were lower than those of the CSNs. Because the CSNs used the combined losses of UAVid and SDD datasets, the total loss was higher. The CSNs were trained so that the sum of the losses was minimized.
Although the accuracy ofthe car classesin SDD wasimproved withCSN, other classes’accuracies were lower than the basic U-Net because SDD has many classes, but the training images were insufficient. However, CSNs can improve prediction accuracy of specific classes when the ratio of the class in UAVid were very small. Ultimately, it was confirmed that segmentation accuracies can be improved for specific classesthat coexist in more than one dataset, aslong as they have similar characteristicsthat can be capitalized through layer-sharing.
The CSNs were trained so that the sum of the losses was minimized. However, it was confirmed that the learning processes of validation dataset withCSNs was more stable than the basic U-Net.
In particular, CSN Case 1 and Case 2 can segment moving cars on the road in the test images. However, the accuracy of CSN Case 1 waslower than that of U-Net. The main reason is that CSN Case 1 predicted background clutter on the road (e.
When comparing the results according to the networks, the accuracies were very similar. However, the basic U-Net performed better than the CSNs. Thus, the total losses of the basic U-Net were lower than those of the CSNs.
Learning using CSN has the drawback of having lesser accuracy and taking longer than learning each data separately. However, there were several classes, such as cars with improved accuracy, which are difficult to obtain sufficient information with a single dataset, and the ratio was small in each class. Through this result, it can be assumed that class information on heterogeneous datasets can be integrated throughCSN.
Alternatively, there was no clear difference in the CSN structure. However,since thisstudy partially added the convolutional block in the encoder and decoder phases based on the basicCSN, the effectmay be insignificant. Nevertheless, in the case of SDD, it can be seen that the human class accuracy is improved when the convolutional block is increased.
Case 1 uses six convolutional blocks in encoding phase and ten convolutional blocksin decoding phase. In Case 2, the number of convolutional blocksin the initialshared encoding phase are increased, but the convolutional blocks in the decoding phase are reduced. In contrast, Case 3 reduces the number of shared convolutional blocksandincreasesthenumberofdecodingconvolutional blocks according to the diagram in Fig.
6 displays the predicted results of two test images from SDD and Table 3 shows the accuracy of randomly selected 50 test images. Like the UAVid results, the overall accuracy, F1 score, and kappa coefficient of all classes were lower with the CSNs. However, the vehicle class was improved with CSNs over the basic U-Net.
However,since thisstudy partially added the convolutional block in the encoder and decoder phases based on the basicCSN, the effectmay be insignificant. Nevertheless, in the case of SDD, it can be seen that the human class accuracy is improved when the convolutional block is increased.
Because the CSNs used the combined losses of UAVid and SDD datasets, the total loss was higher. The CSNs were trained so that the sum of the losses was minimized. However, it was confirmed that the learning processes of validation dataset withCSNs was more stable than the basic U-Net.
However, the accuracy of CSN Case 1 waslower than that of U-Net. The main reason is that CSN Case 1 predicted background clutter on the road (e.g., roadlike materials). This is because the background clutter class contains parking lots and car-free areas and it has very similar spectral characteristics with road class.
6k and l). This is because CSNs can learn more reliable vehicle characteristics by sharing UAVid and SDD in the encoding phase.
However, the basic U-Net performed better than the CSNs. Thus, the total losses of the basic U-Net were lower than those of the CSNs. Because the CSNs used the combined losses of UAVid and SDD datasets, the total loss was higher.
The total epoch was set to 50. UAVid training data accuracy was almost 0.8 and validation data accuracy was between 0.5 and 0.7. Similarly, the accuracy of SDD training data reached 0.
However, CSNs can improve prediction accuracy of specific classes when the ratio of the class in UAVid were very small. Ultimately, it was confirmed that segmentation accuracies can be improved for specific classesthat coexist in more than one dataset, aslong as they have similar characteristicsthat can be capitalized through layer-sharing.

참고문헌 (13)

Althnian, A., AlSaeed, D., Al-Baity, H., Samha, A.,？Dris, A.B., Alzakari, N.H. et al., 2021. Impact？of dataset size on classification performance:？an empirical evaluation in the medical domain.？Applied Sciences, 11(2), 796. https://doi.org/10.3390/app11020796

상세보기
Cui,B.,Chen,X.,andLu,Y.,2020. Semantic segmentation？of remote sensing images using transfer learning？and deep convolutional neural network with？dense connection. IEEE Access, 8, 116744-？116755. https://doi.org/10.1109/ACCESS.2020.3003914

상세보기
Lee, H., Eum, S., and Kwon, H., 2018. Cross-domain？CNN for hyperspectral image classification. In？Proceedings of the IGARSS 2018-2018 IEEE？International Geoscience and Remote Sensing？Symposium, Valencia, Spain, July 22-27, pp.？3627-3630. https://doi.org/10.1109/IGARSS.2018.8519419
Li, J., Huang, X., and Gong, J., 2019. Deep neural？network for remote-sensing image interpretation:？status and perspectives.National ScienceReview,？6(6), 1082-1086. https://doi.org/10.1093/nsr/nwz058

상세보기
Li,Y., Ma,J., and Zhang,Y., 2021.Image retrieval from？remote sensing big data:Asurvey. Information？Fusion, 67, 94-115. https://doi.org/10.1016/j.inffus.2020.10.008

상세보기
Lyu, Y., Vosselman, G., Xia, G., Yilmaz,A., and Yang,？M.Y., 2020. UAVid: A semantic segmentation？dataset for UAV imagery. ISPRS Journal of？Photogrammetry and Remote Sensing, 165,？108-119. https://doi.org/10.1016/j.isprsjprs.2020.05.009

상세보기
Meletis, P. and Dubbelman, G., 2018. Training of？convolutional networks on multiple heterogeneous？datasets for street scene semantic segmentation.？In Proceedings of the 2018 IEEE Intelligent？Vehicles Symposium (IV), Changshu, China,？June 26-30, pp. 1045-1050. https://doi.org/10.1109/IVS.2018.8500398
Graz University of Technology, 2019. Semantic Drone？Dataset. Available online: http://dronedataset.icg.tugraz.at (accessed on Feb. 25, 2023).
Panboonyuen, T., Jitkajornwanich, K., Lawawirojwong,？S., Srestasathiern, P., and Vateekul, P., 2019.？Semantic segmentation on remotely sensed？images using an enhanced global convolutional？network with channel attention and domain？specific transfer learning. Remote Sensing,？11(1), 83. https://doi.org/10.3390/rs11010083

상세보기
Song,A. and Kim,Y., 2020. Semantic segmentation of？remote-sensing imagery using heterogeneous？big data : International society for photogrammetry？and remote sensing potsdam and cityscape？datasets. ISPRS International Journal of Geo Information, 9(10), 601. https://doi.org/10.3390/ijgi9100601

상세보기
Valada,A.,Vertens,J., Dhall,A., and Burgard,W., 2017.？Adapnet: Adaptive semantic segmentation in？adverse environmental conditions. In Proceedings？of the 2017 IEEE International Conference on？Robotics and Automation (ICRA), Singapore,？May 29-June 3, pp. 4644-4651. https://doi.org/10.1109/ICRA.2017.7989540
Yang, K., Hu, X., Wang, K., and Stiefelhagen, R.,？2020. In Defense of Multi-Source Omni Supervised Efficient Conv Net for Robust？Semantic Segmentation in Heterogeneous？Unseen Domains. In Proceedings of the 2020？IEEE Intelligent Vehicles Symposium (IV),？Las Vegas, NV, USA, Oct. 19-Nov. 13, pp.？1386-1393. https://doi.org/10.1109/IV47402.2020.9304768
Ying, X., 2019. An overview of over fitting and its？solutions. Journal of Physics: Conference？Series, 1168, 022022. https://doi.org/10.1088/1742-6596/1168/2/022022？

상세보기

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증