[논문]DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Ullah, Hayat; Irfan, Muhammad; Han, Kyungjin; Lee, Jong Weon

doi:10.3390/s20226457

[해외논문] DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment 원문보기

Sensors, v.20 no.22, 2020년, pp.6457 -

Ullah, Hayat , Irfan, Muhammad , Han, Kyungjin , Lee, Jong Weon

Abstract ▼ AI-Helper

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

주제어

참고문헌 (56)

1. Perfecto C. Elbamby M.S. Del Ser J. Bennis M. Taming the latency in multi-user VR 360°: A QoE-aware deep learning-aided multicast framework IEEE Trans. Commun. 2020 68 2491 2508 10.1109/TCOMM.2020.2965527

상세보기
2. Bhagat K.K. Liou W.-K. Chang C.-Y. A cost-effective interactive 3D virtual reality system applied to military live firing training Virtual Real. 2016 20 127 140 10.1007/s10055-016-0284-x

상세보기
3. Lacko J. Health Safety Training for Industry in Virtual Reality Proceedings of the 2020 Cybernetics & Informatics (K & I) Velke Karlovice, Czech Republic 29 January？1 February 2020 1 5
4. Zhu D. Zhou Q. Han T. Chen Y. 360 Degree Panorama Synthesis from Sequential Views Based on Improved FC-Densenets IEEE Access 2019 7 180503 180511 10.1109/ACCESS.2019.2958111

상세보기
5. Kang J. Kim J. Lee I. Kim K. Minimum Error Seam-Based Efficient Panorama Video Stitching Method Robust to Parallax IEEE Access 2019 7 167127 167140 10.1109/ACCESS.2019.2953705

상세보기
6. Deng F. Yang J. Panoramic Image Generation Using Centerline-Constrained Mesh Parameterization for Arbitrarily Shaped Tunnel Lining IEEE Access 2020 8 7969 7980 10.1109/ACCESS.2020.2964121

상세보기
7. Ullah H. Zia O. Kim J.H. Han K. Lee J.W.J.S. Automatic 360° Mono-Stereo Panorama Generation Using a Cost-Effective Multi-Camera System Sensors 2020 20 3097 10.3390/s20113097 32486231

상세보기
8. Wang W. Wu G. Cai W. Zeng L. Chen J.J.I.A. Robust Prior-Based Single Image Super Resolution Under Multiple Gaussian Degradations IEEE Access 2020 8 74195 74204 10.1109/ACCESS.2020.2987911

상세보기
9. Kim Y. Soh J.W. Cho N.I. AGARNet: Adaptively Gated JPEG Compression Artifacts Removal Network for a Wide Range Quality Factor IEEE Access 2020 8 20160 20170 10.1109/ACCESS.2020.2968944

상세보기
10. Gungor M.A. Gencol K.J.O. Developing a Compression Procedure Based Oo the Wavelet Denoising and JPEG2000 Compression Optik 2020 218 164933 10.1016/j.ijleo.2020.164933

상세보기
11. Bardenet R. Flamant J. Chainais P.J.A. Analysis C.H. On the zeros of the spectrogram of white noise Appl. Comput. Harmon. Anal. 2020 48 682 705 10.1016/j.acha.2018.09.002

상세보기
12. Patra J.P. Singh P. Improved Signal Detection Techniques for QOSTBC System in Fast Fading Channel J. Telecommun. Inf. Technol. 2020 1 6 12 10.26636/jtit.2020.138019

상세보기
13. Yan B. Bare B. Ma C. Li K. Tan W. Deep objective quality assessment driven single image super-resolution IEEE Trans. Multimed. 2019 21 2957 2971 10.1109/TMM.2019.2914883

상세보기
14. Kong Y. Cui L. Hou R. Full-reference IPTV image quality assessment by deeply learning structural cues Signal Process. Image Commun. 2020 83 115779 10.1016/j.image.2020.115779

상세보기
15. Deng C. Wang S. Bovik A.C. Huang G.-B. Zhao B. Blind noisy image quality assessment using sub-band kurtosis IEEE Trans. Cybern. 2019 50 1146 1156 10.1109/TCYB.2018.2889376 30629529

상세보기
16. Tan H.L. Li Z. Tan Y.H. Rahardja S. Yeo C. A perceptually relevant MSE-based image quality metric IEEE Trans. Image Process. 2013 22 4447 4459 24057005

상세보기
17. Yan B. Bare B. Tan W. Naturalness-aware deep no-reference image quality assessment IEEE Trans. Multimed. 2019 21 2603 2615 10.1109/TMM.2019.2904879

상세보기
18. Po L.-M. Liu M. Yuen W.Y. Li Y. Xu X. Zhou C. Wong P.H. Lau K.W. Luk H.-T. A novel patch variance biased convolutional neural network for no-reference image quality assessment IEEE Trans. Circuits Syst. Video Technol. 2019 29 1223 1229 10.1109/TCSVT.2019.2891159

상세보기
19. Liu L. Wang T. Huang H. Pre-attention and spatial dependency driven no-reference image quality assessment IEEE Trans. Multimed. 2019 21 2305 2318 10.1109/TMM.2019.2900941

상세보기
20. Kim W. Nguyen A.-D. Lee S. Bovik A.C. Dynamic Receptive Field Generation for Full-Reference Image Quality Assessment IEEE Trans. Image Process. 2020 29 4219 4231 10.1109/TIP.2020.2968283

상세보기
21. Li J. Yu K. Zhao Y. Zhang Y. Xu L. Cross-reference stitching quality assessment for 360 omnidirectional images Proceedings of the 27th ACM International Conference on Multimedia Nice, France 21？25 October 2019 2360 2368
22. Yan W. Yue G. Fang Y. Chen H. Tang C. Jiang G. Perceptual objective quality assessment of stereoscopic stitched images Signal Process. 2020 172 107541 10.1016/j.sigpro.2020.107541

상세보기
23. Upenik E. eabek M. Ebrahimi T. Testbed for subjective evaluation of omnidirectional visual content Proceedings of the 2016 Picture Coding Symposium (PCS) Nuremberg, Germany 4？7 December 2016 1 5
24. Sun W. Gu K. Ma S. Zhu W. Liu N. Zhai G. A large-scale compressed 360-degree spherical image database: From subjective quality evaluation to objective model comparison Proceedings of the 2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP) Vancouver, BC, Canada 29？31 August 2018 1 6
25. Duan H. Zhai G. Min X. Zhu Y. Fang Y. Yang X. Perceptual quality assessment of omnidirectional images Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS) Florence, Italy 27？30 May 2018 1 5
26. Zhang Y. Wang Y. Liu F. Liu Z. Li Y. Yang D. Chen Z. Subjective panoramic video quality assessment database for coding applications IEEE Trans. Broadcast. 2018 64 461 473 10.1109/TBC.2018.2811627

상세보기
27. Lopes F. Ascenso J. Rodrigues A. Queluz M.P. Subjective and objective quality assessment of omnidirectional video Proceedings of the Applications of Digital Image Processing XLI San Diego, CA, USA 20？23 August 2018
28. Zhu S. Zhang Y. Tao L. Liu T. Liu Y. A Novel Method for Quality Assessment of Image Stitching Based on the Gabor Filtering Proceedings of the 2018 IEEE International Conference on Information and Automation (ICIA) Wuyi Mountain, China 11？13 August 2018 1605 1610
29. Huang M. Shen Q. Ma Z. Bovik A.C. Gupta P. Zhou R. Cao X. Modeling the perceptual quality of immersive images rendered on head mounted displays: Resolution and compression IEEE Trans. Image Process. 2018 27 6039 6050 10.1109/TIP.2018.2865089 30106732

상세보기
30. Jabar F. Ascenso J. Queluz M.P. Objective Assessment of Perceived Geometric Distortions in Viewport Rendering of 360° Images IEEE J. Sel. Top. Signal Process. 2019 14 49 63 10.1109/JSTSP.2019.2962970

상세보기
31. Yang L. Tan Z. Huang Z. Cheung G. A content-aware metric for stitched panoramic image quality assessment Proceedings of the IEEE International Conference on Computer Vision Workshops Venice, Italy 22？29 October 2017 2487 2494
32. Zhou X. Zhang H. Wang Y. A multi-image stitching method and quality evaluation Proceedings of the 2017 4th International Conference on Information Science and Control Engineering (ICISCE) Changsha, China 21？23 July 2017 46 50
33. Ling S. Cheung G. Le Callet P. No-reference quality assessment for stitched panoramic images using convolutional sparse coding and compound feature selection Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME) San Diego, CA, USA 23？27 July 2018 1 6
34. Madhusudana P.C. Soundararajan R.J. Subjective and objective quality assessment of stitched images for virtual reality IEEE Trans. Image Process. 2019 28 5620 5635 10.1109/TIP.2019.2921858 31217105

상세보기
35. Li J. Zhao Y. Ye W. Yu K. Ge S. Attentive Deep Stitching and Quality Assessment for 360° Omnidirectional Images IEEE J. Sel. Top. Signal Process. 2019 14 209 221 10.1109/JSTSP.2019.2953950

상세보기
36. Sun W. Min X. Zhai G. Gu K. Duan H. Ma S. MC360IQA: A Multi-channel CNN for Blind 360-Degree Image Quality Assessment IEEE J. Sel. Top. Signal Process. 2019 14 64 77 10.1109/JSTSP.2019.2955024

상세보기
37. Xu J. Zhou W. Chen Z.J. Blind Omnidirectional Image Quality Assessment with Viewport Oriented Graph Convolutional Networks IEEE Trans. Circuits Syst. Video Technol. 2020 10.1109/TCSVT.2020.3015186

상세보기
38. He K. Gkioxari G. Dollar P. Girshick R. Mask r-cnn Proceedings of the IEEE International Conference on Computer Vision Venice, Italy 22？29 October 2017 2961 2969
39. Xia K. Huang J. Wang H. LSTM-CNN Architecture for Human Activity Recognition IEEE Access 2020 8 56855 56866 10.1109/ACCESS.2020.2982225

상세보기
40. Ullah A. Muhammad K. Del Ser J. Baik S.W. de Albuquerque V.H. Activity recognition using temporal optical flow convolutional features and multilayer LSTM IEEE Trans. Ind. Electron. 2018 66 9692 9702 10.1109/TIE.2018.2881943

상세보기
41. Muhammad K. Hussain T. Tanveer M. Sannino G. de Albuquerque V.H. Cost-effective video summarization using deep CNN with hierarchical weighted fusion for IoT surveillance networks IEEE Internet Things J. 2019 7 4455 4463 10.1109/JIOT.2019.2950469

상세보기
42. Hussain T. Muhammad K. Ullah A. Cao Z. Baik S.W. de Albuquerque V.H. Cloud-assisted multiview video summarization using CNN and bidirectional LSTM IEEE Trans. Ind. Inform. 2019 16 77 86 10.1109/TII.2019.2929228

상세보기
43. Wang G. Guo J. Chen Y. Li Y. Xu Q. A PSO and BFO-based learning strategy applied to faster R-CNN for object detection in autonomous driving IEEE Access 2019 7 18840 18859 10.1109/ACCESS.2019.2897283

상세보기
44. Sajjad M. Irfan M. Muhammad K. Del Ser J. Sanchez-Medina J. Andreev S. Ding W. Lee J.W. An Efficient and Scalable Simulation Model for Autonomous Vehicles with Economical Hardware IEEE Trans. Intell. Transp. Syst. 2020 10.1109/TITS.2020.2980855

상세보기
45. Muhammad K. Ahmad J. Baik S.W. Early fire detection using convolutional neural networks during surveillance for effective disaster management Neurocomputing 2018 288 30 42 10.1016/j.neucom.2017.04.083

상세보기
46. Khan S. Muhammad K. Mumtaz S. Baik S.W. de Albuquerque V.H. Energy-efficient deep CNN for smoke detection in foggy IoT environment IEEE Internet Things J. 2019 6 9237 9245 10.1109/JIOT.2019.2896120

상세보기
47. Girshick R. Fast r-cnn Proceedings of the IEEE International Conference on Computer Vision Santiago, Chile 7？13 December 2015 1440 1448
48. He K. Zhang X. Ren S. Sun J. Deep residual learning for image recognition Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 27？30 June 2016 770 778
49. Abdullah W. Mask R-CNN for Object Detection and Instance Segmentation on Keras and TensorFlow Available online: https://github.com/matterport/Mask_RCNN (accessed on 6 August 2020)
50. Common Objects in Context (COCO) Available online: http://cocodataset.org/#home (accessed on 11 August 2020)
51. Google Street View Dataset Available online: https://www.crcv.ucf.edu/data/GMCP_Geolocalization/ (accessed on 15 August 2020)
52. SUN360 Dataset Available online: http://people.csail.mit.edu/jxiao/SUN360/main.html (accessed on 15 August 2020)
53. Mittal A. Moorthy A.K. Bovik A.C. No-reference image quality assessment in the spatial domain IEEE Trans. Image Process. 2012 21 4695 4708 10.1109/TIP.2012.2214050 22910118

상세보기
54. Moorthy A.K. Bovik A.C. Blind image quality assessment: From natural scene statistics to perceptual quality IEEE Trans. Image Process. 2011 20 3350 3364 10.1109/TIP.2011.2147325 21521667

상세보기
55. Mittal A. Soundararajan R. Bovik A.C. Making a “completely blind” image quality analyzer IEEE Signal Process. Lett. 2012 20 209 212 10.1109/LSP.2012.2227726

상세보기
56. Moorthy A.K. Bovik A.C. A two-step framework for constructing blind image quality indices IEEE Signal Process. Lett. 2010 17 513 516 10.1109/LSP.2010.2043888

상세보기

LOADING...

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

[해외논문] DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment 원문보기

Abstract ▼ AI-Helper

주제어

참고문헌 (56)

이 논문을 인용한 문헌

연구과제 타임라인

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

연관된 기능

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

연합인증

[해외논문] DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment 원문보기

Abstract ▼ AI-Helper

주제어

참고문헌 (56)

이 논문을 인용한 문헌

연구과제 타임라인

전체(0) 논문(0) 특허(0) 보고서(0)

전체(0) 논문(0) 특허(0) 보고서(0)

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

연관된 기능

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트