[논문]Single Image Depth Estimation With Integration of Parametric Learning and Non-Parametric Sampling

Jung, Hyungjoo; Sohn, Kwanghoon

doi:10.9717/kmms.2016.19.9.1659

[국내논문] Single Image Depth Estimation With Integration of Parametric Learning and Non-Parametric Sampling 원문보기

멀티미디어학회논문지 = Journal of Korea Multimedia Society, v.19 no.9, 2016년, pp.1659 - 1668

Jung, Hyungjoo (School of Electrical & Electronic Engineering, Yonsei University) , Sohn, Kwanghoon (School of Electrical & Electronic Engineering, Yonsei University)

Abstract ▼ AI-Helper

Understanding 3D structure of scenes is of a great interest in various vision-related tasks. In this paper, we present a unified approach for estimating depth from a single monocular image. The key idea of our approach is to take advantages both of parametric learning and non-parametric sampling method. Using a parametric convolutional network, our approach learns the relation of various monocular cues, which make a coarse global prediction. We also leverage the local prediction to refine the global prediction. It is practically estimated in a non-parametric framework. The integration of local and global predictions is accomplished by concatenating the feature maps of the global prediction with those from local ones. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods both qualitatively and quantitatively.

주제어

AI 본문요약
AI-Helper

제안 방법

The proposed method consists of three components as follows: parametric model for coarse depth estimation, non-parametric sampling framework for warped depth maps, and CNN layers for depth refinement. The overall framework of the proposed method is shown in Fig.

이론/모형

This paper has presented depth estimation method from a single monocular scene using popular FCN and non-parametric method. FCN can extract globally plausible depth, whereas outputs of non-parametric method preserve local structures, due to warping process based on Patch Match.

성능/효과

, batch normalization and residual net) in our deep networks learning, they converge fast and produce better results. Experimental results show that the proposed method can outperform other existing methods in estimating both of global and local structures of depth. In experiments, we tried to use large RGB-D dataset.
This paper has presented depth estimation method from a single monocular scene using popular FCN and non-parametric method. FCN can extract globally plausible depth, whereas outputs of non-parametric method preserve local structures, due to warping process based on Patch Match. Each virtues is aggregated by CNN, which only learn residual.
However, large database makes it hard to retrieve training images in non-parametric method. Fast image search algorithms can improve the proposed method. Also, in future works, we will extend the proposed method by jointly training parametric and non-parametric methods.
Non-parametric methods can also make better performance in large database. However, large database makes it hard to retrieve training images in non-parametric method. Fast image search algorithms can improve the proposed method.
It is well-known that large number of training images make a good performance in deep neural networks, and it helps to avoid overfitting problem. Non-parametric methods can also make better performance in large database. However, large database makes it hard to retrieve training images in non-parametric method.
Each virtues is aggregated by CNN, which only learn residual. Since we adapt the newest techniques (i.e., batch normalization and residual net) in our deep networks learning, they converge fast and produce better results. Experimental results show that the proposed method can outperform other existing methods in estimating both of global and local structures of depth.

후속연구

Fast image search algorithms can improve the proposed method. Also, in future works, we will extend the proposed method by jointly training parametric and non-parametric methods.

참고문헌 (32)

X. Ren, L. Bo, and D. Fox, "RGB-(D) Scene Labeling: Features and Algorithms," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition , pp. 2759-2766, 2012.
D. Lin, S. Fidler, and R. Urtasun, "Holistic Scene Understanding for 3D Object Detection with RGBD Cameras," Proceeding of IEEE International Conference on Computer Vision , pp. 1417- 1424, 2013.
J. Jeon, S. Cho, X. Tong, and S. Lee, "Intrinsic Image Decomposition Using Structure-Texture Separation and Surface Normals," Proceeding of European Conference on Computer Vision , pp. 218-233, 2014.
O. Wang, M. Lang, and M. Gross, "Stereo Brush: Interactive 2D to 3D Conversion using Discontinuous Warps," Proceedings of the 8th Eurographics Symposium on Sketch-based Interfaces and Modeling , pp. 47-54, 2011.
H. Yuan, S. Wu, P. Cheng, P. An, and S. Bao, "Nonlocal Random Walks Algorithms for Semi-Automatic 2D-to-3D Image Convertsion," IEEE Signal Processing Letters , Vol. 22, No. 3, pp. 371-374, 2015.

상세보기
J. Atick, P. Griffin, and N. Redlich, "Statistical Approach to Shape fomr Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images," Neural Computation , Vol. 8, No. 6, pp. 132-1340, 1996.

상세보기
R. Szeliski and P. Torr, "Geometrically Constrained Structure from Motion: Points on Planes," 3D Structure from Multiple Images of Large-Scale Environments , Springer Berlin Heideberg , pp. 171-186, 1998.
A. Oliva and A. Torralba, "Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope," International Journal of Computer Vision , Vol. 42, No. 3, pp. 145-175, 2001.
N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proceeding of IEEE Computer Vision and Pattern Recognition , Vol. 1, pp. 886-893, 2005.
K. Karsch, C. Liu, and S. Kang, "Depth Extraction from Video Using Non-parametric Sampling," Proceeding of European Conference on Computer Vision , pp. 775-788, 2012.
S. Choi, D. Min, B. Ham, Y. Kim, C. Oh, and K. Sohn, "Depth Analogy: Data-Driven Approach for Single Image Depth Estimation using Gradient Samples," IEEE Transaction on Image Processing , Vol. 24, No. 12, pp. 5953-5966, 2015.

상세보기
A. Saxena, M. Sun, and A. Ng, "MAKE3D: Learning 3D Scene Structure from a Single Still Image," IEEE Transaction on Pattern Analysis machine Intelligence , Vol. 31, No. 5, pp. 824-840, 2009.

상세보기
D. Eigen, C. Puhrsch, and R. Fergus, "Depth Map Prediction from a Single Image Using a Multi-Scale Deep Network," Advances in Neural Information Processing Systems , pp. 2366-2374, 2014.
F. Liu, C. Shen, and G. Lin, "Deep Convolutional Neural Fields for Depth Estimation from a Single Image," Proceeding of IEEE Conference on Computer Vision and Recognition , pp. 5162-5170, 2015.
J. Long, E. Shelhamer, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition , pp. 3431-3440, 2015.
C. Liu, J. Yuen, and A. Torralba, "SIFT Flow: Depth Correspondence across Scenes and Its Applications," IEEE Transaction Patterns Analysis Machine Intelligence , Vol. 33, No. 33, pp. 978-994, 2011.

상세보기
C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman, "PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing," ACM Transactions on Graphics , Vol. 28, No. 3, pp. 24, 2009.

상세보기
J. Konrad, M. Wang, P. Ishwar, C. Wu, and D. Mukherjee, "Learning-based, Automatic 2D-to-3D Image and Video Conversion," IEEE Transaction Image Processing , Vol. 22, No. 9, pp. 3485-3496, 2013.

상세보기
L. C. Chen, G. Papandreou, I. Kokkonos, K. Murphy, and A.L. Yuille, "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs," arXiv: 1412.7062, 2014.
L. C. Chen, J. Barron, and A. L. Yulle, "Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform," arXiv: 1511.03328, 2015.
F. Liu, C. Shen, G. Lin, and I. D. Reid, "Learning Depth from Single Monocular Images using Deep Convolutional Neural Fields," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition , pp. 5162-5170, 2015.
W. Chen, Z. Fu, D. Yang, and J. Dong, "Single-Image Depth Perception in the Wild" , arXiv: 1604.03901, 2016.
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Reconstruction," arXiv: 1409.1556, 2014.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A.C. Berg, and L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision , Vol. 115, No. 3, pp. 211-252, 2015.

상세보기
A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Advances in Neural Information Processing Systems , pp. 1097-1105, 2012.
J. Kim, J. Lee, and K. Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks," arXiv: 1511.04587, 2015.
N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor Segmentation and Support Inference from RGBD Images," Proceeding of European Conference on Computer Vision , pp. 746-760, 2012.
S. Loffe and c. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," Proceeding of International Conference on Machine Learning , pp. 448-456, 2015.
J. Kopf, M.F. Cohen, D. Lischinski, and M. Uyttendaele, "Joint Bilateral Upsampling," ACM Transactions on Graphics , Vol. 26, No. 3, pp. 96, 2007.

상세보기
K. He, J. Sun, and X. Tang, "Guided Image Filtering," Proceeding of European Conference on Computer Vision , pp. 1-14, 2010.
D. Lee, and S. Kwon, "A Recognition Method for Moving Objects Using Depth and Color Information," Journal of Korea Multimedia Society , Vol. 19, No. 4, pp. 681- 688, 2016.

원문보기 상세보기
S. Kim, and H. Kang, "Semantic Segmentation of Indoor Scenes Using Depth Superpixel," Journal of Korea Multimedia Society , Vol. 19, No. 3, pp. 531-538, 2016.

원문보기 상세보기

저자의 다른 논문 :

LOADING...

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증