[논문]Multiple Color and ToF Camera System for 3D Contents Generation

Ho, Yo-Sung

doi:10.5573/ieiespc.2017.6.3.175

Multiple Color and ToF Camera System for 3D Contents Generation 원문보기

IEIE Transactions on Smart Processing and Computing, v.6 no.3, 2017년, pp.175 - 182

Ho, Yo-Sung (School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology)

Abstract ▼ AI-Helper

In this paper, we present a multi-depth generation method using a time-of-flight (ToF) fusion camera system. Multi-view color cameras in the parallel type and ToF depth sensors are used for 3D scene capturing. Although each ToF depth sensor can measure the depth information of the scene in real-time, it has several problems to overcome. Therefore, after we capture low-resolution depth images by ToF depth sensors, we perform a post-processing to solve the problems. Then, the depth information of the depth sensor is warped to color image positions and used as initial disparity values. In addition, the warped depth data is used to generate a depth-discontinuity map for efficient stereo matching. By applying the stereo matching using belief propagation with the depth-discontinuity map and the initial disparity information, we have obtained more accurate and stable multi-view disparity maps in reduced time.

주제어

AI 본문요약
AI-Helper

* AI 자동 식별 결과로 적합하지 않은 문장이 있을 수 있으니, 이용에 유의하시기 바랍니다.

가설 설정

In this paper, we proposed a multi-depth generation method using a ToF fusion system. After capturing multi-view color and ToF depth images, we performed postprocessing to reduce the inherent error and distortion in the original images.

제안 방법

For all test image sets, the results of the proposed method generated more accurate intermediate view images, taking less processing time. The processing time is the average of the disparity generation times of View 2, View 3, and View 4. The accuracy increment and time reduction were obtained by skipping the data cost calculations and decreasing the search range for matching based on the three cases and the depth-discontinuity information.
In this paper, we proposed a multi-depth generation method using a ToF fusion system. After capturing multi-view color and ToF depth images, we performed postprocessing to reduce the inherent error and distortion in the original images. Then, the ToF depth images were warped to color image positions and used for generation of a depth-discontinuity map and the initial disparity values for stereo matching.
We regard the depth information obtained by the depth sensors as the initial disparity of a multi-view image. In order to match the depth images that have different resolutions compared to the color images, and that are captured at different positions from the color cameras, we back-project the ToF depth images to the world coordinates using 3D warping. Then, the depth information in the 3D space is re-projected onto each image plane of the video camera.
Using the depth-discontinuity map and the initial disparity, we increased the accuracy of the disparity map and reduced the processing time. Since the proposed method processes the result based on the initial disparity values, and it considers the ToF term in the energy function, we obtain more accurate and stable disparity values in weak texture regions and occluded regions than the results from previous methods. We expect that the proposed ToF fusion system and multi-depth generation method could be useful for 3D content generation and practical 3D applications.
We also propose a post-processing for captured images and multi-depth generation process. The contribution of this work is that we minimize the inherent error and distortion in the captured images, and generate multi-view depth information based on stereo matching using the post-processed color and ToF depth images. Since the ToF depth data are utilized as assistance for stereo matching, we can obtain more accurate depth information in reduced time.
After capturing multi-view color and ToF depth images, we performed postprocessing to reduce the inherent error and distortion in the original images. Then, the ToF depth images were warped to color image positions and used for generation of a depth-discontinuity map and the initial disparity values for stereo matching. Using the depth-discontinuity map and the initial disparity, we increased the accuracy of the disparity map and reduced the processing time.

대상 데이터

In this paper, we propose a ToF fusion system for a multi-depth-generation. It is composed of multiple color cameras and up to three ToF depth sensors. We also propose a post-processing for captured images and multi-depth generation process.
We captured five-view color images and three-view ToF depth images. The ToF depth sensors were located below the second, third, and fourth color cameras. The horizontal and vertical distances between two cameras were 6.

성능/효과

The proposed method outperform the other methods, as shown in Table 1. In order to compare their results, we synthesized the intermediate views at the position for View 3 using the color images and disparity maps of View 2 and View 4, and we calculated the peak signal-to-noise ratio (PSNR) between the original and synthesized images of View 3. For all test image sets, the results of the proposed method generated more accurate intermediate view images, taking less processing time. The processing time is the average of the disparity generation times of View 2, View 3, and View 4.

참고문헌 (14)

A. Smolic and P. Kauff, "Interactive 3D Video Representation and Coding Technologies," Proceedings of the IEEE, Spatial Issue on Advances in Video Coding and Delivery, vol. 93, no. 1, pp. 99-110, Jan. 2005.
J. Sun, N.N. Zheng, and H.Y. Shum, "Stereo Matching Using Belief Propagation," IEEE Transactions of Pattern Analysis and Machine Intelligence (PAMI), vol. 25, no. 5, pp. 787-800, July 2003.

상세보기
S. A. Gudmundsson, H. Aanaes, and R. Larsen, "Fusion of Stereo Vision and Time-of-Filght Imaging for Improved 3D Estimation," International Journal of Intelligent Systems Technologies and Applications, vol. 5, no. 3, pp. 425-433, Nov. 2008.

상세보기
B. Bartczak and R. Koch, "Dense Depth Maps from Low Resolution Time-of-Flight Depth and High Resolution Color Views," Proc. of 5th International Symposium on Visual Computing, pp. 1-12, Nov. 2009.
J. Zhu, L. Wang, R. Yang, and J. Davis, "Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps," Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 231-236, June 2008.
K. D. Kuhnert and M. Stommel, "Fusion of Stereo-Camera and PMD-Camera Data for Real-time Suited Precise 3D Environment Reconstruction," Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4780-4785, Mar. 2006.
SR4000 User Manual Version 0.1.2.2, Mesa Imaging AG, pp. 18-19.
A. Wang, T. Qiu, and L. Shao, "A Simple Method of Radial Distortion Correction with Centre of Distortion Estimation," Journal of Mathematical Imaging and Vision, vol. 35, no. 3, pp. 165-172, Nov. 2009.

상세보기
Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 22, no. 11, pp. 1330-1334, Oct. 2000.

상세보기
Y.S. Kang and Y.S. Ho, "Geometrical Compensation for Multi-view Video in Multiple Camera Array," Proc. of International Symposium on Electronics and Marine (ELMAR), pp. 83-86, Sept. 2008.
N. Joshi, B. Wilburn, V. Vaish, M. Levoy, and M. Horowitz, "Automatic Color Calibration for Large Camera Arrays," in UCSD CSE Tech. Rep. CS2005-0821, May 2005.
G. Gilboa, N. Sochen, and Y.Y. Zeevi, "Regularized Shock Filters and Complex Diffusion", ECCV 2002, LNCS 2350, pp. 399-313, Springer-Verlag, May 2002.
Y.S. Ho and Y.S. Kang, "Multi-view Depth Generation using Multi-Depth Camera System," Proc. of International Conference on 3D Systems and Application (3DSA), pp. 1-4, May 2010.
P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Belief Propagation for Early Vision," International Journal of Computer Vision, vol. 70, no. 1, pp. 41-54, Oct. 2006.

상세보기

저자의 다른 논문 :

LOADING...

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증