Real-time capturing and generating viewpoint images and videos with a monoscopic low power mobile device
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-013/02
H04N-013/00
H04N-005/232
출원번호
US-0491187
(2014-09-19)
등록번호
US-9509980
(2016-11-29)
발명자
/ 주소
Wang, Haohong
Li, Hsiang-Tsun
Manjunath, Sharath
출원인 / 주소
QUALCOMM INCORPORATED
대리인 / 주소
Boyd, Brent
인용정보
피인용 횟수 :
1인용 특허 :
12
초록▼
A monoscopic low-power mobile device is capable of creating real-time images and videos for multiple viewpoints from a single captured view. The device uses statistics from an autofocusing process to create a block depth map of a single capture view. Artifacts in the block depth map are reduced and
A monoscopic low-power mobile device is capable of creating real-time images and videos for multiple viewpoints from a single captured view. The device uses statistics from an autofocusing process to create a block depth map of a single capture view. Artifacts in the block depth map are reduced and an image depth map is created. Alternate views are created from the image depth map using a Z-buffer based surface recover process and a disparity map which is a function of a geometric vision model.
대표청구항▼
1. A monoscopic low-power mobile device comprising: a single-sensor camera configured to capture an image from a first viewpoint;an autofocusing module configured to determine a best focus position by moving a lens of the single-sensor camera through an entire focusing range via a focusing process a
1. A monoscopic low-power mobile device comprising: a single-sensor camera configured to capture an image from a first viewpoint;an autofocusing module configured to determine a best focus position by moving a lens of the single-sensor camera through an entire focusing range via a focusing process and to select the focus position with a maximum focus value when capturing the image;a depth map generator assembly in data communication with the autofocusing module and the single-sensor camera, the depth map generator assembly configured to: in a first-stage, develop a block-level depth map automatically using statistics from the autofocusing module,in a second-stage to develop an image depth map, the block-level depth map including a depth value for each of a plurality of portions of the captured image, the image depth map including a pixel depth value for a pixel in a portion of the plurality of portions, andduring the second stage, obtain a depth value for corner pixels of each block included in the block-level depth map, the depth value for a corner pixel based at least in part on an average of depth values for middle points of neighboring blocks of a respective block, the neighboring blocks included in the block-level depth map;a bilinear filter in data communication with the depth map generator assembly, the bilinear filter configured to generate the pixel depth value for the pixel based at least in part on depth values for each corner pixel, the depth values for each corner pixel weighted based on a ratio between a distance of the pixel to a respective corner pixel and a total distance of the pixel to each corner pixel, the bilinear filter configured to generate the pixel depth value during the second-stage; andan image generator configured to create a second image having a second viewpoint from the captured image, the second viewpoint being different from the first viewpoint. 2. The device of claim 1, wherein the image generator comprises: a disparity map module configured to calculate a disparity map based on a distance in pixels between image points in views of a vision geometry model for the captured image, the captured image representing one view of the views of the vision geometry model;a surface recover module configured to construct a surface for the second image from the second viewpoint; anda view generator module configured to generate the second image based on a projection of the constructed surface for the second image onto a projection plane. 3. The device of claim 1, wherein generating the pixel depth value for the pixel based at least in part on depth values for each corner pixel comprises generating the pixel depth value dP for pixel P (xP, yP, dP) in accordance with the equation dp=(xp-xA)4+(yp-yA)4(xp-xA)4+(yp-yA)4+(xp-xB)4+(yp-yB)4+(xp-xC)4+(yp-yC)4+(xp-xD)4+(yp-yD)4dA+(xp-xB)4+(yp-yB)4(xp-xA)4+(yp-yA)4+(xp-xB)4+(yp-yB)4+(xp-xC)4+(yp-yC)4+(xp-xD)4+(yp-yD)4dB+(xp-xC)4+(yp-yC)4(xp-xA)4+(yp-yA)4+(xp-xB)4+(yp-yB)4+(xp-xC)4+(yp-yC)4+(xp-xD)4+(yp-yD)4dC+(xp-xD)4+(yp-yD)4(xp-xA)4+(yp-yA)4+(xp-xB)4+(yp-yB)4+(xp-xC)4+(yp-yC)4+(xp-xD)4+(yp-yD)4dDwhere position values and the depth values for the corner pixels (A, B, C, and D) of the block are denoted as (xA, yA, dA), (xB, yB, dB), (xC, yC, dC), (xD, yD, dD). 4. The device of claim 1, wherein the focusing process of the autofocusing module in a still image mode performs an exhaustive search focusing process to capture a still-image, and in a video mode, to achieve real-time capturing of a video clip, is initiated with the exhaustive search focusing process and follows with a climbing-hill focusing process. 5. The device of claim 4, further comprising a video coding module for coding the video clip captured and providing statistics information for calculating the block-level depth map, the video coding module configured to determine motion estimation, and the depth map generator assembly configured in the second stage to detect and estimate depth information for real-time capturing and generation of video from the second viewpoint using the statistics information from the motion estimation, the focusing process, and history data plus heuristic rules to obtain a final block depth map from which the image depth map is derived. 6. The device of claim 4, wherein the autofocusing module is further configured to process the captured video clip and estimate depth information of remote objects in a scene, the autofocusing module configured to generate an autofocusing block depth map and an autofocusing focus value map for frames of the captured video clip, and wherein the video coding module is configured to generate a video coding block depth map and a video coding focus value map for frames of the captured video clip based at least in part on the determined motion estimation, andwherein the device further comprises: an image depth map module operable to detect and estimate depth information for real-time capturing and generation of video from the second viewpoint based on a final block depth map from which an image depth map is derived, the block depth map including a depth value for each of a plurality of portions of the captured video clip, the image depth map including a pixel depth value for a pixel in a portion of the plurality of portions, the image depth map module configured to generate the final block depth map based on values included in the autofocusing block depth map, the autofocusing focus value map, the video coding block depth map, and the video coding focus value map. 7. The device of claim 6, wherein the image depth map module is configured to predict an internal block depth map (Pn(i,j)) and a focus value map (Tn(i,j)) of a current frame n from those of a previous frame by the following equations Pn(i,j)={Dn-1(a,b)ifVn(i,j)-Fn-1(a,b)
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (12)
Oh,Teik; Flack,Julien; Harman,Philip Victor, 3D image synthesis from depth encoded source view.
Haruhiko Murata JP; Yukio Mori JP; Shuugo Yamashita JP; Akihiro Maenaka JP; Seiji Okada JP; Kanji Ihara JP, Device and method for converting two-dimensional video into three-dimensional video.
Nakagawa Yasuo (Chigasaki PA JPX) Nayer Shree K. (Pittsburgh PA), Method of detecting solid shape of object with autofocusing and image detection at each focus level.
Eleftheriadis Alexandros ; Anastassiou Dimitris ; Chang Shif-Fu ; Nayar Shree, Methods and apparatus for performing digital image and video segmentation and compression using 3-D depth information.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.