[특허]Systems and methods for generating depth maps using images captured by camera arrays

Systems and methods for generating depth maps using images captured by camera arrays 원문보기

IPC분류정보
국가/구분	United States(US) Patent 등록
국제특허분류(IPC7판)	H04N-005/225 G06T-007/00 G06T-003/40 H04N-005/228 H04N-005/33 H04N-009/083
출원번호	US-0447508 (2014-07-30)
등록번호	US-8957976 (2015-02-17)
발명자 / 주소	Venkataraman, Kartik Jabbi, Amandeep S. Mullis, Robert H.
출원인 / 주소	Pelican Imaging Corporation
대리인 / 주소	KPPB LLP
인용정보	피인용 횟수 : 0 인용 특허 : 80

초록 ▼

A camera array, an imaging device and/or a method for capturing image that employ a plurality of imagers fabricated on a substrate is provided. Each imager includes a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. Each imager may be associated with an optical element fabricated using a wafer level optics (WLO) technology.

대표청구항 ▼

1. A method of generating a depth map from a plurality of images captured from different viewpoints, comprising: normalizing a set of images using calibration data stored in a storage device using an address conversion module, where: the set of images comprises a plurality of images that are: captured from different viewpoints;include different occlusions sets;wherein the occlusion set of a first image is the portion of a scene visible in a second image that is occluded in the first image; andthe set of images includes: images captured using the same imaging characteristics; andimages captured using different imaging characteristics;detecting and metering parallax using a parallax confirmation and measurement module, where detecting and metering parallax comprises: determining the parallax that yields the highest correlation between pixels from images captured using the same imaging characteristics accounting for the positions of the imagers that captured the images by: performing redundant pair-wise measurements between pixels from images captured using the same imaging characteristics to determine pixel correlation for different parallax-induced shifts;keeping track of various pair-wise measurements and calculating a parallax difference that yields the highest pixel correlation ignoring pixels in the images that are in an exposed occlusion set; andgenerating a depth map using the parallax confirmation and measurement module based upon the measured parallax. 2. The method of claim 1, wherein: the calibration data indicates mappings between addresses of physical pixels in imagers and logical addresses within an image; andthe address conversion module resolves phase differences between the set of images by converting addresses of physical pixels in imagers to logical addresses. 3. The method of claim 1, wherein calculating a parallax difference that yields the highest pixel correlation comprises determining parallax based upon the set of pair-wise measurements that have the best least squares fit. 4. The method of claim 1, wherein aligning portions of images captured by different imagers to compensate for parallax using an image pixel correlation module further comprises determining appropriate X and Y offsets to be applied to logical pixel address calculations using an address conversion module based upon the detected and metered parallax and the stored calibration data. 5. The method of claim 1, further comprising: aligning portions of different images to compensate for parallax using an image pixel correlation module based upon the measured parallax and the stored calibration data; andobtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images using a super-resolution module. 6. The method of claim 5, wherein obtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images comprises fusing the aligned image portions. 7. The method of claim 6, wherein obtaining a higher resolution image having a resolution that is higher than the resolutions of the images comprises performing super-resolution processing on the fused image portions to synthesize a super-resolution image. 8. The method of claim 5, wherein obtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images comprises performing super-resolution processing on the aligned image portions to synthesize a super-resolution image. 9. The method of claim 5, further comprising selecting at least one distance as a focal plane and applying blurring to pixels in the higher resolution image with depths in the depth map that are not proximate a focal plane using the super-resolution processing module. 10. The method of claim 1, further comprising estimating distance to an object visible in the set of images using the parallax confirmation and measurement module. 11. The method of claim 1, further comprising measuring the absolute size and shape of a physical item visible in the set of images using the parallax confirmation and measurement module. 12. The method of claim 1, wherein the set of images is captured by an array of between 2×2 and 6×6 imagers. 13. The method of claim 1, wherein the set of images is captured by a set of imagers that form at least a 3×3 configuration of imagers comprising: an imager having a green filter at the center of a 3×3 configuration of imagers;two imagers having red filters located on opposite sides of the 3×3 configuration of imagers;two imagers having blue filters located on opposite sides of the 3×3 configuration of imagers; andfour imagers having green filters surrounding the central imager having a green filter. 14. The method of claim 1, wherein the set of images is captured by a linear array of imagers. 15. The method of claim 1, wherein the linear array of imagers is selected from the group consisting of: a 1×4 array of imagers; and a 1×10 array of imagers. 16. The method of claim 1, wherein each image in the set of images is obtained by an imager including a filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral filters. 17. The method of claim 1, wherein at least one of the images in the set of images is captured by an imager selected from the group consisting of: a polychromatic imager; and a near-IR imager. 18. The method of claim 1, wherein imagers in the array of imagers have the same resolution. 19. The method of claim 1, wherein the images in the set of images have different resolutions. 20. A method of generating a depth map from a plurality of images captured from different viewpoints, comprising: normalizing a set of images using calibration data stored in a storage device using an address conversion module, where: the set of images comprises a plurality of images that are: captured from different viewpoints;include different occlusions sets;wherein the occlusion set of a first image is the portion of a scene visible in a second image that is occluded in the first image; andthe set of images includes: images captured using the same imaging characteristics; andimages captured using different imaging characteristics;detecting and metering parallax using a parallax confirmation and measurement module, where detecting and metering parallax comprises: determining the parallax that yields the highest correlation between pixels from images captured using the same imaging characteristics accounting for the positions of the imagers that captured the images by: performing redundant pair-wise measurements between pixels from images captured using the same imaging characteristics to determine pixel correlation for different parallax-induced shifts;keeping track of various pair-wise measurements and calculating a parallax difference that yields the highest pixel correlation ignoring pixels in the images that are in an exposed occlusion set; andaligning portions of different images to compensate for parallax using an image pixel correlation module based upon the measured parallax and the stored calibration data; andobtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images using a super-resolution module; andgenerating a depth map for the higher resolution image using the parallax confirmation and measurement module based upon the measured parallax;wherein obtaining an image having a resolution that is higher than the resolutions of the images in the set of images comprises: fusing the aligned image portions;performing super-resolution processing on the fused image portions to synthesize a super-resolution image; andselecting at least one distance as a focal plane and applying blurring to pixels in the higher resolution image with depths in the depth map that are not proximate a focal plane.

이 특허에 인용된 특허 (80)

Wilburn, Bennett; Joshi, Neel; Levoy, Marc C.; Horowitz, Mark, Apparatus and method for capturing a scene using staggered triggering of dense camera arrays.
상세보기
Adamo, Thomas J.; Montelius, Lars, Apparatus for imaging using an array of lenses.
상세보기
Montalbano Anthony J. (Lake Barrington Shores IL), Apparatus for making a tool.
상세보기
Olsen,Richard Ian; Sato,Darryl L.; Moller,Borden; Vitomirov,Olivera; Brady,Jeffrey A.; Gunawan,Ferry; Oten,Remzi; Sun,Feng Qing; Gates,James, Apparatus for multiple camera devices and method of operating same.
상세보기
Boisvert, David Michael; McMahon, Andrew Kenneth John, CCD output processing stage that amplifies signals from colored pixels based on the conversion efficiency of the colored pixels.
상세보기
Lambert, Howard Shelton, Camera system for three dimensional images and video.
상세보기
Fossum Eric R. ; Zhou Zhimin ; Pain Bedabrata, Capacitively coupled successive approximation ultra low power analog-to-digital converter.
상세보기
Duparre, Jacques, Capturing and processing of images using monolithic camera array with heterogeneous imagers.
상세보기
Meyers Mark Marshall, Compact digital camera with segmented fields of view.
상세보기
Meyers Mark Marshall, Compact image capture device.
상세보기
Murayama Tomio (Kanagawa JPX) Moriguchi Fujio (Kanagawa JPX) Kurata Masami (Kanagawa JPX) Omori Takashi (Kanagawa JPX), Control system for reading device.
상세보기
Satoh,Kiyohide, Depth information measurement apparatus and mixed reality presentation system.
상세보기
Kobayashi, Akio; Okada, Hidefumi, Digital camera which produces a single image based on two exposures.
상세보기
Yu, Yanbin; Zhang, Zhongxuan, Digital cameras using multiple sensors with multiple lenses.
상세보기
Tomikawa, Masahiko; Watanabe, Ryoichi; Sasabayashi, Takashi, Display device.
상세보기
Ward, Gregory John; Seetzen, Helge; Heidrich, Wolfgang, Electronic camera having multiple sensors for capturing high dynamic range images and related methods.
상세보기
Grage Ludger (Eichenau DEX) Dato Paul (Emmering DEX), FLIR system with two optical channels for observing a wide and a narrow field of view.
상세보기
Duparre, Jacques, Fabrication process for mastering imaging lens arrays.
상세보기
Berestov, Alexander, Fast epipolar line adjustment of stereo pairs.
상세보기
Mack Walter J., Focal plane processor for scaling information from image sensors.
상세보기
Ishak, Andrew W.; Haddock, Joshua N.; Kokonaski, William; Duston, Dwight P.; Iyer, Venkatramani S.; Blum, Ronald D.; McGinnis, Sean P.; Packard, Michael B., High performance selective light wavelength filtering providing improved contrast sensitivity.
상세보기
Stettner Roger (Santa Barbara CA) Bailey Howard (Santa Barbara CA), High speed crystallography detector.
상세보기
Jacobowitz,Lawrence; DeCusatis,Casimer, Hybrid optical/electronic structures fabricated by a common molding process.
상세보기
Nakano, Mikio; Nagai, Takayuki; Mizutani, Akira; Nakamura, Tomoaki; Funakoshi, Kotaro, Image capturing device, method of searching for occlusion region, and program.
상세보기
Hashimoto, Seiji; Yuki, Osamu; Endo, Toshiaki; Hayashi, Hidetoshi, Image sensing apparatus.
상세보기
Suda,Yasuo, Image sensing apparatus and its control method, control program, and storage medium.
상세보기
Richards, Austin A.; D'Souza, Shariff, Infrared and near-infrared camera hyperframing.
상세보기
Jack Michael D. ; Ray Michael ; Wyles Richard H., Integrated IR, visible and NIR sensor and methods of fabricating same.
상세보기
Burger Robert J., Lenslet array systems and methods.
상세보기
Burger Robert J., Lenslet array systems and methods.
상세보기
Marcellin, Michael W.; Bilgin, Ali; Lalgudi, Hariharan G.; Nadar, Mariappan S., Lifting-based view compensated compression and remote visualization of volume rendered images.
상세보기
Abell Gurdon R. (West Woodstock CT) Cook Francis J. (Topsfield MA) Howes Peter D. (Sudbury MA), Method and apparatus for arraying image sensor modules.
상세보기
Lelescu, Dan; Mullis, Robert; Rao, Pravin; Venkataraman, Kartik; Lu, Cheng; Chen, Junqing, Method and apparatus for extended depth-of-field image restoration.
상세보기
Telymonde Timothy D., Method and apparatus for image acquisition from a plurality of cameras.
상세보기
Tsutsumi Teruo (Tama JPX), Method and apparatus for reading image information formed on material.
상세보기
Sawhney,Harpreet Singh; Tao,Hai; Kumar,Rakesh; Hanna,Keith, Method and apparatus for synthesizing new video and/or still imagery from a collection of real video and/or still imagery.
상세보기
Schmitt, Peter; Schmidt, Dieter; Kostka, Guenther; Behrendt, Rolf, Method and camera (apparatus) for optically capturing a screen.
상세보기
Tam, Wa James; Vázquez, Carlos, Method and graphical user interface for modifying depth maps.
상세보기
Vetro, Anthony; Yea, Sehoon; Matusik, Wojciech; Pfister, Hanspeter; Zwicker, Matthias, Method and system for decoding and displaying 3D light fields.
상세보기
Edgar, Albert D., Method and system for enhancing digital images.
상세보기
Levoy Marc ; Hanrahan Pat, Method and system for light field rendering.
상세보기
Wu, Jeng-Feng; Chen, Wen-Chao; Tang, Cheng-Yuan, Method for producing image with depth by using 2D images.
상세보기
Portniaguine, Oleg N.; Zhdanov, Michael S., Method of digital image enhancement and sharpening.
상세보기
Georgiev, Todor G.; Lumsdaine, Andrew, Methods and apparatus for reducing plenoptic camera artifacts.
상세보기
Georgiev, Todor G.; Lumsdaine, Andrew, Methods and apparatus for rich image capture with focused plenoptic cameras.
상세보기
Nishikawa, Takao, Microlens array substrate, method of manufacturing the same, and display device.
상세보기
Levoy, Marc S.; Ng, Yi-Ren; Horowitz, Mark A., Microscopy arrangements and approaches.
상세보기
Alexander David H. (Santa Monica CA) Hershman George H. (Carlsbad CA) Jack Michael D. (Carlsbad CA) Koda N. John (Vista CA) Lloyd Randahl B. (San Marcos CA), Monolithic imager for near-IR.
상세보기
Smith, Scott, Multi-array sensor with integrated sub-array for parallax detection and photometer functionality.
상세보기
Bosco,Angelo; Mancuso,Massimo, Noise filter for Bayer pattern image data.
상세보기
Blackwell, Jr., Richard J., On-wafer butted microbolometer imaging array.
상세보기
Duparre, Jacques, Optical arrangements for use with an array camera.
상세보기
Chow Alan Y. ; Chow Vincent Y., Optical processor based imaging system.
상세보기
Thompson ; Kenneth B., Optical viewing apparatus.
상세보기
Tangen, Reidar E.; Gudesen, Hans Gude; Nordal, Per-Erik; Leistad, Geirr I., Optoelectronic camera and method for image formatting in the same.
상세보기
Philip J. Ireland ; James E. Green, Passivation integrity improvements.
상세보기
Dierickx Bart,BEX ; Ricquier Nico,BEX, Pixel structure, image sensor using such pixel structure and corresponding peripheral circuitry.
상세보기
Shum, Heung Yeung; Sun, Jian; Yamazaki, Shuntaro; Li, Yin, Pop-up light field.
상세보기
Gallery Richard D.,GBX ; Bliss Nathan D.,GBX, Post-processing generation of focus/defocus effects for computer graphics images.
상세보기
Omatsu, Shigeru; Mitsukura, Yasue; Fukumi, Minoru; Mitsukura, Kensuke; Kita, Koji; Kugo, Kouichi, Region detecting method and region detecting apparatus.
상세보기
Hornbaker ; III Cecil V. (New Carrolton MD) Driggers Thomas C. (Falls Church VA) Bindon Edward W. (Fairfax VA), Scanning apparatus using multiple CCD arrays and related method.
상세보기
Tan, Kar-Han; Xiao, Jing, Small memory footprint light transport matrix capture.
상세보기
Honda, Hiroto; Iida, Yoshinori; Egawa, Yoshitaka; Itoh, Goh, Solid-state image sensor.
상세보기
Rennie, John, Solid-state imaging device.
상세보기
Inuiya, Masafumi, Solid-state imaging device, imaging apparatus, and method for manufacturing solid-state imaging device.
상세보기
Suda, Yasuo, Structure of semiconductor device with improved reliability.
상세보기
Georgiev, Todor G.; Lumsdaine, Andrew, Super-resolution with the focused plenoptic camera.
상세보기
Bassi, Zorawar; Vakili, Masoud, System and method for automated calibration and correction of display geometry and color.
상세보기
Venkataraman, Kartik; Nisenzon, Semyon, Systems and methods for decoding light field image files.
상세보기
Venkataraman, Kartik; Nisenzon, Semyon, Systems and methods for decoding light field image files.
상세보기
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for parallax detection and correction in images captured using array cameras that contain occlusions using subsets of images to perform depth estimation.
상세보기
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for performing depth estimation using image data from multiple spectral channels.
상세보기
McMahon, Andrew Kenneth John, Systems and methods for transmitting and receiving array camera image data.
상세보기
McMahon, Andrew Kenneth John, Systems and methods for transmitting and receiving array camera image data.
상세보기
Boettiger, Ulrich C., Uniform color filter arrays in a moat.
상세보기
Joseph, Domenick Montalbo, Use of a lenslet array with a vertically stacked pixel array.
상세보기
Berezin, Vladimir, Using a single control line to provide select and reset signals to image sensors in two rows of a digital imaging device.
상세보기
Tan, Kar-Han; Sonehara, Tomio, View projection matrix based high performance low latency display pipeline.
상세보기
Blake,Andrew; Criminisi,Antonio, Virtual image artifact detection.
상세보기
Rieger Albert,DEX ; Barclay David ; Chapman Steven ; Kellner Heinz-Andreas,DEX ; Reibl Michael,DEX ; Rydelek James G. ; Schweizer Andreas,DEX, Watertight body for accommodating a photographic camera.
상세보기

IPC	Description
A	생활필수품
A62	인명구조; 소방(사다리 E06C)
A62B	인명구조용의 기구, 장치 또는 방법(특히 의료용에 사용되는 밸브 A61M 39/00; 특히 물에서 쓰이는 인명구조 장치 또는 방법 B63C 9/00; 잠수장비 B63C 11/00; 특히 항공기에 쓰는 것, 예. 낙하산, 투출좌석 B64D; 특히 광산에서 쓰이는 구조장치 E21F 11/00)
A62B-1/08	.. 윈치 또는 풀리에 제동기구가 있는 것

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 국가코드, 자료구분, 상태, 출원번호, 출원일자, 공개번호, 공개일자, 등록번호, 등록일자, 발명명칭(한글), 발명명칭(영문), 출원인(한글), 출원인(영문), 출원인코드, 대표IPC 관리번호, 국가코드, 자료구분, 상태, 출원번호, 출원일자, 공개번호, 공개일자, 공고번호, 공고일자, 등록번호, 등록일자, 발명명칭(한글), 발명명칭(영문), 출원인(한글), 출원인(영문), 출원인코드, 대표출원인, 출원인국적, 출원인주소, 발명자, 발명자E, 발명자코드, 발명자주소, 발명자 우편번호, 발명자국적, 대표IPC, IPC코드, 요약, 미국특허분류, 대리인주소, 대리인코드, 대리인(한글), 대리인(영문), 국제공개일자, 국제공개번호, 국제출원일자, 국제출원번호, 우선권, 우선권주장일, 우선권국가, 우선권출원번호, 원출원일자, 원출원번호, 지정국, Citing Patents, Cited Patents
저장형식	Text(ASCII format) Excel format PIAS분석(.xls)
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

Systems and methods for generating depth maps using images captured by camera arrays 원문보기

초록 ▼

대표청구항 ▼

연구과제 타임라인

이 특허에 인용된 특허 (80)

관련 콘텐츠

특허 원문 보기

IPC 상위 출원인

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

연합인증

Systems and methods for generating depth maps using images captured by camera arrays 원문보기

초록 ▼

대표청구항 ▼

연구과제 타임라인

전체(0) 논문(0) 특허(0) 보고서(0)

전체(0) 논문(0) 특허(0) 보고서(0)

이 특허에 인용된 특허 (80)

관련 콘텐츠

특허 원문 보기

IPC 상위 출원인

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트