Capturing and processing of images using monolithic camera array with heterogeneous imagers
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G02B-027/10
G03B-021/56
G03B-021/60
H01L-027/146
G02B-013/00
H04N-005/225
H04N-005/341
H04N-009/04
출원번호
US-0948054
(2013-07-22)
등록번호
US-8861089
(2014-10-14)
발명자
/ 주소
Duparre, Jacques
출원인 / 주소
Pelican Imaging Corporation
대리인 / 주소
KPPB LLP
인용정보
피인용 횟수 :
108인용 특허 :
58
초록▼
Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance wit
Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance with many embodiments of the invention include lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form a plurality of optical channels, at least one aperture located within each optical channel, at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light, and light blocking materials located within the lens stack array to optically isolate the optical channels.
대표청구항▼
1. A camera array, comprising: a lens stack array, comprising: lens elements arranged to form a plurality of optical channels, where the optical channels are configured to form an image of an object space within a scene from slightly different viewpoints;at least one aperture located within each opt
1. A camera array, comprising: a lens stack array, comprising: lens elements arranged to form a plurality of optical channels, where the optical channels are configured to form an image of an object space within a scene from slightly different viewpoints;at least one aperture located within each optical channel;at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light; andlight blocking materials located within the lens stack array to optically isolate the optical channels; andan imager array, comprising:a plurality of imagers, where each imager comprises at least two pixels; andcircuitry for controlling imaging parameters that is configured to trigger each imager independently;wherein the plurality of optical channels and the plurality of imagers form a plurality of cameras configured to capture different low resolution images representing slightly different viewpoints of the same scene that provide non-redundant information about the scene;wherein the low resolution images captured by the plurality of cameras include different occlusions sets, where the occlusion set of a given camera is the portion of a scene visible to a first of the plurality of cameras that is occluded from the view of the given camera;wherein the shape of at least one lens element forming a first optical channel for a camera of a first type that includes a spectral filter configured to pass a first spectral band of light differs from the shape of at least one lens element forming a second optical channel for a camera of a second type that includes a spectral filter configured to pass a second spectral band of light; andwherein the plurality of cameras includes at least one camera of each type that captures pixels around an edge of a foreground object that is visible to the first of the plurality of cameras and is in the occlusion set of another of the plurality of cameras. 2. The camera array of claim 1, wherein the light blocking materials in the lens stack array are selected from the group consisting of opaque materials, reflective materials and combinations thereof. 3. The camera array of claim 1, wherein each spectral filter is selected from the group consisting of an organic color filter, an absorptive material, a dielectric coating, an interference filter, a multilayer coating, and combinations thereof. 4. The camera array of claim 1, further comprising at least one polarizing filter located within each optical channel. 5. The camera array of claim 1, wherein the prescription of at least one surface of a lens within each optical channel is a function of the specific spectral band of light passed by the spectral filter within the optical channel. 6. The camera array of claim 5, wherein the back focal lengths of each optical channel in the lens stack array are the same irrespective of the spectral band of light passed by the spectral filter within the optical channel. 7. The camera array of claim 1, wherein a combination of high and low Abbe number materials is used in the construction of each of the lens elements in an optical channel to reduce chromatic aberrations. 8. The camera array of claim 1, wherein each spectral filter is located within each optical channel so that the spectral filter is proximate the aperture stop. 9. The camera array of claim 8, wherein each aperture stop is formed by a light blocking material selected from the group consisting of metal materials, oxide materials, black particle filled photoresists and combinations thereof. 10. The camera array of claim 1, wherein: at least one lens surface of a lens in each optical channel differs based upon the specific spectral band of light passed by the spectral filter within the optical channel; andeach lens surface is selected from the group consisting of diffractive, Fresnel, refractive and combinations thereof. 11. The camera array of claim 10, wherein the radii of curvature of the lens surfaces differ based upon the specific spectral band of light passed by the spectral filter within the optical channel. 12. The camera array of claim 10, wherein each of the optical channels have the same back focal length. 13. The camera array of claim 1, wherein at least one of the lens elements in an optical channel is a negative lens element, and the negative lens element is proximate the image formed by the optical channel. 14. The camera array of claim 1, wherein at least two of the optical channels have different focal lengths. 15. The camera array of claim 1, further comprising a mechanical zoom mechanism within an optical channel configured to smoothly transition between different fields of view. 16. The camera array of claim 1, wherein the lens stack array comprises an M×N array of optical channels, where at least one of N and M is greater than 2. 17. The camera array of claim 1, wherein the lens stack array is fabricated using techniques consisting of wafer level optics techniques, injection molding, glass molding and combinations thereof. 18. The camera array of claim 1, wherein the light blocking materials located within the lens stack array to optically isolate the optical channels comprise at least two opaque surfaces located on a substrate within an optical channel, where the two opaque surfaces have openings arranged in axial alignment with the optical channel. 19. The camera array of claim 1, where the sampling diversity of the low resolution images is dependent upon distance to the part of the object space that is sampled. 20. The camera array of claim 1, wherein: the lens elements in the lens stack array are formed on substrates separated by spacers; andat least one spectral filter is located within a space defined by the substrates separated by spacers. 21. The camera array of claim 10, wherein at least one spectral filter is disposed on the surface of a lens element within the lens stack array. 22. The camera array of claim 1, wherein at least one spectral filter is located on the surface of the imager array. 23. The camera array of claim 1, wherein at least one camera is a polychromatic camera. 24. The camera array of claim 1, wherein at least one camera is a near-IR camera. 25. The camera array of claim 1, wherein the at least one spectral filter located within each optical channel forms at least a 3×3 array of cameras comprising: a camera having a green filter at the center of a 3×3 configuration of cameras;two cameras having red filters located on opposite sides of the 3×3 configuration of cameras;two cameras having blue filters located on opposite sides of the 3×3 configuration of cameras; andfour cameras having green filters surrounding the central camera having a green filter. 26. The camera array of claim 1, wherein the camera array comprises an array of between 2×2 and 6×6 cameras. 27. The camera array of claim 26, wherein the camera array comprises a 3×3 array of cameras. 28. The camera array of claim 26, wherein the camera array comprises a 4×4 array of cameras. 29. The camera array of claim 26, wherein the camera array comprises a 5×5 array of cameras. 30. The camera array of claim 1, wherein the camera array comprises a one-dimensional array of cameras. 31. The camera array of claim 30, wherein the camera array comprises a 1×10 array of cameras. 32. The camera array of claim 1, wherein the pixels captured by the at least one camera of each type that captures pixels around an edge of a foreground object that is visible to the first of the plurality of cameras and is in the occlusion set of another of the plurality of cameras enable reconstruction of color information around the edge of the foreground object that is visible to the first of the plurality of cameras.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (58)
Wilburn, Bennett; Joshi, Neel; Levoy, Marc C.; Horowitz, Mark, Apparatus and method for capturing a scene using staggered triggering of dense camera arrays.
Boisvert, David Michael; McMahon, Andrew Kenneth John, CCD output processing stage that amplifies signals from colored pixels based on the conversion efficiency of the colored pixels.
Abell Gurdon R. (West Woodstock CT) Cook Francis J. (Topsfield MA) Howes Peter D. (Sudbury MA), Method and apparatus for arraying image sensor modules.
Sawhney,Harpreet Singh; Tao,Hai; Kumar,Rakesh; Hanna,Keith, Method and apparatus for synthesizing new video and/or still imagery from a collection of real video and/or still imagery.
Alexander David H. (Santa Monica CA) Hershman George H. (Carlsbad CA) Jack Michael D. (Carlsbad CA) Koda N. John (Vista CA) Lloyd Randahl B. (San Marcos CA), Monolithic imager for near-IR.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for performing depth estimation using image data from multiple spectral channels.
Rieger Albert,DEX ; Barclay David ; Chapman Steven ; Kellner Heinz-Andreas,DEX ; Reibl Michael,DEX ; Rydelek James G. ; Schweizer Andreas,DEX, Watertight body for accommodating a photographic camera.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Array cameras including an array camera module augmented with a separate camera.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating monolithic array camera modules with high MTF lens stacks for capture of images used in super-resolution processing.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating optics with modulation transfer functions greater than sensor Nyquist frequency for capture of images used in super-resolution processing.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit K.; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Autofocus system for a conventional camera that uses depth information from an array camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including cameras dedicated to sampling luma and cameras dedicated to sampling chroma.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including heterogeneous optics.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using camera array incorperating Bayer cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of near-IR images including occlusions using camera arrays incorporating near-IR light sources.
Srikanth, Manohar; Ramamoorthi, Ravi; Venkataraman, Kartik; Chatterjee, Priyam, System and methods for depth regularization and semiautomatic interactive matting using RGB-D images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, System and methods for measuring depth using an array camera employing a bayer filter.
Nayar, Shree; Venkataraman, Kartik; Pain, Bedabrata; Lelescu, Dan, Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Duparré, Jacques, Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for generating depth maps and corresponding confidence maps indicating depth estimation reliability.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for generating depth maps using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using images captured by camera arrays incorporating cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using light focused on an image sensor by a lens element array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth in the presence of occlusions using a subset of images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth using an array of independently controllable cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by a camera array including cameras surrounding a central camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by monolithic camera arrays including at least one bayer camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for normalizing image data captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for parallax measurement using camera arrays incorporating 3 x 3 camera configurations.
Venkataraman, Kartik; Huang, Yusong; Jain, Ankit K.; Chatterjee, Priyam, Systems and methods for performing high speed video capture and depth estimation using array cameras.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using a set of geometrically registered images.
Lelescu, Dan; Duong, Thang, Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for synthesizing higher resolution images using a set of images containing a baseline image.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.