Capturing and processing of images using monolithic camera array with heterogeneous imagers
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-005/225
H04N-001/195
H04N-009/04
H01L-027/146
H04N-005/369
H04N-003/14
H04N-005/232
H04N-009/083
출원번호
US-0935504
(2009-05-20)
등록번호
US-8902321
(2014-12-02)
국제출원번호
PCT/US2009/044687
(2009-05-20)
§371/§102 date
20100929
(20100929)
국제공개번호
WO2009/151903
(2009-12-17)
발명자
/ 주소
Venkataraman, Kartik
Jabbi, Amandeep S.
Mullis, Robert H.
출원인 / 주소
Pelican Imaging Corporation
대리인 / 주소
KPPB LLP
인용정보
피인용 횟수 :
111인용 특허 :
64
초록▼
A camera array, an imaging device and/or a method for capturing image that employ a plurality of imagers fabricated on a substrate is provided. Each imager includes a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager havin
A camera array, an imaging device and/or a method for capturing image that employ a plurality of imagers fabricated on a substrate is provided. Each imager includes a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. Each imager may be associated with an optical element fabricated using a wafer level optics (WLO) technology.
대표청구항▼
1. A camera array, comprising: a plurality of cameras forming an array comprising a plurality of rows of cameras and a plurality of columns of cameras, where each camera includes separate optics, and a plurality of sensor elements, and each camera is configured to independently capture an image of a
1. A camera array, comprising: a plurality of cameras forming an array comprising a plurality of rows of cameras and a plurality of columns of cameras, where each camera includes separate optics, and a plurality of sensor elements, and each camera is configured to independently capture an image of a scene;wherein the optics of each camera are configured so that each camera has a field of view of the same scene that is shifted with respect to the field-of-views of the other cameras so that each shift includes a sub-pixel shifted view of the same scene;wherein the camera array is a monolithic integrated module comprising: a single semiconductor substrate on which all of the sensor elements and control circuitry are formed, where the control circuitry is configured to enable each camera to operate independently; andoptics including a plurality of lens elements, where each lens element forms part of the separate optics for one of the cameras; andwherein the plurality of cameras comprises a plurality of different types of camera, where cameras of a given type share the same imaging characteristics and different types of cameras have different imaging characteristics including using different types of filters; andwherein one of the plurality of cameras includes a central camera and the plurality of cameras forms: at least one row of cameras above the central camera;at least one row of cameras below the central camera;at least one column of cameras to the left of the central camera; andat least one column of cameras to the right of the central camera;wherein cameras having the same imaging characteristics and filter are located in locations selected from the group consisting of:locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; andlocations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera. 2. The camera array of claim 1, wherein at least one of each different type of camera is located in each quadrant of the camera array. 3. The camera array of claim 2, wherein the cameras of a given type of camera are symmetrically distributed about the central camera. 4. The camera array of claim 2, wherein each camera includes either a Blue filter, a Green filter, or a Red filter. 5. The camera array of claim 4, wherein: green cameras are located in locations including: at least one row in the camera array above the central camera;at least one row in the camera array below the central camera;at least one column in the camera array to the left of the central camera; andat least one column in the camera array to the right of the central camera. 6. The camera array of claim 5, wherein: red cameras are located in locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; andblue cameras are located in locations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera. 7. The camera array of claim 5, wherein: blue cameras are located in locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; andred cameras are located in locations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera. 8. The camera array of claim 2, wherein each camera includes either a Blue filter, a Green filter, a Red filter, or a near-IR filter. 9. The camera array of claim 1, wherein each camera includes a filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral filters. 10. The camera array of claim 1, wherein different types of cameras are configured by the control circuitry to operate with at least one difference in operating parameters. 11. The camera array of claim 10, wherein the at least one difference in operating parameters includes at least one imaging parameter selected from the group consisting of exposure time, gain, and black level offset. 12. The camera array of claim 10, wherein the control circuitry is configured to stagger the start of the exposure periods for sensor elements in different cameras so that a scene is sampled sequentially while having the sensor elements in several cameras exposed to light at the same time. 13. The camera array of claim 1, wherein the camera array is at least a 4×4 array of cameras. 14. The camera array of claim 13, wherein the camera array is a 4×4 array of cameras. 15. The camera array of claim 13, wherein the camera array is a 5×5 array of cameras. 16. The camera array of claim 13, wherein the camera array is a 6×6 array of cameras. 17. The camera array of claim 1, wherein: the plurality of cameras includes different types of cameras dedicated to sampling luma and dedicated to sampling chroma; andcameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax. 18. The camera array of claim 1, wherein the central camera is a polychromatic camera. 19. The camera array of claim 18, wherein: the plurality of cameras also includes a plurality of cameras dedicated to sampling chroma; andcameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax. 20. A linear camera array, comprising: a plurality of cameras forming a linear array, where each camera includes separate optics, and a plurality of sensor elements, and each camera is configured to independently capture an image of a scene;wherein the optics of each camera are configured so that each camera has a field of view that is shifted with respect to the field-of-views of the other cameras so that each shift includes a sub-pixel shifted view of the scene;wherein the linear camera array is a monolithic integrated module comprising: a single semiconductor substrate on which all of the sensor elements and control circuitry are formed, where the control circuitry is configured to enable each camera to operate independently; andoptics including a plurality of lens elements, where each lens element forms part of the separate optics for one of the cameras; andwherein the plurality of cameras comprises a plurality of different types of camera, where cameras of a given type share the same imaging characteristics and different types of cameras have different imaging characteristics including using different types of filters; andwherein one of the plurality of cameras forms a central camera;wherein cameras having the same imaging characteristics and filter are located on either side of the central camera. 21. The camera array of claim 20, wherein the cameras that include a specific type of filter are symmetrically distributed about the central camera. 22. The camera array of claim 20, wherein each camera includes either a Blue filter, a Green filter, or a Red filter. 23. The camera array of claim 20, wherein each camera includes either a Blue filter, a Green filter, a Red filter, or a near-IR filter. 24. The camera array of claim 20, wherein each camera includes a filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral filters. 25. The camera array of claim 20, wherein different types of cameras operate with at least one difference in operating parameters. 26. The camera array of claim 25, wherein the at least one difference in operating parameters includes at least one imaging parameter selected from the group consisting of exposure time, gain, and black level offset. 27. The camera array of claim 25, wherein the control circuitry is configured to stagger the start of the exposure periods for sensor elements in different cameras so that a scene is sampled sequentially while having the sensor elements in several cameras exposed to light at the same time. 28. The camera array of claim 20, wherein: the plurality of cameras includes different types of cameras dedicated to sampling luma and dedicated to sampling chroma; andcameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax. 29. The camera array of claim 20, wherein the central camera is a polychromatic camera. 30. The camera array of claim 29, wherein: the plurality of cameras also includes a plurality of cameras dedicated to sampling chroma; andcameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (64)
Wilburn, Bennett; Joshi, Neel; Levoy, Marc C.; Horowitz, Mark, Apparatus and method for capturing a scene using staggered triggering of dense camera arrays.
Boisvert, David Michael; McMahon, Andrew Kenneth John, CCD output processing stage that amplifies signals from colored pixels based on the conversion efficiency of the colored pixels.
Ward, Gregory John; Seetzen, Helge; Heidrich, Wolfgang, Electronic camera having multiple sensors for capturing high dynamic range images and related methods.
Abell Gurdon R. (West Woodstock CT) Cook Francis J. (Topsfield MA) Howes Peter D. (Sudbury MA), Method and apparatus for arraying image sensor modules.
Sawhney,Harpreet Singh; Tao,Hai; Kumar,Rakesh; Hanna,Keith, Method and apparatus for synthesizing new video and/or still imagery from a collection of real video and/or still imagery.
Alexander David H. (Santa Monica CA) Hershman George H. (Carlsbad CA) Jack Michael D. (Carlsbad CA) Koda N. John (Vista CA) Lloyd Randahl B. (San Marcos CA), Monolithic imager for near-IR.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for parallax detection and correction in images captured using array cameras that contain occlusions using subsets of images to perform depth estimation.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for performing depth estimation using image data from multiple spectral channels.
Rieger Albert,DEX ; Barclay David ; Chapman Steven ; Kellner Heinz-Andreas,DEX ; Reibl Michael,DEX ; Rydelek James G. ; Schweizer Andreas,DEX, Watertight body for accommodating a photographic camera.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Array cameras including an array camera module augmented with a separate camera.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating monolithic array camera modules with high MTF lens stacks for capture of images used in super-resolution processing.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating optics with modulation transfer functions greater than sensor Nyquist frequency for capture of images used in super-resolution processing.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit K.; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Autofocus system for a conventional camera that uses depth information from an array camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including cameras dedicated to sampling luma and cameras dedicated to sampling chroma.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including heterogeneous optics.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using camera array incorperating Bayer cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of near-IR images including occlusions using camera arrays incorporating near-IR light sources.
Srikanth, Manohar; Ramamoorthi, Ravi; Venkataraman, Kartik; Chatterjee, Priyam, System and methods for depth regularization and semiautomatic interactive matting using RGB-D images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, System and methods for measuring depth using an array camera employing a bayer filter.
Nayar, Shree; Venkataraman, Kartik; Pain, Bedabrata; Lelescu, Dan, Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Duparré, Jacques, Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for generating depth maps and corresponding confidence maps indicating depth estimation reliability.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for generating depth maps using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using images captured by camera arrays incorporating cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using light focused on an image sensor by a lens element array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth in the presence of occlusions using a subset of images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth using an array of independently controllable cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by a camera array including cameras surrounding a central camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by monolithic camera arrays including at least one bayer camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for normalizing image data captured by camera arrays.
Venkataraman, Kartik; Huang, Yusong; Jain, Ankit K.; Chatterjee, Priyam, Systems and methods for performing high speed video capture and depth estimation using array cameras.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using a set of geometrically registered images.
Lelescu, Dan; Duong, Thang, Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for synthesizing higher resolution images using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for synthesizing higher resolution images using images captured by camera arrays.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.