Methods and apparatus for rich image capture with focused plenoptic cameras
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-005/225
G03B-013/00
G03B-003/00
G03B-007/099
G02B-021/00
출원번호
US-0917984
(2010-11-02)
등록번호
US-8345144
(2013-01-01)
발명자
/ 주소
Georgiev, Todor G.
Lumsdaine, Andrew
출원인 / 주소
Adobe Systems Incorporated
대리인 / 주소
Kowert, Robert C.
인용정보
피인용 횟수 :
174인용 특허 :
37
초록▼
Methods and apparatus for rich image capture using focused plenoptic camera technology. A radiance camera employs focused plenoptic camera technology and includes sets of modulating elements that may be used to modulate the sampling of different aspects of the range of plenoptic data. The radiance c
Methods and apparatus for rich image capture using focused plenoptic camera technology. A radiance camera employs focused plenoptic camera technology and includes sets of modulating elements that may be used to modulate the sampling of different aspects of the range of plenoptic data. The radiance camera, via the modulating elements, may capture a particular property of light, such as luminance, color, polarization, etc., differently in different microimages or in different portions of microimages. With the focused plenoptic camera technology, the microimages are captured at the same time in a single image. Thus, multiple microimages of the same image of a scene may be captured at different exposures, different colors, different polarities, and so on, in a single image at the same time. Captured images may be used, for example, in High Dynamic Range (HDR) imaging, spectral imaging, polarization imaging, 3D imaging, and other imaging applications.
대표청구항▼
1. A camera, comprising: a photosensor configured to capture light projected onto the photosensor;an objective lens, wherein the objective lens is configured to refract light from a scene located in front of the camera to form an image of the scene at an image plane of the objective lens;a microlens
1. A camera, comprising: a photosensor configured to capture light projected onto the photosensor;an objective lens, wherein the objective lens is configured to refract light from a scene located in front of the camera to form an image of the scene at an image plane of the objective lens;a microlens array positioned between the objective lens and the photosensor, wherein the microlens array comprises a plurality of microlenses, wherein the plurality of microlenses are focused on the image plane and not on the objective lens, wherein each microlens of the microlens array is configured to project a separate microimage of the image of the scene formed at the image plane by the objective lens onto a separate location on the photosensor; anda modulating element array comprising two or more modulating elements, wherein the modulating element array affects a property of light passing through the modulating array, wherein at least one of the modulating elements in the modulating element array affects the property of light differently than at least one other modulating element in the modulating element array to thus modulate the property of light in the separate microimages projected onto the separate locations on the photosensor by the microlens array. 2. The camera as recited in claim 1, wherein the modulating element array is proximate to the objective lens of the camera so that the modulating element array affects the property of light proximate to the objective lens. 3. The camera as recited in claim 2, wherein the modulating element array is located in front of the objective lens between the objective lens and the scene. 4. The camera as recited in claim 2, wherein the modulating element array is located behind the objective lens between the objective lens and the image plane. 5. The camera as recited in claim 2, wherein each of the separate microimages of the image of the scene projected onto the separate locations on the photosensor by the microlens array includes two or more modulated portions, wherein each modulated portion corresponds to a respective one of the modulating elements in the modulating element array. 6. The camera as recited in claim 1, wherein the modulating element array is integrated in the objective lens, wherein the objective lens is divided into two or more portions, and wherein each portion of the objective lens corresponds to a respective one of the modulating elements. 7. The camera as recited in claim 1, wherein each of the plurality of microlenses includes an aperture, wherein the apertures are approximately identical, wherein the modulating element array is located proximate to the microlens array so that the modulating element array affects the property of light proximate to the microlens array, and wherein each modulating element in the modulating element array corresponds to a particular one of the plurality of microlenses. 8. The camera as recited in claim 7, wherein the modulating element array is located between the microlens array and the photosensor. 9. The camera as recited in claim 1, wherein the modulating element array is the microlens array, wherein the modulating elements are the microlenses in the array, and wherein the modulation is provided by differently positioning different ones of the microlenses relative to a main optical axis of the camera or by interleaving microlenses with differing optical characteristics in the microlens array. 10. The camera as recited in claim 1, wherein the property of light is luminance, and wherein the modulating elements are neutral density filters that filter the light at two or more different levels. 11. The camera as recited in claim 1, wherein the property of light is color, and wherein the modulating elements are color filters that filter the light according to two or more different wavelengths. 12. The camera as recited in claim 1, wherein the property of light is polarity, and wherein the modulating elements are polarizing filters that filter the light according to two or more different polarities. 13. The camera as recited in claim 1, wherein the property of light is focal length, and wherein the modulating elements provide different focal lengths so that different portions of the image of the scene are in focus at the image plane. 14. The camera as recited in claim 1, wherein the modulating elements are sheets of glass with different refractive indexes to thus provide different optical paths for the light passing through the different sheets of glass. 15. The camera as recited in claim 1, wherein the photosensor is configured to capture a flat comprising the separate microimages, wherein each of the separate microimages is in a separate region of the flat, and wherein at least two of the separate microimages or at least two portions of each microimage differ according to the different effects of the modulating elements. 16. The camera as recited in claim 1, wherein the camera is a small camera with an aperture of approximately 1 millimeter or less at the objective lens. 17. The camera as recited in claim 16, wherein the property of light is luminance, wherein the modulating elements are neutral density filters that filter the light at two or more different levels, wherein the camera is configured to capture a flat comprising the separate microimages, and wherein the captured flat is configured to be processed according to one or more rendering methods to generate a high dynamic range (HDR) image of the scene. 18. A method, comprising: receiving light from a scene at an objective lens of a camera;refracting light from the objective lens to form an image of the scene at an image plane of the objective lens;receiving light from the image plane at a microlens array located between the objective lens and a photosensor of the camera and parallel to the image plane, wherein the microlens array comprises a plurality of microlenses, wherein each of the plurality of microlenses is focused on the image plane and not on the objective lens;receiving light from the microlens array at the photosensor, wherein the photosensor receives a separate microimage of the image of the scene formed at the image plane by the objective lens from each microlens of the array at a separate location on the photosensor; andmodulating a property of the light at a modulating element array comprising two or more modulating elements, wherein at least one of the modulating elements in the modulating element array affects the property of light differently than at least one other modulating element in the modulating element array to thus modulate the property of light in the separate microimages projected onto the separate locations on the photosensor by the microlens array. 19. The method as recited in claim 18, wherein the modulating element array is proximate to the objective lens of the camera so that the modulating element array modulates the property of the light proximate to the objective lens. 20. The method as recited in claim 18, wherein each of the plurality of microlenses includes an aperture, wherein the apertures are approximately identical, wherein the modulating element array is located proximate to the microlens array so that the modulating element array modulates the property of light proximate to the microlens array, and wherein each modulating element in the modulating element array corresponds to a particular one of the plurality of microlenses.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (37)
Georgiev Todor, 3D graphics based on images and morphing.
Yamagata, Michihiro; Okayama, Hiroaki; Boku, Kazutake; Tanaka, Yasuhiro; Hayashi, Kenichi; Fushimi, Yoshimasa; Murata, Shigeki; Hayashi, Takayuki, Imaging device including a plurality of lens elements and a imaging sensor.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Array cameras including an array camera module augmented with a separate camera.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating monolithic array camera modules with high MTF lens stacks for capture of images used in super-resolution processing.
Duparre, Jacques; Lelescu, Dan; Venkataraman, Kartik, Array cameras incorporating optics with modulation transfer functions greater than sensor Nyquist frequency for capture of images used in super-resolution processing.
Venkataraman, Kartik; Gallagher, Paul; Jain, Ankit K.; Nisenzon, Semyon; Lelescu, Dan; Ciurea, Florian; Molina, Gabriel, Autofocus system for a conventional camera that uses depth information from an array camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including cameras dedicated to sampling luma and cameras dedicated to sampling chroma.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images captured by camera arrays including heterogeneous optics.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions captured by arrays of luma and chroma cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images including occlusions focused on an image sensor by a lens stack array.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using camera array incorperating Bayer cameras having different fields of view.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images using monolithic camera array with heterogeneous imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using monolithic camera array with heterogeneous imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of near-IR images including occlusions using camera arrays incorporating near-IR light sources.
Georgiev, Todor G.; Chunev, Georgi N., Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data.
Georgiev, Todor G.; Lumsdaine, Andrew, Methods, apparatus, and computer-readable storage media for depth-based rendering of focused plenoptic camera data.
Srikanth, Manohar; Ramamoorthi, Ravi; Venkataraman, Kartik; Chatterjee, Priyam, System and methods for depth regularization and semiautomatic interactive matting using RGB-D images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, System and methods for measuring depth using an array camera employing a bayer filter.
Nayar, Shree; Venkataraman, Kartik; Pain, Bedabrata; Lelescu, Dan, Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Lelescu, Dan; Venkataraman, Kartik, Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing.
Duparré, Jacques, Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan; Venkataraman, Kartik; Molina, Gabriel, Systems and methods for detecting defective camera arrays and optic arrays.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Venkataraman, Kartik; Lelescu, Dan; Molina, Gabriel, Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for generating depth maps and corresponding confidence maps indicating depth estimation reliability.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for generating depth maps using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for generating depth maps using images captured by camera arrays incorporating cameras having different fields of view.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Duparre, Jacques; McMahon, Andrew Kenneth John; Lelescu, Dan, Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth in the presence of occlusions using a subset of images.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for measuring depth using an array of independently controllable cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by a camera array including cameras surrounding a central camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for measuring depth using images captured by monolithic camera arrays including at least one bayer camera.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for normalizing image data captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for parallax measurement using camera arrays incorporating 3 x 3 camera configurations.
Venkataraman, Kartik; Huang, Yusong; Jain, Ankit K.; Chatterjee, Priyam, Systems and methods for performing high speed video capture and depth estimation using array cameras.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for performing post capture refocus using images captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for performing post capture refocus using images captured by camera arrays.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using a set of geometrically registered images.
Lelescu, Dan; Duong, Thang, Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information.
Lelescu, Dan; Molina, Gabriel; Venkataraman, Kartik, Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for synthesizing higher resolution images using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Systems and methods for synthesizing higher resolution images using a set of images containing a baseline image.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for synthesizing higher resolution images using images captured by camera arrays.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for synthesizing higher resolution images using images captured by camera arrays.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
Venkataraman, Kartik; Nisenzon, Semyon; Chatterjee, Priyam; Molina, Gabriel, Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.