Systems and methods for normalizing image data captured by camera arrays
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-005/262
H04N-005/232
H04N-005/33
H04N-005/341
H04N-005/349
H04N-005/355
H04N-005/357
H04N-009/04
H04N-005/247
H04N-009/097
G06T-007/00
G06T-019/20
H04N-009/09
H04N-009/73
G02B-005/20
H04N-005/265
G06T-011/60
H04N-013/00
H04N-005/225
H04N-013/02
출원번호
US-0459288
(2014-08-13)
등록번호
US-9049381
(2015-06-02)
발명자
/ 주소
Venkataraman, Kartik
Jabbi, Amandeep S.
Mullis, Robert H.
Duparre, Jacques
Hu, Shane Ching-Feng
출원인 / 주소
Pelican Imaging Corporation
대리인 / 주소
KPPB LLP
인용정보
피인용 횟수 :
5인용 특허 :
130
초록▼
Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance wit
Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.
대표청구항▼
1. A method for normalizing image data captured by camera arrays, comprising: obtaining calibration data for an imager array by capturing images using the imager array, where the calibration data indicates mappings between addresses of physical pixels in imagers and logical addresses within an image
1. A method for normalizing image data captured by camera arrays, comprising: obtaining calibration data for an imager array by capturing images using the imager array, where the calibration data indicates mappings between addresses of physical pixels in imagers and logical addresses within an image;storing the calibration data in a storage device;normalizing a set of images with respect to an image captured by a baseline imager within the imager array based upon the calibration data stored in a storage device using an address conversion module, where the set of images comprises a plurality of images that are: captured from different viewpoints;include different occlusions sets;wherein the occlusion set of a first image is the portion of a scene visible in a second image that is occluded in the first image; andwherein normalizing a set of images with respect to an image captured by a baseline image comprises: correcting color differences between the images with respect to an image captured by a baseline imager; andcorrecting geometric distortion differences between the captured images with respect to an image captured by a baseline imager. 2. The method of claim 1, wherein normalizing the set of images further comprises: black level calculation and adjustments;fixed noise compensation; andoptical point spread function deconvolution. 3. The method of claim 1, further comprising performing temperature normalization using an upstream pipeline processing module based upon a temperature measurement. 4. The method of claim 3, wherein performing temperature normalization comprises performing a refractive index normalization using a stored point spread function selected based upon the temperature measurement. 5. The method of claim 1, wherein correcting color differences between the images further comprises: obtaining a normalization plane;fitting a polynomial to the normalization plane by scanning the normalization plane using a space filling curve;storing fitted polynomials as calibration data; andcorrecting color differences between images with respect to an image captured by a baseline imager using the fitted polynomials. 6. The method of claim 5, wherein obtaining a normalization plane comprises: capturing images of a scene with flat reflectance and calculating a color ratio surface;removing the black level offset from the pixel values in the captured images; andlow pass filtering the pixel values in the captured images to reduce noise; andcalculating a normalization plane. 7. The method of claim 6, wherein: the imager array comprises Green imagers that include Green spectral filters including a baseline Green imager, Red imagers that include Red spectral filters, and Blue spectral filters;a normalization plane is obtained for each of the Red and Blue imagers; anda normalization plane is calculated for at least one of Red imagers by determining: Norm R=G(i,j)/(R(i,j)×(Gcenter/Rcenter))where G is the baseline Green imager, R is a Red imager being normalized with respect to the baseline Green imager, (i, j) describe the pixel position, and Gcenter, and Rcenter, are the pixel values at the center position. 8. The method of claim 5, wherein the space filling curve is a spiral space filling curve. 9. The method of claim 8, wherein a 6th order polynomial represented using seven coefficients is fitted to the normalization plane using the space filling curve. 10. The method of claim 5, further comprising: determining a sensitivity adjustment factor during calibration; andapplying the inverse of the sensitivity adjustment factor when correcting color differences. 11. The method of claim 1, further comprising detecting and metering parallax using a parallax confirmation and measurement module, where detecting and metering parallax comprises ignoring pixels in the images that are in an exposed occlusion set. 12. The method of claim 1, further comprising: aligning portions of images captured by different imagers to compensate for parallax using an image pixel correlation module based upon the detected and metered parallax and the stored calibration data; andobtaining a higher resolution image having a resolution that is higher than the resolutions of the images in the set of images using a super-resolution module, where color information around the edge of the foreground object that is visible to the baseline image and in the occlusion set of the second image is reconstructed in the higher resolution image using the pixels captured by the first image. 13. The method of claim 12, wherein aligning portions of images captured by different imagers to compensate for parallax using an image pixel correlation module further comprises determining appropriate X and Y offsets to be applied to logical pixel address calculations using an address conversion module based upon the detected and metered parallax and the stored calibration data. 14. The method of claim 13, wherein the address conversion module, the parallax confirmation and measurement module, the image pixel correlation module, and the super-resolution module are implemented using a general-purpose computer selectively reconfigured by a computer program stored in the computer. 15. The method of claim 12, wherein obtaining an image having a resolution that is higher than the resolutions of the images in the set of images comprises performing at least on process selected from the group consisting of: fusing the aligned image portions;obtaining an image having a resolution that is higher than the resolutions of the images comprises performing super-resolution processing on the fused image portions to synthesize a super-resolution image. 16. The method of claim 1, wherein each imager in the array of imagers includes a spectral filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral fillers. 17. The method of claim 16, wherein the baseline imager includes a Green filter. 18. The method of claim 1, wherein the imagers in the imager array have different resolutions. 19. A method for normalizing image data captured by camera arrays, comprising: obtaining calibration data for imagers in an imager array by capturing images using the imager array, where the calibration data indicates mappings between addresses of physical pixels in imagers and logical addresses within an image;storing the calibration data in a storage device;normalizing a set of images with respect to an image captured by a baseline imager within the imager array based upon the calibration data stored in a storage device using an address conversion module, where the set of images comprises a plurality of images that are: captured from different viewpoints;include different occlusions sets;wherein the occlusion set of a first image is the portion of a scene visible in a second image that is occluded in the first image; andwherein obtaining calibration data comprises: obtaining a normalization plane by: capturing images of scene with flat reflectance and calculating a color ratio surface;removing the black level offset from the pixel values in the captured images;low pass filtering the pixel values in the captured images to reduce noise; andcalculating a normalization plane;fitting a polynomial to the normalization plane by scanning the normalization plane using a space filling curve; andstoring fitted polynomials as calibration data in the storage device; andwherein normalizing a set of images with respect to an image captured by a baseline image comprises: correcting color differences between images with respect to an image captured by a baseline imager using the fitted polynomials: andcorrecting geometric distortion differences between the captured images with respect to an image captured by a baseline imager.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (130)
Wilburn, Bennett; Joshi, Neel; Levoy, Marc C.; Horowitz, Mark, Apparatus and method for capturing a scene using staggered triggering of dense camera arrays.
Iwase Toshihiro (Nara JPX) Kanekura Hiroshi (Yamatokouriyama JPX), Apparatus for and method of converting a sampling frequency according to a data driven type processing.
Boisvert, David Michael; McMahon, Andrew Kenneth John, CCD output processing stage that amplifies signals from colored pixels based on the conversion efficiency of the colored pixels.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Capturing and processing of images using monolithic camera array with heterogeneous imagers.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H.; Duparre, Jacques; Hu, Shane Ching-Feng, Capturing and processing of images using monolithic camera array with heterogeneous imagers.
Yamashita,Syugo; Murata,Haruhiko; Iinuma,Toshiya; Nakashima,Mitsuo; Mori,Takayuki, Device and method for converting two-dimensional video to three-dimensional video.
Ward, Gregory John; Seetzen, Helge; Heidrich, Wolfgang, Electronic camera having multiple sensors for capturing high dynamic range images and related methods.
Abell Gurdon R. (West Woodstock CT) Cook Francis J. (Topsfield MA) Howes Peter D. (Sudbury MA), Method and apparatus for arraying image sensor modules.
Sawhney,Harpreet Singh; Tao,Hai; Kumar,Rakesh; Hanna,Keith, Method and apparatus for synthesizing new video and/or still imagery from a collection of real video and/or still imagery.
Alexander David H. (Santa Monica CA) Hershman George H. (Carlsbad CA) Jack Michael D. (Carlsbad CA) Koda N. John (Vista CA) Lloyd Randahl B. (San Marcos CA), Monolithic imager for near-IR.
Hornbaker ; III Cecil V. (New Carrolton MD) Driggers Thomas C. (Falls Church VA) Bindon Edward W. (Fairfax VA), Scanning apparatus using multiple CCD arrays and related method.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for parallax detection and correction in images captured using array cameras that contain occlusions using subsets of images to perform depth estimation.
Venkataraman, Kartik; Jabbi, Amandeep S.; Mullis, Robert H., Systems and methods for parallax measurement using camera arrays incorporating 3 x 3 camera configurations.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for performing depth estimation using image data from multiple spectral channels.
Ludwig, Lester F., Vignetted optoelectronic array for use in synthetic image formation via signal processing, lensless cameras, and integrated camera-displays.
Rieger Albert,DEX ; Barclay David ; Chapman Steven ; Kellner Heinz-Andreas,DEX ; Reibl Michael,DEX ; Rydelek James G. ; Schweizer Andreas,DEX, Watertight body for accommodating a photographic camera.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.