Methods and apparatus for super-resolution in integral photography
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-003/14
H04N-005/335
H04N-005/225
G02B-027/10
출원번호
US-0957316
(2010-11-30)
등록번호
US-8724000
(2014-05-13)
발명자
/ 주소
Georgiev, Todor G.
Chunev, Georgi N.
Lumsdaine, Andrew
출원인 / 주소
Adobe Systems Incorporated
대리인 / 주소
Wolfe-SBMC
인용정보
피인용 횟수 :
11인용 특허 :
87
초록▼
Methods and apparatus for super-resolution in integral photography are described. Several techniques are described that, alone or in combination, may improve the super-resolution process and/or the quality of super-resolved images that may be generated from flats captured with a focused plenoptic ca
Methods and apparatus for super-resolution in integral photography are described. Several techniques are described that, alone or in combination, may improve the super-resolution process and/or the quality of super-resolved images that may be generated from flats captured with a focused plenoptic camera using a super-resolution algorithm. At least some of these techniques involve modifications to the focused plenoptic camera design. In addition, at least some of these techniques involve modifications to the super-resolution rendering algorithm. The techniques may include techniques for reducing the size of pixels, techniques for shifting pixels relative to each other so that super-resolution is achievable at more or all depths of focus, and techniques for sampling using an appropriate filter or kernel. These techniques may, for example, reduce or eliminate the need to perform deconvolution on a super-resolved image, and may improve super-resolution results and/or increase performance.
대표청구항▼
1. A camera, comprising: an objective lens configured to refract light from a scene located in front of the camera to form an image of the scene at an image plane of the objective lens;a microlens array, positioned between the objective lens and a photosensor that comprises a plurality of pixels con
1. A camera, comprising: an objective lens configured to refract light from a scene located in front of the camera to form an image of the scene at an image plane of the objective lens;a microlens array, positioned between the objective lens and a photosensor that comprises a plurality of pixels configured to capture light projected on to the photosensor, the microlens array comprises a plurality of microlenses that are each configured to sample a respective region of the image of the scene formed at the image plane by the objective lens, each region of the image of the scene sampled by the microlenses partially overlaps at least one other region of the image of the scene sampled by the microlenses;a plurality of optical elements fabricated as apertures in a mask layer interposed between the pixels of the microlens array and the photosensor to reduce areas of the image of the scene sampled by the pixels so that the area of the image of the scene sampled by pixels at respective locations on the photosensor do not overlap the areas of the image of the scene sampled by pixels at other locations on the photosensor onto which an overlapping region of the image of the scene is projected, each said location on the photosensor samples radiance from a particular area of the image of the scene; andthe photosensor configured to capture a flat that includes each of the regions of the image of the scene projected onto the photosensor by the microlenses in a separate microimage in the flat that is processed by a super-resolution technique that interleaves pixels in neighboring microimages according to a subpixel shift to render a high-resolution image of the scene, the super-resolution technique comprising, for each point in the high-resolution image, sampling values at pixels in two or more different microimages in the flat and blending the sampled values to generate a value for a pixel at the respective point in the high-resolution image. 2. The camera as recited in claim 1, wherein the reduction provided by the optical elements of the photosensor reduces blur in the high-resolution image of the scene generated by the super-resolution technique. 3. The camera as recited in claim 1, wherein the optical elements are configured to form corresponding said pixels with gaps between adjacent ones of the pixels. 4. The camera as recited in claim 1, wherein the mask layer includes an aperture at each pixel, and the apertures restrict light at respective pixels of the photosensor to effectively form small pixels with gaps between adjacent ones of the pixels. 5. The camera as recited in claim 1, wherein the optical elements are lenslets between the pixels of the photosensor and the microlens array, a lenslet being located at each pixel, and the lenslets affecting light projected onto respective pixels of the photosensor to effectively form small pixels with gaps between adjacent ones of the pixels. 6. The camera as recited in claim 1, wherein said sampling the values comprises applying a sharpening kernel to sample the values at the pixels in the microimages, the sharpening kernel applied to sample the values at the pixels in the microimages reducing blur in the high-resolution image. 7. The camera as recited in claim 1, wherein varying pitch between the centers of the areas of the image of the scene sampled by the pixels of the photosensor enables super-resolution to be performed at more depths of focus than possible with a flat captured with a fixed pitch between centers of the areas of the image of the scene sampled by the pixels. 8. A camera, comprising: an objective lens configured to refract light from a scene located in front of the camera to form an image of the scene at an image plane of the objective lens;a microlens array, positioned between the objective lens and a photosensor that comprises a plurality of pixels configured to capture light projected onto the photosensor, the microlens array comprising a plurality of microlenses that are each configured to sample a respective region of the image of the scene formed at the image plane by the objective lens, each region of the image of the scene sampled by the microlenses partially overlaps at least one other region of the image of the scene sampled by the microlenses;a plurality of optical elements interposed between the microlens array and the pixels of the photosensor, a pitch for the placement of the optical elements randomly varying the distance between centers of areas of the image of the scene sampled by the pixels of the photosensor, each said pixel on the photosensor sampling radiance from a particular area of the image of the scene; andthe photosensor configured to capture a flat that includes each of the regions of the image of the scene projected onto the photosensor by the microlenses in a separate microimage in the flat that is processed by a super-resolution technique that interleaves pixels in neighboring microimages according to a subpixel shift to render a high-resolution image of the scene, the super-resolution technique comprising, for each point in the high-resolution image, sampling values at pixels in two or more different microimages in the flat and blending the sampled values to generate a value for a pixel at the respective point in the high-resolution image. 9. The camera as recited in claim 8, wherein said varying pitch between the centers of the areas of the image of the scene sampled by the pixels of the photosensor enables super-resolution to be performed at more depths of focus than possible with a flat captured with a fixed pitch between centers of the areas of an image of a scene sampled by the pixels. 10. The camera as recited in claim 8, wherein a reduction provided by the optical elements of the photosensor reduces blur in the high-resolution image of the scene generated by the super-resolution technique. 11. The camera as recited in claim 8, wherein the optical elements are configured to form corresponding said pixels with gaps between adjacent ones of the pixels. 12. The camera as recited in claim 8, wherein said sampling the values comprises a sharpening kernel applied to sample the values at the pixels in the microimages reducing blur in the high-resolution image. 13. The camera as recited in claim 8, wherein the optical elements lenslets affecting the light projected onto respective pixels of the photosensor to effectively form small pixels with gaps between adjacent ones of the pixels. 14. A method for capturing light-field images, comprising: refracting light at an objective lens of a camera to form an image of a scene at an image plane of the objective lens;receiving light from the image plane at a microlens array positioned between the objective lens and a photosensor of the camera, the photosensor comprising a plurality of pixels configured to capture light projected onto the photosensor, the microlens array comprising a plurality of microlenses that each sample a respective region of the image of the scene formed at the image plane by the objective lens, and each region of the image of the scene sampled by the microlenses partially overlaps at least one other region of the image of the scene sampled by the microlenses;reducing, by optical elements fabricated as apertures in a mask layer interposed between the microlens array and the pixels of the photosensor, areas of the image of the scene received at the pixels so that the area of the image of the scene sampled by a given pixel at a given location on the photosensor does not overlap the areas of the image of the scene sampled by pixels at other locations on the photosensor onto which an overlapping region of the image of the scene is projected;receiving light from the microlens array at the photosensor, the regions of the image of the scene sampled by the microlenses are received at separate locations on the photosensor, each pixel associated with each said location on the photosensor receives radiance from a particular area of the image of the scene;capturing a flat that includes each of the regions of the image of the scene projected onto the photosensor by the microlenses in a separate microimage in the flat; andinterleaving pixels in neighboring microimages according to a subpixel shift to render a high-resolution image of the scene. 15. The method as recited in claim 14, further comprising: applying a super-resolution technique to the flat to render the high-resolution image of the scene, the reduction provided by the optical elements of the photosensor reducing blur in the high-resolution image of the scene generated by the super-resolution technique. 16. The method as recited in claim 15, wherein said applying the super-resolution technique comprises, for each point in the high-resolution image, sampling values at pixels in two or more different microimages in the flat and blending the sampled values to generate a value for a pixel at the respective point in the high-resolution image. 17. The method of claim 16, wherein said sampling the values comprises applying a sharpening kernel to sample the values at the pixels in the microimages, and said applying the sharpening kernel to sample the values at the pixels in the microimages reducing blur in the high-resolution image. 18. The method as recited in claim 14, wherein the mask layer includes an aperture at each pixel, and the apertures restrict light at respective pixels of the photosensor to effectively form small pixels with gaps between adjacent ones of the pixels. 19. The method as recited in claim 14, wherein the optical elements are lenslets between the photosensor and the microlens array, a lenslet being located at each pixel, and the lenslets affecting light projected onto respective pixels of the photosensor to effectively form small pixels with gaps between adjacent ones of the pixels. 20. The method as recited in claim 14, wherein varying pitch between the centers of the areas of the image sampled by the pixels of the photosensor enables super-resolution to be performed at more depths of focus than possible with a flat captured with a fixed pitch between centers of the areas of the image of the scene sampled by the pixels.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (87)
Georgiev Todor, 3D graphics based on images and morphing.
Loce Robert P. (Rochester NY) Cianciosi Michael S. (Rochester NY) Kingsley Jeffrey D. (Williamson NY), Image resolution conversion method that employs statistically generated multiple morphological filters.
Yamagata, Michihiro; Okayama, Hiroaki; Boku, Kazutake; Tanaka, Yasuhiro; Hayashi, Kenichi; Fushimi, Yoshimasa; Murata, Shigeki; Hayashi, Takayuki, Imaging device including a plurality of lens elements and a imaging sensor.
de Montebello Roger L. (New York NY) Globus Ronald P. (New York NY) Buck Howard S. (New York NY), Integral photography apparatus and method of forming same.
Mindler, Robert F.; Calkins, Guy T., Method and apparatus for thermal printing of longer length images by the use of multiple dye color patch triads or quads.
Vetro, Anthony; Yea, Sehoon; Matusik, Wojciech; Pfister, Hanspeter; Zwicker, Matthias, Method and system for acquiring, encoding, decoding and displaying 3D light fields.
Georgiev, Todor G.; Chunev, Georgi N., Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data.
Corle Timothy R. (Santa Clara County CA) Kino Gordon S. (Santa Clara County CA) Mansfield Scott M. (San Mateo County CA), Optical recording system employing a solid immersion lens.
Patton, David L.; Spoonhower, John P.; Bohan, Anne E.; Paz-Pujalt, Gustavo R., Solid immersion lens array and methods for producing a solid immersion lens array.
Georgiev, Todor G.; Lumsdaine, Andrew, Methods, apparatus, and computer-readable storage media for depth-based rendering of focused plenoptic camera data.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.