Image stabilization related methods and apparatus
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-005/228
H04N-005/232
G02B-027/64
H04N-005/225
G02B-027/58
출원번호
US-0530687
(2014-10-31)
등록번호
US-9426365
(2016-08-23)
발명자
/ 주소
Laroia, Rajiv
Shroff, Sapna A
출원인 / 주소
THE LIGHTCO INC.
대리인 / 주소
Straub, Michael P.
인용정보
피인용 횟수 :
15인용 특허 :
30
초록▼
Methods and apparatus for processing images captured by a camera device including multiple optical chains, e.g., camera modules, are described. Three, 4, 5 or more optical chains maybe used. Different optical chains capture different images due to different perspectives. Multiple images, e.g., corre
Methods and apparatus for processing images captured by a camera device including multiple optical chains, e.g., camera modules, are described. Three, 4, 5 or more optical chains maybe used. Different optical chains capture different images due to different perspectives. Multiple images, e.g., corresponding to different perspectives, are captured during a time period and are combined to generate a composite image. In some embodiments one of the captured images or a synthesized image is used as a reference image during composite image generation. The image used as the reference image is selected to keep the perspective of sequentially generated composite images consistent despite unintentional came movement and/or in accordance with an expected path of travel. Thus, which camera module provides the reference image may vary over time taking into consideration unintended camera movement. Composite image generation may be performed external to the camera device or in the camera device.
대표청구항▼
1. A method of generating images, the method comprising: detecting an amount of motion, said detected amount of motion being a detected amount of motion of a camera device including multiple optical chains or a detected amount of motion between an image corresponding to a second time period and an i
1. A method of generating images, the method comprising: detecting an amount of motion, said detected amount of motion being a detected amount of motion of a camera device including multiple optical chains or a detected amount of motion between an image corresponding to a second time period and an image corresponding to a first time period;producing a second reference image, from a first plurality of images captured by different optical chains of said camera device during the second time period, producing a second reference image including at least one of: i) selecting the second reference image from images captured by different optical chains during said second time period based on the detected amount of motion or ii) synthesizing a reference image from at least two of said first plurality of images captured by different optical chains during said second time period based on the detected amount of motion; andusing the second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period. 2. The method of claim 1, wherein said composite image is an image having the same perspective as a first reference image. 3. The method of claim 1, wherein said composite image corresponding to said second time period is a frame of a video sequence including composite images corresponding to multiple sequential frame times;wherein the method further comprises operating said camera device including said multiple optical chains to capture said first plurality of images, said camera device being a handheld camera device; andwherein said detecting, producing and using steps are performed by i) a processor included in said handheld camera device or a processor which receives said first plurality of images from said handheld camera device. 4. The method of claim 1, wherein selecting the second reference image is based on detected motion of said camera device including said plurality of optical chains having different perspectives, different ones of said first plurality of images being captured by different ones of said plurality of optical chains. 5. The method of claim 4, wherein said selecting includes selecting as said second reference image an image captured by an optical chain of said camera device having a second perspective which is the closest to a first perspective of an optical chain of said camera that was used to capture a first reference image. 6. The method of claim 1, wherein selecting a second reference image is based on detected motion in images captured by optical chains s of said camera device, said motion being detected by comparing the content of an image corresponding to said first time period to an image corresponding to said second time period. 7. The method of claim 6, wherein comparing the content of an image corresponding to said first time period to an image corresponding to said second time period includes comparing a first image captured by a first optical chain of said camera device during a first time period to a second image captured by said first optical chain during said second time period. 8. The method of claim 7, wherein said selecting includes selecting as said second reference image an image captured by an optical chain of said camera device having a second perspective which is the closest to a first perspective of an optical chain of said camera that was used to capture a first reference image. 9. The method of claim 1, wherein using said second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period includes: generating for said at least one other image, first image shift information indicating an amount of a first image shift between the second reference image and said at least one other image. 10. The method of claim 9, wherein using said second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period further includes: generating for a third image captured by an optical chain of said camera device during said second time period, third image shift information indicating an amount of a third image shift between the second reference image and said third image. 11. The method of claim 10, wherein using said second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period further includes: performing at least one of a shift, a warp or another image distortion operation to said at least one other image as a function of said first image shift information to generate first modified image data. 12. The method of claim 11, wherein using said second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period further includes: performing at least one of a shift, a warp or another image distortion operation to said third image as a function of said third image shift information to generate modified third image data; andcombining image data corresponding to at least two of said second reference image, said first modified image data and said modified third image data to generate said composite image. 13. A camera device comprising: a plurality of optical chains;a module configured to detect an amount of motion;a module configured to produce a second reference image, from a first plurality of images captured by different optical chains of said camera device during a second time period, producing the second reference image including at least one of: i) selecting the second reference image from images captured by different optical chains during said second time period based on the detected amount of motion or ii) synthesizing a reference image from at least two of said first plurality of images captured by different optical chains during said second time period based on the detected amount of motion; anda module configured to use the second reference image and at least one other image in said first plurality of images to generate a composite image corresponding to said second time period. 14. The camera device of claim 13, wherein said module configured to detect an amount of motion includes a module configured to detect an amount of motion of the camera device including multiple optical chains, e.g., based on gyroscope and/or accelerometer measurements. 15. The camera device of claim 13, wherein said module configured to detect an amount of motion includes a module configured to detect an amount of motion between an image corresponding to a second time period and an image corresponding to the first time period. 16. The camera device of claim 13, wherein said composite image is an image having the same perspective as a first reference image. 17. The camera device of claim 13, further comprising: a module configured to compare the content of an image corresponding to a first time period to an image corresponding to the second time period; andwherein said module configured to produce a second reference image includes a module configured to select the second reference image based on detected motion in images captured by optical chains of said camera device, said motion being detected by comparing the content of an image corresponding to a first time period to an image corresponding to said second time period. 18. The camera device of claim 13, wherein said module configured to produce a second reference image includes a module configured to select the second reference image based on detected motion of said camera device including said plurality of optical chain modules having different perspectives, different ones of said first plurality of images being captured by different ones of said plurality of optical chains. 19. The camera device of claim 18, wherein said module configured to produce a second reference image includes a module configured to select as said second reference image an image captured by an optical chain of said camera device having a second perspective which is the closest to a first perspective of an optical chain of said camera device that was used to capture a first reference image. 20. A non-transitory computer readable media comprising stored instructions which when executed by a processor of an image processing system control the image processing system to: detect an amount of motion, said detected amount of motion being a detected amount of motion of a camera device including multiple optical chains or a detected amount of motion between an image corresponding to a second time period and an image corresponding to a first time period;produce a second reference image, from a first plurality of images captured by different optical chains of said camera device during the second time period, producing a second reference image including at least one of: i) selecting the second reference image from images captured by different optical chains during said second time period based on the detected amount of motion or ii) synthesizing a reference image from at least two of said first plurality of images captured by different optical chains during said second time period based on the detected amount of motion; andgenerate a composite image corresponding to said second time period from the second reference image and at least one other image in said first plurality of images.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (30)
Katayama Tatsushi,JPX ; Takiguchi Hideo,JPX ; Yano Kotaro,JPX ; Hatori Kenji,JPX, Apparatus and method for combining a plurality of images.
Watanabe, Makoto; Magaki, Yosuke; Nakazawa, Sachiko; Onumata, Yuichi, Camera, storage medium having stored therein camera control program, and camera control method.
Georgiev, Todor G.; Chunev, Georgi N., Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data.
Ciurea, Florian; Venkataraman, Kartik; Molina, Gabriel; Lelescu, Dan, Systems and methods for parallax detection and correction in images captured using array cameras that contain occlusions using subsets of images to perform depth estimation.
Pulli, Kari; Shroff, Nitesh; Shroff, Sapna A., Methods and apparatus for compensating for motion and/or changing light conditions during image capture.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.