IPC분류정보
국가/구분 |
United States(US) Patent
등록
|
국제특허분류(IPC7판) |
|
출원번호 |
US-0677139
(2003-09-30)
|
등록번호 |
US-7340109
(2008-03-04)
|
발명자
/ 주소 |
- Steinberg,Eran
- Prilutsky,Yury
- Corcoran,Peter
- Bigioi,Petronel
- Zamfir,Adrian
- Buzuloiu,Vasile
- Ursu,Danutz
- Zamfir,Marta
|
출원인 / 주소 |
- FotoNation Vision Limited
|
대리인 / 주소 |
|
인용정보 |
피인용 횟수 :
22 인용 특허 :
42 |
초록
▼
A method of automatically correcting dust artifact within images acquired by a system including a digital camera includes determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image. The dust artifact regions are associated with one or more extracted p
A method of automatically correcting dust artifact within images acquired by a system including a digital camera includes determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image. The dust artifact regions are associated with one or more extracted parameters relating to a variable optical system of the digital camera when the first image was acquired. An associated statistical dust map is formed including mapped dust regions based on known correspondence between effects on dust artifact data of values of extracted parameters. Pixels corresponding to correlated dust artifact regions within a second image acquired with said digital camera are corrected according to one or more different values of the extracted parameters relating to the variable optical system based on the associated statistical dust map.
대표청구항
▼
What is claimed is: 1. A method of automatically correcting dust artifact within images acquired by a system including a digital camera, comprising: (a) determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image; (b) associating the dust artifact reg
What is claimed is: 1. A method of automatically correcting dust artifact within images acquired by a system including a digital camera, comprising: (a) determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image; (b) associating the dust artifact regions with one or more extracted parameters relating to a variable optical system of said digital camera when the first image was acquired; (c) forming an associated statistical dust map including mapped dust regions based on the dust artifact determining and associating operations and known correspondence between effects on dust artifact data of one or more different values of said one or more extracted parameters; and (d) correcting pixels corresponding to correlated dust artifact regions within a second image acquired with said digital camera according to one or more different values of said one or more extracted parameters relating to said variable optical system based on said associated statistical dust map; and (e) wherein said image correction method is automatically performed within a digital camera that comprises said optical system, said sensor array, said processing electronics and said memory. 2. The method of claim 1, wherein said first and second images were acquired of different objects. 3. The method of claim 1, said one or more extracted parameters comprising aperture size, F-number, magnification, lens type or focal length of an optical system of the digital camera, or combinations thereof. 4. The method of claim 3, said one or more extracted parameters are calculated empirically from comparison of one or more said dust artifact regions within said multiple original digital images with said digital acquisition device. 5. The method of claim 1, said one or more extracted parameters comprising aperture size or focal length or both. 6. The method of claim 1, further comprising correcting pixels corresponding to correlated dust artifact regions within said first image based on the associated statistical dust map. 7. The method of claim 6, said one or more extracted parameters comprising aperture size, F-number, magnification, lens type or focal length of an optical system of the digital camera, or combinations thereof. 8. The method of claim 6, said one or more extracted parameters comprising aperture size or focal length or both. 9. The method of claim 6, further comprising repeating for further digitally-acquired images said determining and associating, and updating said statistical dust map including updating said mapped dust regions based on the additional dust artifact determining and associating. 10. The method of claim 9, further comprising limiting updating one or more of said further and original images based on said updating of said associated statistical dust map to updates that do not include appearance of new dust or movement of existing dust. 11. The method of claim 9, further comprising limiting updating one or more of said further and original images based on said updating of said associated statistical dust map to updates that include previously determined dust artifact regions. 12. The method of claim 6, further comprising creating a version description of changes in said statistical dust map. 13. The method of claim 12, wherein said version is based on a chronological time stamp. 14. The method of claim 12, wherein said version is based on replacement of lens. 15. The method of claim 12, wherein said version information comprises change of said probabilities in said statistical dust map that certain pixels correspond to dust artifact regions. 16. The method of claim 12, wherein said version information includes one or more parameters comprising change in dust location, change in dust position, appearance of new dust region, or disappearance of existing dust region, or combinations thereof. 17. The method of claim 6, further comprising determining whether the dust map is to be replaced based on determining that sufficient disparity exists based on the amount and quality of said changes in said statistical dust map. 18. The method of claim 17, further comprising updating one or more of said further and original images based on said updating of said associated statistical dust map based on said version description. 19. The method of claim 6, further comprising digitally-acquiring one or more additional images with said digital camera, repeating said determining and associating, and updating said statistical dust map including updating said mapped dust regions based on the additional dust artifact determining and associating. 20. The method of claim 19, further comprising correcting pixels corresponding to correlated dust artifact regions within said additional images based on the updated, associated statistical dust map. 21. The method of claim 19, further comprising updating said second image based on said updating of said associated statistical dust map. 22. The method of claim 21, further comprising updating one or more of said first image and said additional images based on said updating of said associated statistical dust map. 23. The method of claim 1, said image correction method being performed at least in part within an external processing device that couples with a digital camera that comprises said optical system and said sensor array to form a digital image acquisition and processing system that also comprises said processing electronics and said memory. 24. The method of claim 23, the programming instructions being stored on a memory within the external device which performs the image correction method. 25. The method of claim 1, said determining comprising determining probabilities that certain pixels correspond to dust artifact regions within said first acquired image. 26. The method of claim 25, further comprising eliminating certain suspected dust artifact regions as having a probability below a first threshold value. 27. The method of claim 26, further comprising judging certain further dust artifact regions as having a probability above said threshold value, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said eliminating operation. 28. The method of claim 26, further comprising judging certain probable dust artifact regions as having a probability above a second threshold value such as to be subject to said correcting operation. 29. The method of claim 28, wherein said first and second threshold values are different. 30. The method of claim 29, further comprising judging certain further dust artifact regions as having a probability between said first and said second threshold values, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said correcting operation. 31. The method of claim 25, further comprising judging certain probable dust artifact regions as having a probability above a threshold value such as to be subject to said correcting operation. 32. The method of claim 31, further comprising judging certain further dust artifact regions as having a probability below said threshold value, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said correcting operation. 33. The method of claim 25, wherein said probability determining includes weighting suspected dust artifact regions according to one or more predetermined probability weighting assessment conditions. 34. The method of claim 33, said one or more weighting assessment conditions comprising size, shape, brightness or opacity of said suspected dust artifact regions, or degree of similarity in size, shape, brightness, opacity or location with one or more suspected dust artifact regions in one or more other images, or combinations thereof. 35. The method of claim 1, wherein said determining is based at least in part on a comparison of suspected dust artifact regions within multiple images including said first image and at least one other digitally-acquired image. 36. The method of claim 35, said at least one other digitally-acquired image comprising said second image. 37. A method of automatically correcting dust artifact regions within images acquired by a system including a digital image acquisition device including a digital camera with a lens assembly, comprising: (a) determining probabilities that certain pixels within multiple digitally-acquired images correspond to one or more dust artifact regions; (b) associating probable dust artifact regions with values of one or more extracted parameters relating to a lens system of the digital image acquisition device when the images were acquired; (c) forming a statistical dust map including mapped dust regions based on the dust artifact determining and associating operations; and (d) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the statistical dust map; (e) wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region; and (f) wherein said determining with respect to a shadow region is based on an extracted parameter-dependent shadow region analysis, wherein the shadow region analysis presumes that certain regions on a sensor of the digital image acquisition device are fully obscured by said dust. 38. The method of claim 37, further comprising repeating the dust artifact determining and associating operations for further acquired images, values of the one or more extracted parameters relating to the optical system differing between at least two acquired images. 39. The method of claim 38, said at least two acquired images having been acquired of different objects. 40. The method of claim 38, said forming said statistical dust map being based upon multiple acquired images and including mathematically correlating dust artifact regions of the multiple acquired images according to their extracted parameter values including said one or more different extracted parameter values of said at least two acquired images. 41. The method of claim 37, wherein said different images are acquired of a different scene. 42. The method of claim 37, the dust artifact determining including calculating effects on image data of actual dust based on values of one or more of said extracted parameters relating to said lens system. 43. The method of claim 37, the method further comprising filtering from the statistical dust map images including dust artifact regions having computed dust artifact probabilities below a threshold level. 44. The method of claim 37, wherein at least two determined probabilities that certain pixels within said multiple digitally-acquired images correspond to one or more dust artifact regions have different values between 0 and 1. 45. The method of claim 44, whether said correcting is performed for pixels of said digitally-acquired images depending on values of determined probabilities that said certain pixels correspond to dust artifact regions. 46. The method of claim 37, the method being automatically performed within a digital camera that--comprises said lens system, a sensor array, processing electronics and a memory. 47. The method of claim 37, the method being performed at least in part within an external processing device that couples with a digital camera that comprises said lens system and a sensor array to form a digital image acquisition and processing system that also comprises processing electronics and a memory. 48. The method of claim 37, the method being performed on raw image data as captured by a camera sensor. 49. The method of claim 37, said image correction method being performed on a processed image after being converted from raw format to a known red, green, blue representation. 50. The method of claim 37, wherein said correcting includes replacing said pixels within said one or more digitally-acquired images with new pixels. 51. The method of claim 37, wherein correcting instructions are kept in an external location to the image data. 52. The method of claim 51, said external location comprising an image header. 53. The method of claim 37, the digital data corresponding to pixels within one or more digitally-acquired images being correlated with mapped dust regions of the statistical dust map by configuring the dust map with variable inputs corresponding to values of said one or more extracted parameters, inputting particular values of the one or more extracted parameters corresponding to the one or more digitally-acquired images, and relating said mapped dust regions with said digital data corresponding to said pixels. 54. The method of claim 37, the digital data corresponding to pixels within one or more digitally-acquired images being correlated with mapped dust regions of the statistical dust map by configuring the dust map according to particular values of said one or more extracted parameters, and mathematically relating the dust map and digitally-acquired images according to same values of said one or more extracted parameters. 55. The method of claim 37, the dust artifact determining operation including: (I) loading the statistical dust map; (II) loading extracted parameter information of a present image; (III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and (IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data. 56. The method of claim 55, the extracted parameter information including values of aperture size and focal length. 57. The method of claim 56, the extracted parameter information further including lens type information. 58. The method of claim 37, the dust artifact determining operation including: (I) loading the statistical dust map; (II) loading extracted parameter information of a present image; (III) performing a calculation for relating the statistical dust map with the present image according to a selected value of an extracted parameter which is otherwise uncorrelated between the present image and the dust map; and (IV) comparing dust artifact detection data with the now correlated statistical dust map data. 59. The method of claim 58, the extracted parameter information including values of aperture size and focal length. 60. The method of claim 59, the extracted parameter information further including lens type information. 61. The method of claim 37, the extracted parameters comprising focal length, magnification or type of the lens assembly, or aperture size or position, or combinations thereof. 62. The method of claim 61, the extracted parameters further comprising distance of the actual dust object from one or more positions in plane of the electronic sensor array. 63. The method of claim 37, the method further comprising determining that sufficient disparity exists between dust artifact determinations within sequential images to determine that the camera may have been cleaned and that the dust artifact determinations prior to the cleaning will not be used in the dust map. 64. The method of claim 37, the correcting comprising in-painting or restoration, or both. 65. The method of claim 64, the in-painting operation being performed on pixels determined to have substantially no image-relevant spectral information, or insufficient image-relevant spectral information, associated with their corresponding data. 66. The method of claim 65, the determining of insufficiency or no image-relevant spectral information being dependent on a signal to noise calculation. 67. The method of claim 65, the in-painting including creation of new pixel values based on characteristics of surrounding pixels to the dust region. 68. The method of claim 67, one or more characteristics of the surrounding pixels upon which said in-painting is based include the color, brightness, gradient, edge detection, noise, pattern, texture, geometry, or combinations thereof. 69. The method of claim 64, the restoration being performed on pixels determined to have sufficient image-relevant spectral information associated with their corresponding data. 70. The method of claim 69, the restoration including modifying brightness based on an inverse relationship to the opacity of the dust artifact or a brightness characteristic of the pixels within the image, or both. 71. The method of claim 70, the restoration further including enhancing color based on a present color characteristic of pixels within the image. 72. The method of claim 37, the correcting operation comprising in-painting or restoration, or both. 73. The method of claim 72, said determining with respect to a shadow region being based on an extracted parameter-dependent shadow region analysis, wherein the shadow region analysis presumes that certain regions on a sensor of the digital image acquisition device are fully obscured by said dust. 74. The method of claim 73, wherein the shadow region analysis includes calculating effects of differences in values of the one or more extracted parameters in different images on dust artifact illumination, shape, position, reflection or transmission properties, distance of dust to the sensor, aperture, exit pupil, or focal length, or combinations thereof. 75. The method of claim 74, said different images having been acquired of different objects. 76. The method of claim 74, wherein said different images are acquired of different scene. 77. The method of claim 72, said correcting including in-painting the shadow region. 78. The method of claim 77, said in-painting including determining and applying shadow region correction spectral information based on spectral information obtained from pixels outside said shadow region. 79. The method of claim 72, said correcting including restoration of the aura region. 80. The method of claim 79, said restoration including determining and applying aura region correction spectral information based on spectral information obtained from pixels within said aura region. 81. A method of automatically correcting dust artifact regions within images acquired by a system including a digital image acquisition device including a digital camera with a lens assembly, comprising: (a) determining probabilities that certain pixels within multiple digitally-acquired images correspond to one or more dust artifact regions; (b) associating probable dust artifact regions with values of one or more extracted parameters relating to a lens system of the digital image acquisition device when the images were acquired; (c) forming a statistical dust map including mapped dust regions based on the dust artifact determining and associating operations; and (d) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the statistical dust map; (e) wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region; and (f) wherein said determining with respect to an aura region being based on an extracted parameter-dependent aura region analysis, wherein the aura region analysis presumes that certain regions on a of the digital image acquisition device are partially obscured by said dust. 82. The method of claim 81, wherein said aura region analysis includes calculating effects of differences in values of the one or more extracted parameters in different images on dust artifact illumination, shape, position, reflection or transmission properties, distance of dust to the sensor, aperture, exit pupil, or focal length, or combinations thereof. 83. The method of claim 82, said different images having been acquired of different objects. 84. The method of claim 82, wherein said different images are acquired of different scene. 85. The method of claim 81, the correcting operation comprising in-painting or restoration, or both. 86. The method of claim 85, said correcting including in-painting the shadow region. 87. The method of claim 86, said in-painting including determining and applying shadow region correction spectral information based on spectral information obtained from pixels outside said shadow region. 88. The method of claim 85, said correcting including restoration of the aura region. 89. The method of claim 88, said restoration including determining and applying aura region correction spectral information based on spectral information obtained from pixels within said aura region. 90. A method of automatically correcting dust artifact regions within images acquired by a system including a digital image acquisition device comprising a digital camera with a lens assembly, comprising: (a) determining that certain pixels within one or more digitally-acquired images correspond to one or more dust artifact regions; (b) associating the dust artifact regions with values of one or more extracted parameters relating to a lens system of the digital image acquisition device when the images were acquired; (c) forming a dust map including mapped dust regions based on the dust artifact determining and associating operations; (d) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the dust map; (e) repeating the dust artifact determining and associating operations for further acquired images after forming said statistical dust map; (f) modifying said statistical dust map based on said further images including mathematically correlating determined and associated dust artifact regions of said further acquired images with dust artifact regions in said dust map including combining data associated with dust artifact regions within images acquired with different values of one or more extracted parameters; and (g) correcting digital data corresponding to the correlated dust artifact regions within acquired images based on the modified statistical dust map. 91. The method of claim 90, the modifying including varying probabilities based on combining new probability data with probability data within said original dust map. 92. The method of claim 90, the modifying including eliminating a dust artifact region of said original dust map due to its probability falling below a threshold value between 0 and 1 after combining new probability data with probability data within said original dust map. 93. The method of claim 90, the modifying including inserting a new dust artifact region not present within said original dust map due to its having a probability, based on analysis of multiple images, that is now above a threshold value between 0 and 1 after combining new probability data with probability data within said original dust map. 94. The method of claim 90, further comprising reapplying said correcting digital data for images previously acquired after said modifying statistical dust map showed a better said statistical probability to said one or more said detected dust regions. 95. The method of claim 90, said dust artifact regions including an aura region partially obscured by dust and a shadow region substantially obscured by dust, the shadow region being inside said aura region. 96. The method of claim 95, said determining with respect to a shadow region being based on an extracted parameter-dependent shadow region analysis, wherein the shadow region analysis presumes that certain regions on a sensor of the digital image acquisition device are fully obscured by said dust. 97. The method of claim 96, wherein the shadow region analysis includes calculating effects of differences in values of the one or more extracted parameters in different images on dust artifact illumination, shape, position, reflection or transmission properties, distance of dust to the sensor, aperture, exit pupil, or focal length, or combinations thereof. 98. The method of claim 97, said different images having been acquired of different objects. 99. The method of claim 97, wherein said different images are acquired of different scenes. 100. The method of claim 95, said determining with respect to an aura region being based on an extracted parameter-dependent aura region analysis, wherein the aura region analysis presumes that certain regions on a of the digital image acquisition device are partially obscured by said dust. 101. The method of claim 100, wherein said aura region analysis includes calculating effects of differences in values of the one or more extracted parameters in different images on dust artifact illumination, shape, position, reflection or transmission properties, distance of dust to the sensor, aperture, exit pupil, or focal length, or combinations thereof. 102. The method of claim 101, said different images having been acquired of different objects. 103. The method of claim 100, wherein said different images are acquired of different scenes. 104. The method of claim 95, the correcting operation comprising in-painting or restoration, or both. 105. The method of claim 104, said correcting including in-painting the shadow region. 106. The method of claim 105, said in-painting including determining and applying shadow region correction spectral information based on spectral information obtained from pixels outside said shadow region. 107. The method of claim 104, said correcting including restoration of the aura region. 108. The method of claim 107, said restoration including determining and applying aura region correction spectral information based on spectral information obtained from pixels within said aura region. 109. The method of claim 95, the correcting operation including calculating said aura region and said shadow region. 110. The method of claim 109, the correcting operation further including correcting a shadow region approximately based on the correcting of the aura region. 111. The method of claim 90, the statistical dust map including extracted parameter dependent-variables such that dust artifact region effects in one image are predicted to differ based upon values of the variables within the statistical dust map. 112. The method of claim 90, the dust artifact determining being based at least in part on dust artifact probabilities within said statistical dust map by concentrating on probable regions. 113. The method of claim 90, the dust artifact determining including comparing detected dust artifact regions with dust artifact regions previously detected in different images. 114. The method of claim 113, said different images having been acquired of different objects. 115. The method of claim 113, wherein said different images are acquired of different scenes. 116. The method of claim 113, said different images having been acquired with different values of said one or more extracted parameters. 117. The method of claim 90, the statistical dust map including meta-data dependent dust artifact regions. 118. The method of claim 90, the method further comprising filtering from the statistical dust map dust artifact regions having computed dust artifact probabilities below a threshold level. 119. The method of claim 90, the method further comprising filtering from the statistical dust map dust artifact regions having computed dust artifact probabilities below a threshold level. 120. The method of claim 90, wherein statistical probabilities between dust artifact regions within dust artifact regions within at least two images are determined to have different probabilities between 0 and 1. 121. The method of claim 120, wherein said correcting means varies between images based on different statistical probability values. 122. The method of claim 90, wherein statistical probabilities between at least two dust artifact regions within a same image are determined to have different probabilities between 0 and 1. 123. The method of claim 90, further comprising combining images including dust artifact regions individually computed to have dust artifact probabilities below a threshold level, yet having a combined probability sufficient to determine the dust artifact regions. 124. The method of claim 90, further comprising adding a new dust artifact region to existing dust artifact regions within the dust map when a new dust artifact region is determined. 125. The method of claim 90, further comprising removing a previously determined dust artifact region when it is determined no longer to have dust artifact probability above a threshold level. 126. The method of claim 125, further comprising: (h) deciding whether a predetermined number of dust regions have been removed; and (i) resetting the dust map when said predetermined number have been removed. 127. The method of claim 90, further comprising allocating greater dust artifact determination weights to more recently acquired images. 128. The method of claim 127, further comprising determining that sufficient disparity exists between dust artifact determinations within sequential images to determine that the camera has been cleaned and that the dust artifact determinations prior to the cleaning will not be used in the dust for images obtained after the cleaning. 129. The method of claim 90, further comprising selectively sampling images from said one or more further acquired images for said modifying said statistical dust map. 130. The method of claim 129, said selectively sampling being determined by sequentially displaced images for dust artifact determinations. 131. The method of claim 129, said selectively sampling being determined by one or more criteria comprising total number of images, statistical difference between images, acquisition time, aperture setting, lens type, focal length, exposure time, or overall exposure of images, or combinations thereof. 132. The method of claim 131, said selectively sampling further comprising one or more statistical probabilities of the dust regions in the dust map. 133. The method of claim 90, the method further comprising determining that sufficient dust artifacts exist to prompt a user to physically clean the camera. 134. The method of claim 90, the dust artifact determining comprising categorizing regions within an image into non-dust, dust and inconclusive regions. 135. The method of claim 90, said image correction method being automatically performed within a digital camera that comprises said optical system, said sensor array, said processing electronics and said memory. 136. The method of claim 135, said statistical dust map being stored on said memory of said digital camera. 137. The method of claim 135, said correction being performed prior to saving the image into memory. 138. The method of claim 135, said correction being performed after saving the image into memory. 139. The method of claim 90, said image correction method being performed at least in part within an external processing device that couples with a digital camera that comprises said lens system and a sensor array to form a digital image acquisition and processing system that also comprises said processing electronics and said memory. 140. The system of claim 139, the programming instructions being stored on a memory within the external device which performs the image correction method. 141. The method of claim 139, said external device including a plurality of statistical dust maps associated with a plurality of lens systems. 142. The method of claim 141, further comprising comparing acquired images to said plurality of maps to determine a correct statistical dust map associated with said acquired image. 143. The method of claim 139, said external device including a plurality of said statistical dust maps associated with a plurality of said digital image acquisition devices. 144. The method of claim 143, further comprising comparing acquired images to said plurality of maps to determine a correct statistical dust map associated with said acquired image. 145. The method of claim 143, further comprising user interaction for loading a correct statistical dust map associated with an acquired image from a selection of said plurality of statistical dust maps. 146. The method of claim 139, said extracted parameters being calculated based on a comparison between said dust regions of said acquired images. 147. The method of claim 139, said statistical dust map being downloaded from said digital camera to said external device.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.