Systems and methods for three dimensional geometric reconstruction of captured image data
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G06T-003/00
출원번호
US-0491901
(2014-09-19)
등록번호
US-9208536
(2015-12-08)
발명자
/ 주소
Macciola, Anthony
Ma, Jiyong
Shustorovich, Alexander
Thrasher, Christopher W.
Amtrup, Jan
출원인 / 주소
Kofax, Inc.
대리인 / 주소
Zilka-Kotab, PC
인용정보
피인용 횟수 :
39인용 특허 :
255
초록▼
In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impracti
In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impractical, and furthermore address unique challenges presented by images captured using a camera rather than a traditional flat-bed scanner, paper-feed scanner, or multifunction peripheral. Notably, the presently disclosed systems and techniques enable three-dimensional reconstruction of objects depicted in image captured using a camera of a mobile device. The reconstruction corrects or compensates for perspective distortion caused by camera-based capture.
대표청구항▼
1. A method of reconstructing a digital image, the method comprising: receiving the digital image comprising a digital representation of an object bounded by a tetragon;dividing the digital image into a plurality of digital image segments;performing a segmentation analysis on the plurality of digita
1. A method of reconstructing a digital image, the method comprising: receiving the digital image comprising a digital representation of an object bounded by a tetragon;dividing the digital image into a plurality of digital image segments;performing a segmentation analysis on the plurality of digital image segments;correcting curvature in the tetragon to form a quadrilateral based at least in part on the segmentation analysis;correcting projective effects in the quadrilateral to form a rectangle based at least in part on the segmentation analysis; andestimating an absolute size of the object based on intrinsic parameters of a capture device used to capture the digital image. 2. The method as recited in claim 1, wherein correcting the curvature comprises transforming one or more portions of the tetragon based at least in part on intrinsic tetragon coordinates using a four-point algorithm. 3. The method as recited in claim 1, wherein the digital image is characterized by a capture angle of about 30 degrees or more deviation from normal with respect to the object. 4. The method as recited in claim 1, wherein correcting the projective effects comprises one or more of singular value decomposition (SVD) and direct linear transformation (DLT). 5. The method as recited in claim 1, further comprising estimating a two-dimensional pixel position for each of up to four pixels in the digital image, each of the up to four pixels corresponding to one of four corners of one or more of the object and the tetragon. 6. The method as recited in claim 5, wherein estimating the two-dimensions pixel position for each of the four pixels is based on a predetermined correspondence between each of the four pixels and four corresponding pixels of a reference image, wherein the reference image depicts the digital representation of the object from a perspective corresponding to a capture angle normal to the object, andwherein each of the four corresponding pixels of the reference image corresponds to one of four corners of the digital representation of the object in the reference image. 7. The method as recited in claim 1, further comprising estimating a two-dimensional pixel position for each of four corners of one or more of the object and the tetragon, and wherein at least one of the four corners is characterized by being one or more of: obstructed in the digital image;not depicted in the digital image; andblurred beyond recognition in the digital image. 8. The method as recited in claim 1, wherein correcting the projective effects comprises estimating a homography transform H,wherein estimating H comprises detecting one or more point correspondences pi Pi′ with pi=(xi,yi,1)T,wherein estimating H comprises detecting four point correspondences, each point correspondence piPi′ corresponding to a corner position pi of the tetragon and a respective corner position P′i of the rectangle, andwherein H satisfies the expression λpi=HP′i, where λ=a focal distance between a three-dimensional reference coordinate position (X, Y, Z) and a corresponding reference coordinate position of a capture device; pi=(xi,yi,1)T;P′I=(Xi,Yi,1)T, and i is the respective index feature of the point p or P′ in the corresponding image. 9. The method as recited in claim 1, wherein correcting the projective effects comprises performing a non-linear refinement. 10. The method as recited in claim 1, further comprising outputting the reconstructed digital image, wherein the reconstructed digital image is characterized by a pixel location error of about 5 pixels or less relative to a scanned 500-DPI image of the object captured using a capture angle normal to the object. 11. The method as recited in claim 1, wherein at least one side of the tetragon is characterized by a second degree or higher order polynomial. 12. The method as recited in claim 1, wherein correcting the projective effects comprises normalizing coordinates of point-correspondences between the digital image and a reconstructed image. 13. The method as recited in claim 1, wherein the object is a document characterized by a known height-to-width ratio. 14. The method as recited in claim 1, wherein the estimating employs an intrinsic parameter matrix A representative of the intrinsic capture device parameters. 15. The method as recited in claim 14, wherein the intrinsic parameter matrix A is characterized by an expression A=(acdbe1),andwherein a and b are each scaling factors which comprise focal length information corresponding to the capture device. 16. The method as recited in claim 15, wherein a=f/dx wherein b=f/dy,wherein f is the focal length,wherein dx is a horizontal scaling factor,wherein dy is a vertical scaling factor,wherein c is a skew parameter, andwherein (d, e) are the coordinates of a principal point in the digital image. 17. The method as recited in claim 15, wherein c =0, wherein either: (d, e) are the coordinates of a principal point in the digital image, ord=0 and e=0. 18. A system, comprising: a processor configured to execute logic; andlogic configured to receive a digital image comprising a digital representation of an object bounded by a tetragon;logic configured to correct curvature in the tetragon to form a quadrilateral; andlogic configured to correct projective effects in the quadrilateral to form a rectangle, wherein correcting the projective effects is based at least in part on estimating corner positions of the object; andlogic configured to estimate an absolute size of the object based on intrinsic parameters of a capture device used to capture the digital image, andwherein at least one corner of the object for which the corner position is estimated is characterized by being one or more of: obstructed in the digital image;not depicted in the digital image; andblurred beyond recognition in the digital image. 19. A computer program product comprising a non-transitory computer readable storage medium having computer readable program code stored thereon, the computer readable program code comprising: computer readable program code configured to receive a digital image comprising a digital representation of an object bounded by a tetragon;computer readable program code configured to correct curvature in the tetragon to form a quadrilateral;computer readable program code configured to correct projective effects in the quadrilateral to form a rectangle; andcomputer readable program code configured to estimate an absolute size of the object based on intrinsic parameters of a capture device used to capture the digital image.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (255)
Kawasaki, Somei; Goden, Tatsuhito, Active matrix type display apparatus and driving method thereof.
Nakatsuka Kimihiro,JPX, Apparatus for determining image processing parameter, method of the same, and computer program product for realizing the method.
Barrett Terence W. (Vienna VA), Automata networks and methods for obtaining optimized dynamically reconfigurable computational architectures and control.
Sang ; Jr. Henry W. (Cupertio CA) Tahn Whei-Tsu H. (Sunnyvale CA) Zhang Xiao B. (Foster City CA), Automated method for creating templates in a forms recognition and processing system.
Iwai, Yoshiaki; Yoshigahara, Takayuki, Camera calibration apparatus and method, image processing apparatus and method, program providing medium, and camera.
McElroy, John F.; Chorvat, Robert J., Cannabinoid receptor antagonists/inverse agonists useful for treating metabolic disorders, including obesity and diabetes.
Nishimura Kazuyuki (Ichikawa JPX) Sato Shinichi (Yokohama JPX), Color picture processing apparatus for reproducing a color picture having a smoothly changed gradation.
Suzuki,Masahiro; Tamune,Michihiro; Chen,Zhe Hong; Juen,Masahiro, Digital camera, storage medium for image signal processing, carrier wave and electronic camera.
Rowe Edward R. ; Priyadarshan Eswar ; Anderson Kenneth S. ; Al-Shamma Nabeel A. ; Taft Edward A. ; McQuarrie Elizabeth M. ; Cohn Richard, Displaying electronic documents with substitute fonts.
Nagatsuka,Tetsuro; Miyachi,Tatsuo; Shimada,Atsuo; Takeya,Kazutoshi; Kemmochi,Eiji; Nakajima,Akiko; Yamasaki,Makoto; Fujita,Katsuhiko, Document classification system and method for classifying a document according to contents of the document.
Borrey Roland G. (19251 Canyon Dr. Villa Park CA 92667) Borrey Daniel G. (19251 Canyon Dr. Villa Park CA 92667), Document identification by characteristics matching.
Clark ; Jr. Louis George (St. Charles MO) Gummow ; Jr. Donald Romaine (O\Fallon MO) Vanacht Marc (St. Louis MO), Hand-held GUI PDA with GPS/DGPS receiver for collecting agronomic and GPS position data.
LeBrun Thomas Q. (Dallas TX) Cage Kerry (Carrollton TX) Arnold Dennis D. (Carrollton TX), Image based document processing and information management system and apparatus.
Mino, Kazuhiro; Yoda, Akira; Ohtsuka, Shuichi; Ono, Shuji; Ito, Wataru; Yamada, Masahiko, Image displaying system and apparatus for displaying images by changing the displayed images based on direction or direction changes of a displaying unit.
Naofumi Yamamoto JP; Haruko Kawakami JP; Gururaj Rao JP, Image processing apparatus for discriminating image field of original document plural times and method therefor.
Appelt, Douglas E.; Arnold, James Frederick; Bear, John S.; Hobbs, Jerry Robert; Israel, David J.; Kameyama, Megumi; Martin, David L.; Myers, Karen Louise; Ravichandran, Gopalan; Stickel, Mark Edward, Information retrieval by natural language querying.
Walnut David Francis ; Berenstein Carlos Alberto ; Liu K. J. Ray ; Rashid-Farrokhi Farrokh, Method and apparatus for processing data from a tomographic imaging system.
Withers,William Douglas, Method and apparatus for recognizing a digitized form, extracting information from a filled-in form, and generating a corrected filled-in form.
Guberman Shelja A. (Moscow RUX) Lossev Ilia (Moscow RUX) Pashintsev Alexander V. (Moscow RUX), Method and apparatus for recognizing cursive writing from sequential input information.
Guberman Shelja A. (Moscow RUX) Lossev Ilia (Moscow RUX) Pashintsev Alexander V. (Moscow RUX), Method and apparatus for recognizing cursive writing from sequential input information.
Polyakov Vladislav G. (Moscow RUX) Ryleev Mikhail A. (Moscow RUX), Method and apparatus for representing image data using polynomial approximation method and iterative transformation-repa.
Green, Stephen J.; Lamere, Paul B.; Alexander, Jeffrey L.; Haberl, Karl R., Method and apparatus for searching and resource discovery in a distributed enterprise system.
Winkelman Kurt-Helfried (Kiel DEX), Method and apparatus for the automatic analysis of density range, color cast, and gradation of image originals on the Ba.
Berman, Arie; Vlahos, Paul; Dadourian, Arpag, Method and apparatus for the automatic generation of subject to background transition area boundary lines and subject shadow retention.
Verstraelen,Boudewijn Joseph Angelus; Verstraelen,Sebastiaan Paul, Method and apparatus for visualization of biological structures with use of 3D position information from segmentation results.
Ejiri Koichi,JPX ; Guan Haike,JPX ; Aoki Shin,JPX, Method and system for generating a composite image from partially overlapping adjacent images taken along a plurality of axes.
Tischler, Karl M., Method arrangement and computer software for the printing of a separator sheet by means of an electrophotographic printer or copier.
Raskar, Ramesh; Willwacher, Thomas H.; van Baar, Jeroen, Method for determining a largest inscribed rectangular image within a union of projected quadrilateral images.
Kurosu Yasuo (Yokosuka JPX) Yokoyama Yoshihiro (Yokohama JPX) Nishikawa Kenichi (Yokohama JPX) Masuzaki Hidefumi (Hadano JPX) Fujinawa Masaaki (Tokyo JPX), Method for determining the amount of skew of image, method for correcting the same, and image data processing system.
Henderson Todd R. ; Spaulding Kevin E. ; Couwenhoven Douglas W., Method for segmenting a digital image into a foreground region and a key color region.
Kohchi Tsukasa JP, Method of and system for extracting predetermined elements from input document based upon model which is adaptively modified according to variable amount in the input document.
Beaulieu Dennis N. (Churchville NY) Compton John T. (LeRoy NY) Wojtanik Eugene R. (Plano TX), Method of calibration of image scanner signal processing circuits.
Dumais Susan T. ; Heckerman David ; Horvitz Eric ; Platt John Carlton ; Sahami Mehran, Methods and apparatus for classifying text and for building a text classifier.
Michimoto Yasuyuki,JPX ; Onda Katsumasa,JPX ; Nishizawa Masato,JPX, Object detecting apparatus in which the position of a planar object is estimated by using hough transform.
Ellis, Stephen M.; Kennedy, Michael J.; Kurani, Ashish Bhoopen; Lowry, Melissa; Meyyappan, Uma; Sahni, Bipin; Stroke, Nikolai, System and method for a mobile wallet.
Woolf,Susan D.; Baird,Andrew; Jiang,Sheng; Beezer,John L.; Rubin,Darryl E., System and method for annotating an electronic document independently of its content.
Vazquez, Nicolas; Kodosky, Jeffrey L.; Kudukoli, Ram; Schultz, Kevin L.; Nair, Dinesh; Caltagirone, Christophe, System and method for automatically generating a graphical program to perform an image processing algorithm.
Emerson,Geoffrey A.; Moon,Rodney G.; Rector,Gerald C.; Stokes,Raymond F.; Sutton,Andrew H., System and method of sorting document images based on image quality.
Heidenreich,James R.; Higgins,Linda S., System and method to customize the facilitation of development of user thinking about and documenting of an arbitrary problem.
Sampath, Meera; Nichols, Stephen J.; Richenderfer, Elizabeth A., Systems and methods for automated image quality based diagnostics and remediation of document processing systems.
Ferlitsch,Andrew Rodney; DeVore,Darwin Alan, Systems and methods for manipulating electronic information using a three-dimensional iconic representation.
Roach, John J.; Nepomniachtchi, Grisha; Couch, Robert; Avergun, Mikhail, Systems and methods for obtaining financial offers using mobile image capture.
Gorski, Nikolai D.; Semenov, Andrey V.; Anisimov, Valery; Maksimov, Sergey K.; Sashov, Sergey N., Systems and methods for recognizing information in objects using a mobile device.
Borrey, Roland G.; Schmidtler, Mauritius A. R.; Taylor, Robert A.; Fechter, Joel S.; Asuri, Hari S., Systems and methods of accessing random access cache for rescanning.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods and computer program products for determining document validity.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods and computer program products for determining document validity.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods, and computer program products for determining document validity.
Ma, Jiyong; Thompson, Stephen Michael; Amtrup, Jan W., Content-based detection and three dimensional geometric reconstruction of objects in image and video data.
Macciola, Anthony; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher; Amtrup, Jan W., Determining distance between an object and a capture device based on captured image data.
Thrasher, Christopher W.; Shustorovich, Alexander; Thompson, Stephen Michael; Amtrup, Jan W.; Macciola, Anthony, Iterative recognition-guided thresholding and data extraction.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving a broadcast radio service offer from an image.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving a broadcast radio service offer from an image.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving a location of a vehicle service center from an image.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving a location of a vehicle service center from an image.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving vehicle information from an image and posting the vehicle information to a website.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for receiving vehicle information from an image and posting the vehicle information to a website.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for recovering a vehicle identification number from an image.
Wilbert, Anthony Russell; Wach, Hans Brandon; Chung, David Ching-Chien, Method and apparatus for recovering a vehicle identification number from an image.
Shustorovich, Alexander; Thrasher, Christopher W.; Ma, Jiyong; Macciola, Anthony; Amtrup, Jan W., Mobile document detection and orientation based on reference object characteristics.
Wilbert, Anthony Russell; Chung, David Ching-Chien; Wach, Hans Brandon; Rauker, Goran Matko; White, Solomon John, System and method for electronic processing of vehicle transactions based on image detection of vehicle license plate.
Amtrup, Jan W.; Macciola, Anthony; Thompson, Steve; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for classifying objects in digital images captured using mobile devices.
Amtrup, Jan W.; Macciola, Anthony; Thompson, Steve; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for classifying objects in digital images captured using mobile devices.
Thrasher, Christopher W.; Shustorovich, Alexander; Thompson, Stephen Michael; Amtrup, Jan W.; Macciola, Anthony; Borrey, Roland G.; Schmidtler, Mauritius A. R.; Taylor, Robert A.; Fechter, Joel S.; Asuri, Hari S., Systems and methods of processing scanned data.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.