Mobile document detection and orientation based on reference object characteristics
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/32
G06K-009/00
G06K-009/18
출원번호
US-0927359
(2015-10-29)
등록번호
US-9760788
(2017-09-12)
발명자
/ 주소
Shustorovich, Alexander
Thrasher, Christopher W.
Ma, Jiyong
Macciola, Anthony
Amtrup, Jan W.
출원인 / 주소
KOFAX, INC.
대리인 / 주소
Zilka-Kotab, P.C.
인용정보
피인용 횟수 :
9인용 특허 :
338
초록▼
In various embodiments, methods, systems, and computer program products for detecting, estimating, calculating, etc. characteristics of a document based on reference objects depicted on the document are disclosed. In one approach, a computer-implemented method for processing a digital image depictin
In various embodiments, methods, systems, and computer program products for detecting, estimating, calculating, etc. characteristics of a document based on reference objects depicted on the document are disclosed. In one approach, a computer-implemented method for processing a digital image depicting a document includes analyzing the digital image to determine one or more of a presence and a location of one or more reference objects; determining one or more geometric characteristics of at least one of the reference objects; defining one or more region(s) of interest based at least in part on one or more of the determined geometric characteristics; and detecting a presence or an absence of an edge of the document within each defined region of interest. Additional embodiments leverage the type of document depicted in the image, multiple frames of image data, and/or calculate or extrapolate document edges rather than locating edges in the image.
대표청구항▼
1. A computer-implemented method for processing a digital image depicting a document, the method comprising: analyzing, using a processor of a mobile device, the digital image to detect one or more of a presence and a location of one or more reference objects;detecting, using the processor of the mo
1. A computer-implemented method for processing a digital image depicting a document, the method comprising: analyzing, using a processor of a mobile device, the digital image to detect one or more of a presence and a location of one or more reference objects;detecting, using the processor of the mobile device, one or more geometric characteristics of at least one of the reference objects, wherein the one or more geometric characteristics are selected from: a height of the at least one of the reference objects, an average height of the at least one of the reference objects, a width of the at least one of the reference objects, an average width of the at least one of the reference objects, an aspect ratio of the at least one of the reference objects, an average aspect ratio of the at least one of the reference objects, an area of the at least one of the reference objects, an average area of the at least one of the reference objects, and a contour of the at least one of the reference objects;defining, using the processor of the mobile device, one or more region(s) of interest of the digital image within which to search for one or more edges of the document, the region(s) of interest being defined based at least in part on one or more of the detected geometric characteristics; anddetecting, using the processor of the mobile device, a presence or an absence of one or more of the edges of the document within each defined region of interest;estimating a bounding box for the document based on the detected presence or absence of the edge of the document within each defined region of interest; andestimating a confidence score of the bounding box based on one or more of: confidence scores computed for each side of the bounding box;a confidence score of an estimated aspect ratio of the bounding box; anda color consistency confidence score. 2. The method as recited in claim 1, wherein the geometric characteristics comprise: the aspect ratio of the at least one of the reference objects. 3. The method as recited in claim 1, wherein detecting the presence of the edge of the document within each defined region of interest further comprises locating a transition from a background region of the digital image to a non-background region of the digital image within the defined region of interest; wherein the non-background region of the digital image depicts at least a portion of the document; andwherein the background region of the digital image does not depict the document. 4. The method as recited in claim 3, wherein the detecting further comprises; determining a longitudinal axis of the located transition; anddetermining whether the longitudinal axis of the located transition corresponds to a longitudinal axis of a region encompassing the one or more reference object(s). 5. The method as recited in claim 4, wherein the longitudinal axes are determined to correspond in response to determining one or more of the following conditions are met: the longitudinal axes are substantially parallel;the longitudinal axes are substantially perpendicular; andat least three points along the longitudinal axis of the located transition are each located within a threshold distance of at least three corresponding points along the longitudinal axis of the region encompassing the one or more reference object(s). 6. The method as recited in claim 1, wherein the reference object(s) comprise one or more of magnetic ink character recognition (MICR) characters and machine readable zone (MRZ) characters. 7. A computer-implemented method for processing a plurality of digital image frames, each frame comprising at least a partial digital representation of a document, and the method comprising: analyzing, using a processor of the computer, a first frame among the plurality of frames to determine one or more of a presence and a location of one or more reference objects within the first frame;detecting, using the processor, one or more geometric characteristics of at least one reference object within the first frame, wherein the one or more geometric characteristics are selected from: a height of the at least one reference object, an average height of the at least one reference object, a width of the at least one reference object, an average width of the at least one reference object, an aspect ratio of the at least one reference object, an average aspect ratio of the at least one reference object, an area of the at least one reference object, an average area of the at least one reference object, and a contour of the at least one reference object;defining, using the processor, one or more region(s) of interest of the first frame within which to search for one or more edges of the document, the one or more region(s) of interest being defined based at least in part on: a priori knowledge regarding a relative position of the at least one reference object within the document; andone or more of the determined geometric characteristics of the at least one reference object within the first frame;defining, using the processor, one or more subregion(s) of interest within which to search for the one or more edges of the document, the one or more subregion(s) of interest being defined within a second frame;detecting, using the processor, a presence or an absence of the one or more of the edges of the document within each defined subregion of interest;wherein each subregion of interest is defined based at least in part on: one or more geometric characteristics of at least one of the region(s) of interest within which the subregion of interest was defined; andone or more of the geometric characteristics of the reference object(s) detected within the first frame; andestimating a bounding box for the document based on the detected presence or absence of the edge of the document within each defined region of interest; andestimating a confidence score of the bounding box based on one or more of: confidence scores computed for each side of the bounding box;a confidence score of an estimated aspect ratio of the bounding box; anda color consistency confidence score. 8. The method as recited in claim 7, wherein the geometric characteristics comprise: the aspect ratio of the at least one reference object. 9. The method as recited in claim 7, wherein detecting the presence of the edge of the document within each defined subregion of interest further comprises: locating a transition from a background region of the digital image to a non-background region of the digital image within the defined region of interest wherein the non-background region of the digital image depicts at least a portion of the document; andwherein the background region of the digital image does not depict the document. 10. The method as recited in claim 9, wherein detecting the presence or the absence of the one or more of the edges of the document within each defined subregion of interest further comprises: determining a longitudinal axis of the located transition; anddetermining whether the longitudinal axis of the located transition corresponds to a longitudinal axis of the subregion encompassing the one or more reference object(s). 11. The method as recited in claim 10, wherein the longitudinal axes are determined to correspond in response to determining one or more of the following conditions are met: the longitudinal axes are substantially parallel;the longitudinal axes are substantially perpendicular; andat least three points along the longitudinal axis of the located transition are each located within a threshold distance of at least three corresponding points along the longitudinal axis of the subregion encompassing the one or more reference object(s). 12. The method as recited in claim 7, wherein the reference object(s) comprise one or more of magnetic ink character recognition (MICR) characters and machine readable zone (MRZ) characters. 13. The method as recited in claim 7, wherein each subregion of interest is characterized by satisfying one or more of the following criteria: each subregion of interest is located within a corresponding region of interest defined for the first frame;each subregion of interest shares at least one boundary with a corresponding region of interest defined for the first frame;each subregion of interest encompasses an entirety of a corresponding region of interest defined for the first frame;each subregion of interest encompasses a region bounding a corresponding region of interest defined for the first frame; andeach subregion of interest excludes an area encompassed by a corresponding region of interest defined for the first frame. 14. A computer-implemented method for processing a digital image comprising a digital representation of a document, the method comprising: determining, using a processor of the computer, the document corresponds to one of a predefined set of document types;analyzing, using the processor, the digital image to determine a location of one or more reference objects within the document;detecting, using the processor, one or more geometric characteristics of at least one of the reference objects, wherein the one or more geometric characteristics are selected from: a height of the at least one of the reference objects, an average height of the at least one of the reference objects, a width of the at least one of the reference objects, an average width of the at least one of the reference objects, an aspect ratio of the at least one of the reference objects, an average aspect ratio of the at least one of the reference objects, an area of the at least one of the reference objects, an average area of the at least one of the reference objects, and a contour of the at least one of the reference objects;extrapolating, using the processor, a location of one or more edges of the document within the digital image based at least in part on: the determined document type, the determined location of the one or more reference objects, and the detected geometric characteristics of the at least one of the reference objects; andestimating a bounding box for the document based on the extrapolated location of the one or more edges of the document; andestimating a confidence score of the bounding box based on one or more of: confidence scores computed for each side of the bounding box;a confidence score of an estimated aspect ratio of the bounding box; anda color consistency confidence score; andoutputting the bounding box, the extrapolated location of the one or more edges of the document, or both to a display of a mobile device. 15. The method as recited in claim 14, further comprising: defining one or more region(s) of interest within the digital image based at least in part on the extrapolated document edge location(s) and one or more of the detected geometric characteristics; anddetecting a presence or an absence of a true edge of the document within each defined region of interest. 16. The method as recited in claim 14, wherein the geometric characteristics comprise: the aspect ratio of the at least one of the reference objects. 17. The computer-implemented method as recited in claim 1, wherein the document is characterized by at least four edges;wherein at least one of the edges is an interior edge defined by a fold within the document; andwherein at least the interior edge is detected within some or all of the one or more region(s) of interest. 18. The method as recited in claim 1, wherein the one or more reference objects comprise an inner frame or border of the document. 19. The method as recited in claim 1, wherein the confidence scores for each side of the bounding box are estimated based on a percentage of white pixels along a corresponding side of a binary edge image generated based on the digital image; and wherein the confidence score of the estimated aspect ratio of the bounding box is either: a ratio of the estimated aspect ratio of the bounding box divided by a real aspect ratio of the document; ora ratio of the real aspect ratio of the document divided by the estimated aspect ratio of the bounding box.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (338)
Kawasaki, Somei; Goden, Tatsuhito, Active matrix type display apparatus and driving method thereof.
Gaborski Roger Stephen (Pittsford NY) Pawlicki Thaddeus Francis (Rochester NY), Apparatus and method for identifying specific bone regions in digital X-ray images.
Nakatsuka Kimihiro,JPX, Apparatus for determining image processing parameter, method of the same, and computer program product for realizing the method.
Barrett Terence W. (Vienna VA), Automata networks and methods for obtaining optimized dynamically reconfigurable computational architectures and control.
Block, James; Graef, H. Thomas; Magee, Paul D.; Nelson, Donald S.; Meek, James; McIntyre, Daniel S.; DiPietro, Mark; Ramachandran, Natarajan, Automated banking machine with remote user assistance.
Sang ; Jr. Henry W. (Cupertio CA) Tahn Whei-Tsu H. (Sunnyvale CA) Zhang Xiao B. (Foster City CA), Automated method for creating templates in a forms recognition and processing system.
Iwai, Yoshiaki; Yoshigahara, Takayuki, Camera calibration apparatus and method, image processing apparatus and method, program providing medium, and camera.
McElroy, John F.; Chorvat, Robert J., Cannabinoid receptor antagonists/inverse agonists useful for treating metabolic disorders, including obesity and diabetes.
Nishimura Kazuyuki (Ichikawa JPX) Sato Shinichi (Yokohama JPX), Color picture processing apparatus for reproducing a color picture having a smoothly changed gradation.
Suzuki,Masahiro; Tamune,Michihiro; Chen,Zhe Hong; Juen,Masahiro, Digital camera, storage medium for image signal processing, carrier wave and electronic camera.
Rowe Edward R. ; Priyadarshan Eswar ; Anderson Kenneth S. ; Al-Shamma Nabeel A. ; Taft Edward A. ; McQuarrie Elizabeth M. ; Cohn Richard, Displaying electronic documents with substitute fonts.
Nagatsuka,Tetsuro; Miyachi,Tatsuo; Shimada,Atsuo; Takeya,Kazutoshi; Kemmochi,Eiji; Nakajima,Akiko; Yamasaki,Makoto; Fujita,Katsuhiko, Document classification system and method for classifying a document according to contents of the document.
Borrey Roland G. (19251 Canyon Dr. Villa Park CA 92667) Borrey Daniel G. (19251 Canyon Dr. Villa Park CA 92667), Document identification by characteristics matching.
Clark ; Jr. Louis George (St. Charles MO) Gummow ; Jr. Donald Romaine (O\Fallon MO) Vanacht Marc (St. Louis MO), Hand-held GUI PDA with GPS/DGPS receiver for collecting agronomic and GPS position data.
LeBrun Thomas Q. (Dallas TX) Cage Kerry (Carrollton TX) Arnold Dennis D. (Carrollton TX), Image based document processing and information management system and apparatus.
Mino, Kazuhiro; Yoda, Akira; Ohtsuka, Shuichi; Ono, Shuji; Ito, Wataru; Yamada, Masahiko, Image displaying system and apparatus for displaying images by changing the displayed images based on direction or direction changes of a displaying unit.
Naofumi Yamamoto JP; Haruko Kawakami JP; Gururaj Rao JP, Image processing apparatus for discriminating image field of original document plural times and method therefor.
Appelt, Douglas E.; Arnold, James Frederick; Bear, John S.; Hobbs, Jerry Robert; Israel, David J.; Kameyama, Megumi; Martin, David L.; Myers, Karen Louise; Ravichandran, Gopalan; Stickel, Mark Edward, Information retrieval by natural language querying.
David L. Patton ; John R. Fredlund ; John D. Buhr, Method and apparatus for modifying a portion of an image in accordance with colorimetric parameters.
Walnut David Francis ; Berenstein Carlos Alberto ; Liu K. J. Ray ; Rashid-Farrokhi Farrokh, Method and apparatus for processing data from a tomographic imaging system.
Withers,William Douglas, Method and apparatus for recognizing a digitized form, extracting information from a filled-in form, and generating a corrected filled-in form.
Guberman Shelja A. (Moscow RUX) Lossev Ilia (Moscow RUX) Pashintsev Alexander V. (Moscow RUX), Method and apparatus for recognizing cursive writing from sequential input information.
Guberman Shelja A. (Moscow RUX) Lossev Ilia (Moscow RUX) Pashintsev Alexander V. (Moscow RUX), Method and apparatus for recognizing cursive writing from sequential input information.
Polyakov Vladislav G. (Moscow RUX) Ryleev Mikhail A. (Moscow RUX), Method and apparatus for representing image data using polynomial approximation method and iterative transformation-repa.
Green, Stephen J.; Lamere, Paul B.; Alexander, Jeffrey L.; Haberl, Karl R., Method and apparatus for searching and resource discovery in a distributed enterprise system.
Winkelman Kurt-Helfried (Kiel DEX), Method and apparatus for the automatic analysis of density range, color cast, and gradation of image originals on the Ba.
Berman, Arie; Vlahos, Paul; Dadourian, Arpag, Method and apparatus for the automatic generation of subject to background transition area boundary lines and subject shadow retention.
Verstraelen,Boudewijn Joseph Angelus; Verstraelen,Sebastiaan Paul, Method and apparatus for visualization of biological structures with use of 3D position information from segmentation results.
Ejiri Koichi,JPX ; Guan Haike,JPX ; Aoki Shin,JPX, Method and system for generating a composite image from partially overlapping adjacent images taken along a plurality of axes.
Tischler, Karl M., Method arrangement and computer software for the printing of a separator sheet by means of an electrophotographic printer or copier.
Raskar, Ramesh; Willwacher, Thomas H.; van Baar, Jeroen, Method for determining a largest inscribed rectangular image within a union of projected quadrilateral images.
Kanda Shinji (Kawasaki JPX) Wakitani Jun (Kawasaki JPX) Maruyama Tsugito (Kawasaki JPX) Morita Toshihiko (Kawasaki JPX), Method for determining orientation of contour line segment in local area and for determining straight line and corner.
Kurosu Yasuo (Yokosuka JPX) Yokoyama Yoshihiro (Yokohama JPX) Nishikawa Kenichi (Yokohama JPX) Masuzaki Hidefumi (Hadano JPX) Fujinawa Masaaki (Tokyo JPX), Method for determining the amount of skew of image, method for correcting the same, and image data processing system.
Henderson Todd R. ; Spaulding Kevin E. ; Couwenhoven Douglas W., Method for segmenting a digital image into a foreground region and a key color region.
Kohchi Tsukasa JP, Method of and system for extracting predetermined elements from input document based upon model which is adaptively modified according to variable amount in the input document.
Beaulieu Dennis N. (Churchville NY) Compton John T. (LeRoy NY) Wojtanik Eugene R. (Plano TX), Method of calibration of image scanner signal processing circuits.
Dumais Susan T. ; Heckerman David ; Horvitz Eric ; Platt John Carlton ; Sahami Mehran, Methods and apparatus for classifying text and for building a text classifier.
Cheong, Cheol Ho; Han, Tack Don; Kim, Jong Young; Kim, Eui Jae; Jeong, Seong Hun; Kim, Jae Yun; Choi, Han Yeong, Mixed code, and method and apparatus for generating the same.
Fast Bruce B. (2600 Prindle Rd. Belmont CA 94402) Allen Dana R. (1745 Hunt Dr. Burlingame CA 94010), OCR image preprocessing method for image enhancement of scanned documents.
Michimoto Yasuyuki,JPX ; Onda Katsumasa,JPX ; Nishizawa Masato,JPX, Object detecting apparatus in which the position of a planar object is estimated by using hough transform.
Wong, Patrick, System and a method for web-based editing of documents online with an editing interface and concurrent display to webpages and print documents.
Ellis, Stephen M.; Kennedy, Michael J.; Kurani, Ashish Bhoopen; Lowry, Melissa; Meyyappan, Uma; Sahni, Bipin; Stroke, Nikolai, System and method for a mobile wallet.
Woolf,Susan D.; Baird,Andrew; Jiang,Sheng; Beezer,John L.; Rubin,Darryl E., System and method for annotating an electronic document independently of its content.
Pizano Arturo (Milpitas CA) Tan May-Inn (Saratoga CA) Gambo Naoto (Tanashi JPX), System and method for automatically classifying heterogeneous business forms.
Vazquez, Nicolas; Kodosky, Jeffrey L.; Kudukoli, Ram; Schultz, Kevin L.; Nair, Dinesh; Caltagirone, Christophe, System and method for automatically generating a graphical program to perform an image processing algorithm.
Oppenlander, Timothy J.; Underhill, James; Jackson, Elizabeth; Cook, Rebecca Ann; Dimel, Gary R.; Ortize, Carlos, System and method for electronic document generation and delivery.
Emerson,Geoffrey A.; Moon,Rodney G.; Rector,Gerald C.; Stokes,Raymond F.; Sutton,Andrew H., System and method of sorting document images based on image quality.
Heidenreich,James R.; Higgins,Linda S., System and method to customize the facilitation of development of user thinking about and documenting of an arbitrary problem.
Sampath, Meera; Nichols, Stephen J.; Richenderfer, Elizabeth A., Systems and methods for automated image quality based diagnostics and remediation of document processing systems.
Amtrup, Jan W.; Macciola, Anthony; Thompson, Stephen Michael; Ma, Jiyong, Systems and methods for classifying objects in digital images captured using mobile devices.
Amtrup, Jan Willers; Macciola, Anthony; Thompson, Steve; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for classifying objects in digital images captured using mobile devices.
Amtrup, Jan W.; Ma, Jiyong; Kilby, Steven; Macciola, Anthony, Systems and methods for identification document processing and business workflow integration.
Amtrup, Jan W.; Thompson, Stephen Michael; Kilby, Steven; Macciola, Anthony, Systems and methods for identification document processing and business workflow integration.
Ferlitsch,Andrew Rodney; DeVore,Darwin Alan, Systems and methods for manipulating electronic information using a three-dimensional iconic representation.
Amtrup, Jan Willers; Macciola, Anthony; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for mobile image capture and processing.
Macciola, Anthony; Amtrup, Jan Willers; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for mobile image capture and processing.
Roach, John J.; Nepomniachtchi, Grisha; Couch, Robert; Avergun, Mikhail, Systems and methods for obtaining financial offers using mobile image capture.
Macciola, Anthony; Amtrup, Jan W.; Ma, Jiyong; Borrey, Roland G.; Schmidtler, Mauritius A. R.; Asuri, Hari S.; Fechter, Joel S.; Taylor, Robert A., Systems and methods for processing video data.
Gorski, Nikolai D.; Semenov, Andrey V.; Anisimov, Valery; Maksimov, Sergey K.; Sashov, Sergey N., Systems and methods for recognizing information in objects using a mobile device.
Macciola, Anthony; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher W.; Amtrup, Jan, Systems and methods for three dimensional geometric reconstruction of captured image data.
Borrey, Roland G.; Schmidtler, Mauritius A. R.; Taylor, Robert A.; Fechter, Joel S.; Asuri, Hari S., Systems and methods of accessing random access cache for rescanning.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods and computer program products for determining document validity.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods and computer program products for determining document validity.
Schmidtler, Mauritius A. R.; Borrey, Roland G.; Amtrup, Jan W.; Thompson, Stephen Michael, Systems, methods, and computer program products for determining document validity.
Macciola, Anthony; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher; Amtrup, Jan W., Determining distance between an object and a capture device based on captured image data.
Thrasher, Christopher W.; Shustorovich, Alexander; Thompson, Stephen Michael; Amtrup, Jan W.; Macciola, Anthony, Iterative recognition-guided thresholding and data extraction.
Amtrup, Jan W.; Macciola, Anthony; Thompson, Steve; Ma, Jiyong; Shustorovich, Alexander; Thrasher, Christopher W., Systems and methods for classifying objects in digital images captured using mobile devices.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.