Image processing method for detecting objects using relative motion
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G06T-007/20
출원번호
US-0113415
(2012-01-20)
등록번호
US-9547795
(2017-01-17)
국제출원번호
PCT/CA2012/000057
(2012-01-20)
§371/§102 date
20131216
(20131216)
국제공개번호
WO2012/145819
(2012-11-01)
발명자
/ 주소
Gupta, Nikhil
출원인 / 주소
MAGNA ELECTRONICS INC.
대리인 / 주소
Gardner, Linn, Burkhart & Flory, LLP
인용정보
피인용 횟수 :
3인용 특허 :
282
초록▼
An image based obstacle detection method. A camera mounted on a vehicle provides a set of image frames while the vehicle is in motion. The image frames define an image plane having a vertical aspect and a horizontal aspect. The relevancy of an object is determined by (i) selecting first and second f
An image based obstacle detection method. A camera mounted on a vehicle provides a set of image frames while the vehicle is in motion. The image frames define an image plane having a vertical aspect and a horizontal aspect. The relevancy of an object is determined by (i) selecting first and second feature points from the object that are spaced apart vertically in a first image frame; (ii) tracking the positions of the first and second feature points over at least a second image frame; and (iii) deciding the object to be relevant if the first and second feature points move dissimilar distances in physical space, within a tolerance, and deciding the object to be irrelevant otherwise. The motion of relevant objects is then estimated to determine if any relevant object is likely to become an obstacle to the vehicle.
대표청구항▼
1. An obstacle detection method, comprising: (a) provisioning a vehicle with a camera having an external field of view, the camera having an imager defining an image plane having a vertical aspect and a horizontal aspect, wherein the imager comprises an array of sensing pixels;(b) acquiring a set of
1. An obstacle detection method, comprising: (a) provisioning a vehicle with a camera having an external field of view, the camera having an imager defining an image plane having a vertical aspect and a horizontal aspect, wherein the imager comprises an array of sensing pixels;(b) acquiring a set of image frames captured by the camera while the vehicle is in motion;(c) determining an object present in the set of captured image frames by edge detection;(d) determining a relevancy of the determined object by (i) selecting first and second feature points of the determined object that are spaced apart vertically in a first captured image frame,(ii) tracking positions of the first and second feature points of the determined object over at least a second captured image frame, and(iii) determining that the determined object is a relevant object responsive to a determination that the first and second feature points move dissimilar distances in physical space between the first and second captured image frames and within a tolerance;(e) estimating motion of determined relevant objects relative to the vehicle to determine if any determined relevant object is likely to become an obstacle to the vehicle;wherein step (d)(iii) is determined by reference to the image plane and, where the first feature point is vertically lower than the second feature point, and wherein the method includes:estimating, from the first captured image frame, a physical distance D1 of the first feature point O1b to the camera in a selected physical horizontal plane;estimating, from the first captured image frame, a physical location DT of the second feature point O1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane;estimating, from the second captured image frame, a physical distance D2 of the first feature point O2b to the camera in the selected physical horizontal plane, and determining the distance ΔD between D2 and D1;estimating a physical location D*T which is equivalent to DT+ΔD, with D*T being located at a height T above the selected physical horizontal plane;computing a nominal feature point coordinate O*2t in the image plane corresponding to D*T; anddetermining that the determined object is the relevant object responsive to a determination that the absolute distance between the nominal feature point coordinate O*2t and the second feature point coordinate in the first captured image frame O1T is less than the absolute distance between the second feature point coordinate in the second captured image frame O2T and the second feature point coordinate in the first captured image frame O1T. 2. A method according to claim 1, wherein step (d)(iii) is determined by reference to physical space and includes: determining first and second physical locations along a selected physical horizontal plane corresponding to pixel positions of the first and second feature points in the first captured image frame;determining third and fourth physical locations along the selected real horizontal plane corresponding to pixel positions of the first and second feature points in the second captured image frame;determining a first distance Δd1 between the third and first physical locations and determining a second distance Δd2 between the fourth and second physical locations; anddetermining that the determined object is the relevant object responsive to a determination that the difference between the first distance Δd1 and the second distance Δd2 exceeds a threshold, and otherwise determining that the determined object is not a relevant object. 3. A method according to claim 2, including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position. 4. A method according to claim 3, wherein the first and second feature points are portions of top and bottom edge contours of the determined object. 5. A method according to claim 1, wherein the selected physical horizontal plane is a ground plane. 6. A method according to claim 5, including unwarping a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position. 7. A method according to claim 6, wherein the first and second feature points are portions of top and bottom edge contours of the determined object. 8. An obstacle detection system for a vehicle, comprising: a camera mounted at the vehicle, the camera capturing a set of image frames while the vehicle is in motion, the camera comprising an imager defining an image plane having a vertical aspect and a horizontal aspect, wherein the imager comprises an array of sensing pixels;a controller coupled to the camera, the controller configured to determine an object in the captured image frames and determine a relevancy of the determined object by (i) selecting first and second feature points on the determined object that are spaced apart vertically in a first captured image frame,(ii) tracking positions of the first and second feature points over at least a second captured image frame, and(iii) determining that the determined object is a relevant object responsive to a determination that the first and second feature points move dissimilar distances in physical space and within a tolerance, and otherwise determining that the determined object is not a relevant object; andthe controller estimating relative motion of determined relevant objects to determine if any relevant object is likely to become an obstacle to the vehicle; andthe controller providing an indication to a driver of the vehicle of determined obstacles;wherein the controller determines the relevancy of the determined object by reference to the image plane and, where the first feature point is vertically lower than the second feature point, the controller:estimates, from the first captured image frame, a physical distance D1 of the first feature point O1b to the camera in a selected physical horizontal plane;estimates, from the first captured image frame, a physical location DT of the second feature point O1T assuming that the second feature point physically lies at a height T above the selected physical horizontal plane;estimates, from the second captured image frame, a physical distance D2 of the first feature point O2b to the camera in the selected physical horizontal plane, and determines the distance ΔD between D2 and D1;estimates a physical location D*T which is equivalent to DT+ΔD, with D*T being located at a height T above the selected physical horizontal plane;computes a nominal feature point coordinate O*2t in the image plane corresponding to D*T; anddetermines the determined object to be the relevant object responsive to a determination that the absolute distance between the nominal feature point coordinate O*2t and the second feature point coordinate in the first captured image frame O1T is less than the absolute distance between the second feature point coordinate in the second captured image frame O2T and the second feature point coordinate in the first captured image frame O1T. 9. An obstacle detection system according to claim 8, wherein the controller determines the relevancy of the determined object by reference to physical space, and wherein the controller is configured to: determine first and second physical locations along a selected physical horizontal plane corresponding to pixel positions of the first and second feature points in the first captured image frame;determine third and fourth physical locations along the selected real horizontal plane corresponding to pixel positions of the first and second feature points in the second captured image frame;determine a first distance Δd1 between the third and first physical locations and determining a second distance Δd2 between the fourth and second physical locations; anddetermine that the determined object is the relevant object if the difference between the first distance Δd1 and the second distance Δd2 exceeds a threshold, and otherwise determining that the determined object is not a relevant object. 10. An obstacle detection system according to claim 9, wherein the controller de-warps a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position. 11. An obstacle detection system according to claim 10, wherein the first and second feature points are portions of top and bottom edge contours of the determined object. 12. An obstacle detection system according to claim 9, wherein the selected physical horizontal plane is a ground plane. 13. An obstacle detection system according to claim 12, wherein the controller de-warps a given pixel position to account for lens distortion prior to determining a physical location corresponding to the given pixel position. 14. An obstacle detection system according to claim 13, wherein the first and second feature points are portions of top and bottom edge contours of the determined object.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (282)
Mazzilli Joseph J., 360 .degree. automobile video camera system.
Mager,Donald, Activating a vehicle's own brake lights and/or brakes when brake lights are sensed in front of the vehicle, including responsively to the proximity of, and/or rate of closure with, a forward vehicle.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Slotkowski Kenneth G. (Westland MI) Michaels Paul A. (Livonia MI) Macks Harold R. (Detroit MI) Ewald Henry J. (Livonia MI), Automatic headlamp dimmer having improved signal discrimination and signal processing.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Blank Rodney K. (Holland MI) Gahan Richard J. (Holland MI) Haselhuhn ; Jr. Howard J. (Holland MI) Schierbeek Kenneth L. (Zeeland MI) Schofield Kenneth (Holland MI), Compensation system for electronic compass.
Leddy Michael (Dallas TX) Boysel Mark (Plano TX) Delong James A. (Shady Shores TX) Florence James M. (Richardson TX) Lin Tsen-Hwang (Dallas TX) Sampsell Jeffrey (Plano TX), Digital micro-mirror based image simulation system.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hardin Larry C. (735 Harrison Bandon OR 97411) Nash Larry V. (31770 Chantrelle La. Gold Beach OR 97444), Electro-optical range finding and speed detection system.
Allemand Pierre M. ; Grimes Randall F. ; Ingle Andrew R. ; Cronin John P. ; Kennedy Steve R. ; Agrawal Anoop ; Boulton Jonathan M., Electrochromic devices.
Lynam Niall R. (Holland MI) O\Farrell Desmond J. (Holland MI) Schierbeek Kenneth L. (Zeeland MI) Hansen Michael A. (West Olive MI), Electrochromic mirror for vehicles.
Brillard Guy (Garges les Gonesse FRX) Brillard Eric (Saint-Maur des Fosses FRX) Crueize Jean-Claude (Moussy-le-Neuf FRX), Electronic anti-collison device carried on board a vehicle.
Petrossian Edward (1140 Elm Ave. ; Apt. 111 Glendale CA 91201) Petrossian Ashkhen A. (1140 Elm Ave. ; Apt. 111 Glendale CA 91201), Enhanced viewing at side and rear of motor vehicles.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Weller, Andrew D.; Larson, Mark L.; McCabe, Ian A.; Lynam, Niall R.; Blank, Rodney K.; McCaw, Joseph P.; Uken, John T., Interior rearview mirror assembly with display.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Wanke, Peter; Herrmann, Torsten; Roszyk, Mario; Wickenhöfer, Thorsten; Pank, Matthias; Konanz, Jochen, Method and system for improving the driving behavior of a vehicle.
Bendicks Norbert (Hemer DEX) Bartling Ralf (Dortmund DEX), Optoelectronic sensor for detecting moisture on a windshield with means to compensate for a metallic layer in the windsh.
Boros Imre,DEX ; Hamann Dieter,DEX ; Maurath Rudolf,DEX ; Pressel Joachim,DEX ; Reiner Michael,DEX, Process for controlling driving dynamics of a street vehicle.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Saneyoshi Keiji (Tokyo JPX) Hanawa Keiji (Tokyo JPX), System for monitoring condition outside vehicle using imaged picture by a plurality of television cameras.
Shaw David C. H. (3312 E. Mandeville Pl. Orange CA 92667) Shaw Judy Z. Z. (3312 E. Mandeville Pl. Orange CA 92667), Vehicle collision avoidance system.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.