Method for dynamically calibrating vehicular cameras
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-007/80
B60R-001/06
B60R-001/00
H04N-017/00
출원번호
US-0161711
(2016-05-23)
등록번호
US-10202077
(2019-02-12)
발명자
/ 주소
Gupta, Nikhil
Faraji, Hilda
He, Daan
Rathi, Ghanshyam
출원인 / 주소
MAGNA ELECTRONICS INC.
대리인 / 주소
Honigman Miller Schwartz and Cohn, LLP
인용정보
피인용 횟수 :
0인용 특허 :
216
초록▼
A method for dynamically correcting misalignment of a vehicular camera includes fixedly disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data while the vehicle is moving generally in a straight line. A plurality of sets of feature points are selected in an
A method for dynamically correcting misalignment of a vehicular camera includes fixedly disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data while the vehicle is moving generally in a straight line. A plurality of sets of feature points are selected in an image frame, with each set including a first feature point and a second feature point. For each set of feature points, a motion trajectory of that set's feature points is tracked in subsequent image frames. For each tracked first and second feature points, a vanishing point is established in the image plane. Based on the established vanishing point, a vanishing line is determined in the image plane. When the vanishing line is determined to be non-horizontal in the image plane, at least one of pitch, roll or yaw of the camera is adjusted to correct rotational misalignment of the camera.
대표청구항▼
1. A method for dynamically correcting rotational misalignment of a vehicular camera, the method comprising: (a) fixedly disposing a camera at a vehicle, said camera having a field of view exterior of the equipped vehicle;(b) driving the equipped vehicle so that the equipped vehicle is in motion;(c)
1. A method for dynamically correcting rotational misalignment of a vehicular camera, the method comprising: (a) fixedly disposing a camera at a vehicle, said camera having a field of view exterior of the equipped vehicle;(b) driving the equipped vehicle so that the equipped vehicle is in motion;(c) operating said camera to acquire multiple frames of image data captured by said camera (i) while the equipped vehicle is moving in a straight line and (ii) while the equipped vehicle is moving along a curve;(d) selecting a plurality of sets of feature points in an image frame of image data captured by said camera, wherein a first set of feature points comprises a first feature point and a second feature point;(e) for the first set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's first feature point and second feature point while the equipped vehicle is moving in the straight line;(f) for each tracked first feature point and second feature point while the equipped vehicle is moving in the straight line, establishing a first vanishing point in the image plane;(g) wherein a second set of feature points comprises a third feature point and a fourth feature point;(h) for the second set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's third feature point and fourth feature point while the equipped vehicle is moving along the curve;(i) for each tracked third feature point and fourth feature point while the equipped vehicle is moving along the curve, establishing a second vanishing point in the image plane;(j) based on establishing first and second vanishing points in the image plane by tracking respective feature points for a plurality of sets of feature points while the equipped vehicle is moving in the straight line and while the equipped vehicle is moving along the curve, determining a vanishing line in the image plane provided by said camera based on a locus of the first and second vanishing points;(k) determining whether the vanishing line is horizontal or non-horizontal in the image plane; and(l) when the vanishing line is determined to be non-horizontal in the image plane, adjusting at least one of pitch, roll or yaw of said camera to correct rotational misalignment of said camera. 2. A method according to claim 1, wherein, when the equipped vehicle turns from travelling in the straight line as a result of a change in its steering angle, motion of the vehicle is approximated over relatively short distances as straight line motion. 3. A method according to claim 2, wherein approximation to straight line motion when the vehicle turns is vehicle speed dependent. 4. A method according to claim 3, wherein approximation to straight line motion when the vehicle turns occurs for 0.5 to 2 seconds of travel time after turning from straight line travel. 5. A method according to claim 1, wherein said camera comprises a wide angle lens and wherein said method comprises removing distortion caused by use of the wide angle lens. 6. A method according to claim 1, wherein said camera is part of a multi-camera vision system of the equipped vehicle. 7. A method according to claim 6, wherein said camera is part of a multi-camera vision system of the equipped vehicle, and wherein said multi-camera vision system of the equipped vehicle comprises said camera and three other vehicular cameras. 8. A method according to claim 7, wherein image data captured by at least two of the four cameras of said multi-camera vision system is stitched together to provide a composite image for display to a driver of the equipped vehicle. 9. A method according to claim 1, wherein a control receives and processes image data captured by said camera, said control receiving vehicle data via a communication bus of the equipped vehicle. 10. A method according to claim 9, wherein the received vehicle data includes vehicle speed data and vehicle steering angle data. 11. A method according to claim 10, wherein the communication bus of the equipped vehicle comprised a CAN bus. 12. A method according to claim 11, wherein said camera captures at least 25 frames of image data per second. 13. A method according to claim 12, wherein image data captured by said camera is provided to a digital signal processor that, at least in part, processes received image data to correct rotational misalignment of said camera. 14. A method for dynamically correcting rotational misalignment of a vehicular camera, the method comprising: (a) fixedly disposing a camera at a portion of a vehicle, said camera having a field of view exterior of the equipped vehicle and wherein said camera comprises a wide angle lens;(b) driving the equipped vehicle so that the equipped vehicle is in motion;(c) operating said camera to acquire multiple frames of image data captured by said camera (i) while the equipped vehicle is in motion moving in a straight line and (ii) while the equipped vehicle is moving along a curve;(d) removing distortion caused by use of the wide angle lens;(e) selecting a plurality of sets of feature points in an image frame of image data captured by said camera, wherein a first set of feature points comprises a first feature point and a second feature point;(f) for the first set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's first feature point and second feature point while the equipped vehicle is moving in the straight line;(g) for each tracked first feature point and second feature point while the equipped vehicle is moving in the straight line, establishing a first vanishing point in the image plane;(h) wherein a second set of feature points comprises a third feature point and a fourth feature point;(i) for the second set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's third feature point and fourth feature point while the equipped vehicle is moving along the curve;(j) for each tracked third feature point and fourth feature point while the equipped vehicle is moving along the curve, establishing a second vanishing point in the image plane;(k) based on establishing first and second vanishing points in the image plane by tracking respective feature points for a plurality of sets of feature points while the equipped vehicle is moving in the straight line and while the equipped vehicle is moving along the curve, determining a vanishing line in the image plane provided by said camera based on a locus of the first and second vanishing points;(l) determining whether the vanishing line is horizontal or non-horizontal in the image plane; and(m) when the vanishing line is determined to be non-horizontal in the image plane, adjusting at least one of pitch, roll or yaw of said camera to correct rotational misalignment of said camera. 15. A method according to claim 14, wherein said camera is part of a multi-camera vision system of the equipped vehicle, and wherein said multi-camera vision system of the equipped vehicle comprises said camera and three other vehicular cameras, and wherein image data captured by at least two of the four cameras of said multi-camera vision system is stitched together to provide a composite image for display to a driver of the equipped vehicle. 16. A method according to claim 15, wherein image data captured by the four cameras of said multi-camera vision system is stitched together to provide a composite 360 degree image for display to a driver of the equipped vehicle. 17. A method according to claim 16, wherein a control receives and process image data captured by said camera, said control receiving vehicle data via a communication bus of the equipped vehicle. 18. A method for dynamically correcting rotational misalignment of a vehicular camera, the method comprising: (a) fixedly disposing a camera at a portion of a vehicle, said camera having a field of view exterior of the equipped vehicle and wherein said camera comprises a wide angle lens;(b) driving the equipped vehicle so that the equipped vehicle is in motion;(c) operating said camera to acquire multiple frames of image data captured by said camera (i) while the equipped vehicle is moving in a straight line and (ii) while the equipped vehicle is moving along a curve;(d) removing distortion caused by use of the wide angle lens;(e) receiving vehicle data via a communication bus of the equipped vehicle, the received vehicle data including speed data and steering angle data of the equipped vehicle;(f) selecting a plurality of sets of feature points in an image frame of image data captured by said camera, wherein a first set of feature points comprises a first feature point and a second feature point;(g) for the first set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's first feature point and second feature point while the equipped vehicle is moving in the straight line;(h) for each tracked first feature point and second feature point while the equipped vehicle is moving in the straight line, establishing a first vanishing point in the image plane;(i) wherein a second set of feature points comprises a third feature point and a fourth feature point;(j) for the second set of feature points, tracking in subsequent image frames of image data captured by said camera a motion trajectory of that set's third feature point and fourth feature point while the equipped vehicle is moving along the curve;(k) for each tracked third feature point and fourth feature point while the equipped vehicle is moving along the curve, establishing a second vanishing point in the image plane;(l) based on establishing first and second vanishing points in the image plane by tracking respective feature points for a plurality of sets of feature points while the equipped vehicle is moving in the straight line and while the equipped vehicle is moving along the curve, determining a vanishing line in the image plane provided by said camera based on a locus of the first and second vanishing points;(m) determining whether the vanishing line is horizontal or non-horizontal in the image plane; and(n) when the vanishing line is determined to be non-horizontal in the image plane, adjusting at least one of pitch, roll or yaw of said camera to correct rotational misalignment of said camera. 19. A method according to claim 18, wherein said camera is part of a multi-camera vision system of the equipped vehicle, and wherein said multi-camera vision system of the equipped vehicle comprises said camera and three other vehicular cameras, and wherein image data captured by at least two of the four cameras of said multi-camera vision system is stitched together to provide a composite image for display to a driver of the equipped vehicle. 20. A method according to claim 19, wherein, when the equipped vehicle turns from travelling in the straight line as a result of a change in its steering angle, motion of the vehicle is approximated over relatively short distances as straight line motion, and wherein approximation to straight line motion when the vehicle turns is vehicle speed dependent.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (216)
Strumolo, Gary Steven, 360° vision system for a vehicle.
Salmeen, Irving Toivo; Miller, Ronald Hugh; DiMeo, David M.; Strumolo, Gary Steven; Dassanayake, Mahendra Somasara; Alles, Sheran Anthony; Wagner, David Anthony, Adaptive vehicle communication controlled lighting system.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Wood Robert B. (Hillsboro OR) Thomas Mark A. (Lake Oswego OR) Desmond John P. (Portland OR), Automobile head-up display system with reflective aspheric surface.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hatae,Yasuhiko; Usui,Shuji; Nakamura,Yoshifumi, Emergency information notifying system, and apparatus, method and moving object utilizing the emergency information notifying system.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Stam, Joseph S.; Mart, Gregory A.; Berends, Keith H.; Bush, Gregory S.; Roberts, John K.; Pierce, Mark W.; Bechtel, Jon H.; Walstra, Eric J.; Rycenga, Brock R., Image acquisition and processing methods for automatic vehicular exterior lighting control.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Yung, Wai Lam; Bittner, Wilfried Alois Anton; Lam, Yat Wah; Chan, Sai Chak; Chan, Wah Pong; Ng, Kwong Lam; Zimmermann, Thomas Paul Richard, Method and apparatus for locating and measuring the distance to a target.
Trissel Richard G. (Cardiff CA) DeFoe Douglas N. (Escondido CA), Optical collimating device employing cholesteric liquid crystal and a non-transmissive reflector.
Bottesch H. Werner (R.D. #6 ; Box 374 Danville PA 17821) Freas David A. (P.O. Box 324 Mifflinville PA 18631), Passive vehicle presence detection system.
Bamji,Cyrus, RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Stam, Joseph S.; Bechtel, Jon H.; Reese, Spencer D.; Roberts, John K.; Tonar, William L.; Poe, G. Bruce; Newhouse, Douglas J., System for controlling exterior vehicle lights.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.