Method and system for dynamically calibrating vehicular cameras
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-007/18
H04N-017/00
G06T-007/00
출원번호
US-0113414
(2012-01-20)
등록번호
US-9357208
(2016-05-31)
국제출원번호
PCT/CA2012/000056
(2012-01-20)
§371/§102 date
20131023
(20131023)
국제공개번호
WO2012/145818
(2012-11-01)
발명자
/ 주소
Gupta, Nikhil
Faraji, Hilda
He, Daan
Rathi, Ghanshyam
출원인 / 주소
MAGNA ELECTRONICS INC.
대리인 / 주소
Gardner, Linn, Burkhart & Flory, LLP
인용정보
피인용 횟수 :
5인용 특허 :
214
초록▼
A method of dynamically ascertaining the alignment of a vehicular camera. The method involves acquiring a sequence of images provided by the camera whilst the vehicle is in motion. For each range of steering angles, the method (i) selects a plurality of feature points in the images, (ii) tracks a mo
A method of dynamically ascertaining the alignment of a vehicular camera. The method involves acquiring a sequence of images provided by the camera whilst the vehicle is in motion. For each range of steering angles, the method (i) selects a plurality of feature points in the images, (ii) tracks a motion trajectory for each selected feature point, and (iii) determines a vanishing point in the image plane based on the tracked motion trajectories. The method determines a vanishing line in the image plane based on a locus of these vanishing points and determines the alignment of the camera based on the position of a central vanishing point (corresponding to the zero degree angle) and the vanishing line.
대표청구항▼
1. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising: (a) establishing a plurality of vehicular steering angle ranges;(b) acquiring a set of image frames in a video stream provided by the camera whils
1. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising: (a) establishing a plurality of vehicular steering angle ranges;(b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane;(c) measuring the steering angle of the vehicle and, for each steering angle range: (i) selecting a plurality of feature points in the image frames;(ii) tracking a motion trajectory of each selected feature point in the set of image frames, and(iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories;(d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points;(e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees;removing distortion in the motion trajectories caused by use of a wide angle camera lens;linearly fitting the undistorted motion trajectories to generate straight lines;wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;wherein the camera is a front facing camera and rotational angles α and β of the camera are determined as a function of d1 and d2, where:α is the rotation of the camera around an X-axis in a camera coordinate system,β is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, andd2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; andwherein a rotational angle γ of the camera is determined by solving for γ as follows: A cos γ+B sin γ=C where,A−f sin α sin βB=f cos αC−(cvpX−axisX) sin α cos βwhere γ is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point. 2. A method according to claim 1, wherein α and β are determined from a table correlating unique values of d1 and d2 with unique values of α and β, said table created by setting the α and β positions of the front camera to known quantities and recording the resultant d1 and d2 values. 3. A method according to claim 1, wherein rotations Rx, Ry and Rz around X-, Y- and Z-axes, respectively, of the vehicle coordinate system are determined based on the rotations α, β and y around the X-, Y- and Z- axes of the camera coordinate system. 4. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising: (a) establishing a plurality of vehicular steering angle ranges;(b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane;(c) measuring the steering angle of the vehicle and, for each steering angle range: (i) selecting a plurality of feature points in the image frames;(ii) tracking a motion trajectory of each selected feature point in the set of image frames, and(iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories;(d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and(e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees;removing distortion in the motion trajectories caused by use of a wide angle camera lens;linearly fitting the undistorted motion trajectories to generate straight lines;wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;wherein the camera is a side facing camera and rotational angles α and β of the side facing camera are determined as a function of d1 and d2, whereα is a rotation of the side facing camera around an X-axis of a side facing camera coordinate system,β is a rotation of the side facing camera around a Y-axis of the side camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, andd2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point;wherein a rotational angle v is determined by measuring an angle projected by the central vanishing point on a horizontal axis of the image plane; andwherein rotations Rx, Ry and Rz, for the side camera around the X-, Y- and Z-axes, respectively, of the vehicle coordinate system are determined from the rotations α, β and γ around the X-, Y- and Z-axes of the camera coordinate system. 5. A method according to claim 4, wherein α and β are determined from a table correlating unique values of d1 and d2 with unique values of Rx and Rz, said table created by setting the α and β positions of the side camera to known quantities and recording the resultant d1 and d2 values. 6. A system for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the system comprising a controller which: (a) establishes a plurality of vehicular steering angle ranges;(b) acquires a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, wherein the image frames define an image plane;(c) measures the steering angle of the vehicle and, for each steering angle range: (i) selects a plurality of feature points in the image frames;(ii) tracks a motion trajectory of each selected feature point in the set of image frames, and(iii) determines a vanishing point in the image plane for the plurality of tracked motion trajectories;(d) determines a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and(e) determines the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, wherein the central vanishing point is the vanishing point that is determined for the steering angle range that encompasses zero degrees;wherein the controller removes distortion in the motion trajectories caused by use of a wide angle camera lens, linearly fits the undistorted motion trajectories to generate straight lines, and determines the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;wherein the camera is a front facing camera and rotational angles α and β of the camera are determined as a function of d1 and d2, whereα is the rotation of the camera around an X-axis in a camera coordinate system,β is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, andd2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point;wherein α and β are determined from a table correlating unique values of d1 and d2 with unique values of α and β, said table created by setting the α and β positions of the front camera to known quantities and recording the resultant d1 and d2 values; andwherein the controller determines a rotational angle γ of the camera by solving for γ as follows: A cos γ+B sin γ=C where,A−f sin α sin βB=f cos αC−(cvpX−axisX) sin α cos βwhere γ is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point. 7. A system for dynamically ascertaining the position of a vehicular camera relative to a vehicle, the system comprising: a camera disposed at a front portion of a vehicle equipped with the system, the camera having a forward field of view, wherein the camera is operable to capture image data;a controller, wherein the controller is operable to establish a plurality of vehicular steering angle ranges of the equipped vehicle;wherein the controller is operable to receive image data captured by the camera and to acquire a set of image frames in a video stream of image data captured by the camera while the equipped vehicle is in motion, and wherein the image frames define an image plane;wherein the controller is operable to measure the steering angle of the equipped vehicle and, for each steering angle range, (i) select a plurality of feature points in the image frames, (ii) track a motion trajectory of each selected feature point in the set of image frames, and (iii) determine a vanishing point in the image plane for the plurality of tracked motion trajectories;wherein the controller determines a vanishing line in the image plane provided by the camera based on a locus of said vanishing points;wherein the controller determines the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, wherein the central vanishing point is the vanishing point that is determined for the steering angle range that encompasses zero degrees; andwherein rotational angles α and β of the camera are determined as a function of d1 and d2, where: α is the rotation of the camera around an X-axis in a camera coordinate system,β is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, andd2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; andwherein the controller determines a rotational angle y of the camera by solving for y as follows: A cos γ+B sin γ=C where,A−f sin α sin βB=f cos αC−(cvpX−axisX) sin α cos βwhere γ is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point. 8. A system according to claim 7, wherein α and β are determined from a table correlating unique values of d1 and d2 with unique values of α and β, said table created by setting the α and β positions of the front camera to known quantities and recording the resultant d1 and d2 values.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (214)
Strumolo, Gary Steven, 360° vision system for a vehicle.
Salmeen, Irving Toivo; Miller, Ronald Hugh; DiMeo, David M.; Strumolo, Gary Steven; Dassanayake, Mahendra Somasara; Alles, Sheran Anthony; Wagner, David Anthony, Adaptive vehicle communication controlled lighting system.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Wood Robert B. (Hillsboro OR) Thomas Mark A. (Lake Oswego OR) Desmond John P. (Portland OR), Automobile head-up display system with reflective aspheric surface.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hatae,Yasuhiko; Usui,Shuji; Nakamura,Yoshifumi, Emergency information notifying system, and apparatus, method and moving object utilizing the emergency information notifying system.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Stam, Joseph S.; Mart, Gregory A.; Berends, Keith H.; Bush, Gregory S.; Roberts, John K.; Pierce, Mark W.; Bechtel, Jon H.; Walstra, Eric J.; Rycenga, Brock R., Image acquisition and processing methods for automatic vehicular exterior lighting control.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Yung, Wai Lam; Bittner, Wilfried Alois Anton; Lam, Yat Wah; Chan, Sai Chak; Chan, Wah Pong; Ng, Kwong Lam; Zimmermann, Thomas Paul Richard, Method and apparatus for locating and measuring the distance to a target.
Trissel Richard G. (Cardiff CA) DeFoe Douglas N. (Escondido CA), Optical collimating device employing cholesteric liquid crystal and a non-transmissive reflector.
Bottesch H. Werner (R.D. #6 ; Box 374 Danville PA 17821) Freas David A. (P.O. Box 324 Mifflinville PA 18631), Passive vehicle presence detection system.
Bamji,Cyrus, RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Stam, Joseph S.; Bechtel, Jon H.; Reese, Spencer D.; Roberts, John K.; Tonar, William L.; Poe, G. Bruce; Newhouse, Douglas J., System for controlling exterior vehicle lights.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
Gulati, Rahul; Jong, Alex Kwang Ho; Wong, John Chi Kit; Gupta, Sanjay; Ikizyan, Ike, Data content integrity in display subsystem for safety critical use cases.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.