Vehicle vision system with targetless camera calibration
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-007/00
B60R-001/00
출원번호
US-0282028
(2014-05-20)
등록번호
US-9563951
(2017-02-07)
발명자
/ 주소
Okouneva, Galina
출원인 / 주소
MAGNA ELECTRONICS INC.
대리인 / 주소
Gardner, Linn, Burkhart & Flory, LLP
인용정보
피인용 횟수 :
1인용 특허 :
192
초록▼
A camera calibration system of a vehicle includes a camera disposed at a vehicle and having a field of view exterior of the vehicle. The camera is operable to capture image data. An image processor operable to process image data captured by the camera. The camera calibration system is operable to ge
A camera calibration system of a vehicle includes a camera disposed at a vehicle and having a field of view exterior of the vehicle. The camera is operable to capture image data. An image processor operable to process image data captured by the camera. The camera calibration system is operable to generate camera calibration parameters utilizing a bundle adjustment algorithm. Responsive to image processing of captured image data during movement of the vehicle along an arbitrary path, and responsive to the bundle adjustment algorithm, the camera calibration system is operable to calibrate the camera. The bundle adjustment algorithm may iteratively refine calibration parameters starting from a known initial estimation.
대표청구항▼
1. A camera calibration system of a vehicle, said camera calibration system comprising: a camera disposed at a vehicle and having a field of view exterior of the vehicle;wherein said camera comprises a two dimensional array having a plurality of photosensing elements;wherein said camera is operable
1. A camera calibration system of a vehicle, said camera calibration system comprising: a camera disposed at a vehicle and having a field of view exterior of the vehicle;wherein said camera comprises a two dimensional array having a plurality of photosensing elements;wherein said camera is operable to capture image data;an image processor operable to process image data captured by said camera;wherein said camera calibration system is operable to generate camera calibration parameters utilizing a bundle adjustment algorithm;wherein said bundle adjustment algorithm uses a kinematic model of vehicle motion derived from image processing by said image processor of multiple frames of captured image data;wherein said bundle adjustment algorithm generates calibration parameters utilizing the equation: Ximage=P*Xw,where Ximage is a selection of image points in captured image data, P is a projective matrix representing motion between frames of captured image data, and Xw is a vector of world coordinates of image points in captured image data;wherein, responsive to image processing by said image processor of multiple frames of captured image data, said camera calibration system extracts and matches features determined in multiple frames of captured image data;wherein, responsive to image processing by said image processor of captured image data during movement of the vehicle along an arbitrary path, and responsive to generation of camera calibration parameters utilizing said bundle adjustment algorithm, said camera calibration system is operable to calibrate said camera; andwherein said camera calibration system calibrates said camera as the vehicle navigates the arbitrary path, and wherein calibration of said camera includes correction for misalignment in yaw, pitch and roll angles of said camera. 2. The camera calibration system of claim 1, wherein said bundle adjustment algorithm iteratively refines calibration parameters starting from a known initial estimation. 3. The camera calibration system of claim 1, wherein said bundle adjustment algorithm solves for calibration parameters and actual coordinates of an object present in the field of view of said camera. 4. The camera calibration system of claim 1, wherein said image processor is operable to calibrate said camera without image processing of image data representative of captured images of a target. 5. The camera calibration system of claim 1, comprising a plurality of cameras disposed at the vehicle and having respective fields of view exterior of the vehicle, wherein said image processor is operable to process image data captured by said cameras, and wherein said image processor is operable to calibrate said cameras without use of targets in the field of view of said cameras and without image processing of image data representative of captured images of targets. 6. The camera calibration system of claim 1, wherein the generated camera calibration parameters comprise at least one of (i) at least one of translational components X, Y, Z and (ii) at least one of rotational components Yaw, Pitch and Roll. 7. The camera calibration system of claim 1, wherein the generated camera calibration parameters comprise (i) translational components X, Y, Z and (ii) rotational components Yaw, Pitch and Roll angles. 8. The camera calibration system of claim 1, wherein said camera calibration system comprises a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle. 9. The camera calibration system of claim 8, wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle. 10. The camera calibration system of claim 9, wherein one of said cameras comprises a rear camera disposed at a rear portion of the vehicle, and another of said cameras comprises a side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and another of said cameras comprises a passenger side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle. 11. A camera calibration system of a vehicle, said camera calibration system comprising: a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver-side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger-side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view;wherein said cameras are operable to capture image data;an image processor operable to process image data captured by said cameras;wherein said camera calibration system is operable to generate camera calibration parameters for each of said cameras utilizing a bundle adjustment algorithm;wherein said bundle adjustment algorithm uses a kinematic model of vehicle motion derived from image processing by said image processor of multiple frames of image data captured by each of said cameras;wherein said bundle adjustment algorithm generates calibration parameters utilizing the equation: Ximage=P*Xw,where Ximage is a selection of image points in captured image data, P is a projective matrix representing motion between frames of captured image data, and Xw is a vector of world coordinates of image points in captured image data;wherein, responsive to image processing by said image processor of multiple frames of image data captured by each of said cameras, said camera calibration system extracts and matches features determined in multiple frames of image data captured by each of said cameras;wherein, responsive to image processing by said image processor of captured image data during movement of the vehicle along an arbitrary path, and responsive to generation of camera calibration parameters utilizing said bundle adjustment algorithm, said camera calibration system is operable to calibrate each of said cameras; andwherein said camera calibration system calibrates each of said cameras as the vehicle navigates the arbitrary path, and wherein calibration of each of said cameras includes correction for misalignment in yaw, pitch and roll angles of each of said cameras. 12. The camera calibration system of claim 11, wherein said bundle adjustment algorithm iteratively refines calibration parameters starting from a known initial estimation. 13. The camera calibration system of claim 11, wherein said bundle adjustment algorithm solves for calibration parameters and actual coordinates of an object present in the field of view of said camera. 14. The camera calibration system of claim 11, wherein the generated camera calibration parameters comprise at least one of (i) at least one of translational components X, Y, Z and (ii) at least one of rotational components Yaw, Pitch and Roll. 15. The camera calibration system of claim 11, wherein the generated camera calibration parameters comprise (i) translational components X, Y, Z and (ii) rotational components Yaw, Pitch and Roll. 16. The camera calibration system of claim 11, wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements. 17. The camera calibration system of claim 11, wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle. 18. A camera calibration system of a vehicle, said camera calibration system comprising: a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle;wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements;wherein said plurality of cameras comprises a rear camera disposed at a rear portion of the vehicle, a driver side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and a passenger side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle;wherein each of said cameras is operable to capture image data;an image processor operable to process image data captured by said cameras;wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle;wherein said camera calibration system is operable to generate camera calibration parameters utilizing a bundle adjustment algorithm;wherein said bundle adjustment algorithm uses a kinematic model of vehicle motion derived from image processing by said image processor of multiple frames of image data captured by each of said cameras;wherein said bundle adjustment algorithm generates calibration parameters utilizing the equation: Ximage=P*Xw,where Ximage is a selection of image points in captured image data, P is a protective matrix representing motion between frames of captured image data, and Xw is a vector of world coordinates of image points in captured image data;wherein, responsive to image processing by said image processor of multiple frames of image data captured by each of said cameras, said camera calibration system extracts and matches features determined in multiple frames of image data captured by each of said cameras;wherein the generated camera calibration parameters comprise (i) at least one of translational components X, Y, Z and (ii) at least one of rotational components Yaw, Pitch and Roll;wherein, responsive to image processing by said image processor of captured image data during movement of the vehicle along an arbitrary path, and responsive to generation of camera calibration parameters utilizing said bundle adjustment algorithm, said camera calibration system is operable to calibrate at least one of said cameras; andwherein said camera calibration system calibrates each of said cameras as the vehicle navigates the arbitrary path, and wherein calibration of each of said cameras includes correction for misalignment in yaw, pitch and roll angles of each of said cameras. 19. The camera calibration system of claim 18, wherein said bundle adjustment algorithm iteratively refines calibration parameters starting from a known initial estimation. 20. The camera calibration system of claim 19, wherein said bundle adjustment algorithm solves for calibration parameters and actual coordinates of an object present in the field of view of said at least one of said cameras.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (192)
Strumolo, Gary Steven, 360° vision system for a vehicle.
Salmeen, Irving Toivo; Miller, Ronald Hugh; DiMeo, David M.; Strumolo, Gary Steven; Dassanayake, Mahendra Somasara; Alles, Sheran Anthony; Wagner, David Anthony, Adaptive vehicle communication controlled lighting system.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Wood Robert B. (Hillsboro OR) Thomas Mark A. (Lake Oswego OR) Desmond John P. (Portland OR), Automobile head-up display system with reflective aspheric surface.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hatae,Yasuhiko; Usui,Shuji; Nakamura,Yoshifumi, Emergency information notifying system, and apparatus, method and moving object utilizing the emergency information notifying system.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Stam, Joseph S.; Mart, Gregory A.; Berends, Keith H.; Bush, Gregory S.; Roberts, John K.; Pierce, Mark W.; Bechtel, Jon H.; Walstra, Eric J.; Rycenga, Brock R., Image acquisition and processing methods for automatic vehicular exterior lighting control.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Yung, Wai Lam; Bittner, Wilfried Alois Anton; Lam, Yat Wah; Chan, Sai Chak; Chan, Wah Pong; Ng, Kwong Lam; Zimmermann, Thomas Paul Richard, Method and apparatus for locating and measuring the distance to a target.
Trissel Richard G. (Cardiff CA) DeFoe Douglas N. (Escondido CA), Optical collimating device employing cholesteric liquid crystal and a non-transmissive reflector.
Bottesch H. Werner (R.D. #6 ; Box 374 Danville PA 17821) Freas David A. (P.O. Box 324 Mifflinville PA 18631), Passive vehicle presence detection system.
Bamji,Cyrus, RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Stam, Joseph S.; Bechtel, Jon H.; Reese, Spencer D.; Roberts, John K.; Tonar, William L.; Poe, G. Bruce; Newhouse, Douglas J., System for controlling exterior vehicle lights.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.