Vehicle vision system with calibration algorithm
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-007/18
G06T-007/00
G06T-003/00
G06T-005/00
G06T-007/80
G06K-009/00
출원번호
US-0996571
(2016-01-15)
등록번호
US-9916660
(2018-03-13)
발명자
/ 주소
Singh, Jagmal
Pliefke, Sebastian
Hess, Holger
출원인 / 주소
Magna Electronics Inc.
대리인 / 주소
Gardner, Linn, Burkhart & Flory, LLP
인용정보
피인용 횟수 :
1인용 특허 :
212
초록▼
A vision system of a vehicle includes a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle. The camera may include a fisheye lens. An image processor is operable to process image data captured by the camera. The vision system provides enhanced camera
A vision system of a vehicle includes a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle. The camera may include a fisheye lens. An image processor is operable to process image data captured by the camera. The vision system provides enhanced camera calibration using a monoview noncoplanar three dimensional calibration pattern. The system may include a plurality of cameras configured to be disposed at the vehicle so as to have respective fields of view exterior of the vehicle.
대표청구항▼
1. A vision system of a vehicle, said vision system comprising: a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to pro
1. A vision system of a vehicle, said vision system comprising: a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to process image data captured by said camera;wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern; andwherein values of distortion polynomial coefficients are estimated using three dimensional calibration points distributed on a semi-sphere by formulating the following: λj·pj×pj=pj×[r1r2r3T]·[XjYjZj1]=0=Δ[ujvja0+a2ρj2+…+aNρjN]×[r1r2r3T]·[XjYjZj1]=0wherein l is an arbitrary depth scale factor, pj is a point in the field of view of said camera, r1, r2, r3 are column vectors of the rotation of the ith pattern pose, T is a translation vector, X, Y, Z are world coordinates of a scene point, coefficients a0, a1, a2, . . . aN are distortion parameter coefficients, and coefficient rj is a point at an imaging plane of said camera where the vector of point pj is mapped, and u, v are the pixel coordinates of an image point. 2. The vision system of claim 1, wherein said camera comprises a fish eye lens. 3. The vision system of claim 1, wherein the values of distortion polynomial coefficients are estimated by reducing the formation to obtain: v′j(r31Xj+r32Yj+r33Zj+t3)−(a0+a2ρj2+ . . . +aNρjN)(r21Xj+r22Yj+r23Zj+t2)=0(a0+a2ρj2+ . . . +aNρjN)(r11Xj+r12Yj+r13Zj+t1)−u′j(r31Xj+r32Yj+r33Zj+t3)=0u′j(r21Xj+r22Yj+r23Zj+t2)−v′j(r11Xj+r12Yj+r13Zj+t1)=0wherein rij and ti are unknown parameters. 4. The vision system of claim 3, wherein the values of distortion polynomial coefficients are estimated by solving the equations to obtain a formulation for distortion polynomial coefficients: [(Aj+Cj)(Aj+Cj)·ρj2(Aj+Cj)·ρj3…(Aj+Cj)·ρjN-(vj′+uj′)] [a0a2a3⋮aNt3]=(vj′+uj′)·(r31Xj+r32Yj+r33Zj)wherein Aj is r21Xj+r22Yj+r23Zj+t2 of the equation of claim 3, and Cj is r11Xj+r12Yj+r13Zj+t1 of the equation of claim 3. 5. The vision system of claim 4, wherein, by providing knowledge about t3 will give an unambiguous solution for distortion polynomial, by setting t3 to zero and reducing to following final equation: [(Aj+Cj)(Aj+Cj)·ρj2(Aj+Cj)·ρj3…(Aj+Cj)·ρjN] [a0a2a3⋮aN]=(vj′+uj′)·(r31Xj+r32Yj+r33Zj)+(vj′+uj′)×t3wherein a0, a1, a2, . . . aN are distortion parameter coefficients. 6. A vision system of a vehicle, said vision system comprising: a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to process image data captured by said camera;wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern; andwherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r3 in: λj·pj=λj·[ujvja0+a2ρj2+…+aNρjN]=P·Xj=[r1r2r3T]·[XjYjZj1][r1r2r3T]=[r11r12r13t1r21r22r23t2r31r32r33t3]wherein l is an arbitrary depth scale factor, pj is a point in the field of view of said camera, P is the projection matrix that relates the reference frame of said camera and the world reference frame, r1, r2 and r3 are column vectors of the rotation of the ith pattern pose, T is a translation vector, X, Y, Z are world coordinates of a scene point, coefficients a0, a1, a2, . . . aN are distortion parameter coefficients, and coefficient rj is a point at an imaging plane of said camera where the vector of point pj is mapped, u, v are the pixel coordinates of an image point, and rij and ti are unknown parameters. 7. The vision system of claim 6, comprising a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle. 8. The vision system of claim 7, wherein said plurality of cameras comprises (i) a forward viewing camera disposed at the vehicle so as to have a forward field of view forward of the vehicle, (ii) a rearward viewing camera disposed at the vehicle so as to have a rearward field of view rearward of the vehicle, (iii) a driver side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a driver side of the vehicle, and (iv) a passenger side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a passenger side of the vehicle. 9. A vision system of a vehicle, said vision system comprising: a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle, wherein said camera comprises a fish eye lens;wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to process image data captured by said camera;wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern;wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere; andwherein values of distortion polynomial coefficients are estimated using three dimensional calibration points distributed on a semi-sphere by formulating the following: λj·pj×pj=pj×[r1r2r3T]·[XjYjZj1]=0=Δ[ujuja0+a2ρj2+…+aNρjN]×[r1r2r3T]·[XjYjZj1]=0wherein l is an arbitrary depth scale factor, pj is a point in the field of view of said camera, r1, r2, r3 are column vectors of the rotation of the ith pattern pose, T is a translation vector, X, Y, Z are world coordinates of a scene point, coefficients an are estimated calibration parameters, and coefficient r is a point at an imaging plane of said camera where the vector of point pj is mapped, and u, v are the pixel coordinates of an image point. 10. The vision system of claim 9, wherein the values of distortion polynomial coefficients are estimated by reducing the formation to obtain: v′j(r31Xj+r32Yj+r33Zj+t3)−(a0+a2ρj2+ . . . +aNρjN)(r21Xj+r22Yj+r23Zj+t2)=0(a0+a2ρj2+ . . . +aNρjN)(r11Xj+r12Yj+r13Zj+t1)−u′j(r31Xj+r32Yj+r33Zj+t3)=0u′j(r21Xj+r22Yj+r23Zj+t2)−v′j(r11Xj+r12Yj+r13Zj+t1)=0wherein Aj is r21Xj+r22Yj+r23Zj+t2 of the equation of claim 9, and Cj is r11Xj+r12Yj+r13Zj+t1 of the equation of claim 9. 11. The vision system of claim 10, wherein the values of distortion polynomial coefficients are estimated by and solving the equations to obtain a formulation for distortion polynomial coefficients: [(Aj+Cj)(Aj+Cj)·ρj2(Aj+Cj)·ρj3…(Aj+Cj)·ρjN-(vj′+uj′)] [a0a2a3⋮aNt3]=(vj′+uj′)·(r31Xj+r32Yj+r33Zj). 12. The vision system of claim 11, wherein, by providing knowledge about t3 will give an unambiguous solution for distortion polynomial, by setting t3 to zero and reducing to following final equation: [(Aj+Cj)(Aj+Cj)·ρj2(Aj+Cj)·ρj3…(Aj+Cj)·ρjN] [a0a2a3⋮aN]=(vj′+uj′)·(r31Xj+r32Yj+r33Zj)+(vj′+uj′)×t3wherein a0, a1, a2, . . . aN are distortion parameter coefficients. 13. A vision system of a vehicle, said vision system comprising: a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle, wherein said camera comprises a fish eye lens;wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to process image data captured by said camera;wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern;wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;wherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r3 in: λj·pj=λj·[ujvja0+a2ρj2+…+aNρjN]=P·Xj=[r1r2r3T]·[XjYjZj1][r1r2r3T]=[r11r12r13t1r21r22r23t2r31r32r33t3]wherein l is an arbitrary depth scale factor, pj is a point in the field of view of said camera, P is the projection matrix that relates the reference frame of said camera and the world reference frame, r1, r2 and r3 are column vectors of the rotation of the ith pattern pose, T is a translation vector, X, Y, Z are world coordinates of a scene point, coefficients a0, a1, a2, . . . aN are distortion parameter coefficients, and coefficient rj is a point at an imaging plane of said camera where the vector of point pj is mapped, u, v are the pixel coordinates of an image point, and rij and ti are unknown parameters. 14. A vision system of a vehicle, said vision system comprising: a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle, wherein at least one camera of said plurality of cameras comprises a fish eye lens;wherein each of said cameras comprises a pixelated imaging array having a plurality of photosensing elements;an image processor operable to process image data captured by said at least one camera;wherein said vision system provides camera calibration of said at least one camera using a monoview noncoplanar three dimensional semi-spherical calibration pattern; andwherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r3 in: λj·pj=λj·[ujvja0+a2ρj2+…+aNρjN]=P·Xj=[r1r2r3T]·[XjYjZj1][r1r2r3T]=[r11r12r13t1r21r22r23t2r31r32r33t3]wherein l is an arbitrary depth scale factor, pj is a point in the field of view of said camera, r1, r2 and r3 are column vectors of the rotation of the ith pattern pose, T is a translation vector, X, Y, Z are world coordinates of a scene point, coefficients a0, a1, a2, . . . aN are distortion parameter coefficients, and coefficient rj is a point at an imaging plane of said camera where the vector of point pj is mapped, u, v are the pixel coordinates of an image point, and rij and ti are unknown parameters. 15. The vision system of claim 14, wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere. 16. The vision system of claim 14, wherein said plurality of cameras comprises (i) a forward viewing camera disposed at the vehicle so as to have a forward field of view forward of the vehicle, (ii) a rearward viewing camera disposed at the vehicle so as to have a rearward field of view rearward of the vehicle, (iii) a driver side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a driver side of the vehicle, and (iv) a passenger side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a passenger side of the vehicle, and wherein said image processor is operable to process image data captured by said cameras, and wherein said vision system provides camera calibration of said cameras using said monoview noncoplanar three dimensional semi-spherical calibration pattern.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (212)
Strumolo, Gary Steven, 360° vision system for a vehicle.
Salmeen, Irving Toivo; Miller, Ronald Hugh; DiMeo, David M.; Strumolo, Gary Steven; Dassanayake, Mahendra Somasara; Alles, Sheran Anthony; Wagner, David Anthony, Adaptive vehicle communication controlled lighting system.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Wood Robert B. (Hillsboro OR) Thomas Mark A. (Lake Oswego OR) Desmond John P. (Portland OR), Automobile head-up display system with reflective aspheric surface.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hatae,Yasuhiko; Usui,Shuji; Nakamura,Yoshifumi, Emergency information notifying system, and apparatus, method and moving object utilizing the emergency information notifying system.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Stam, Joseph S.; Mart, Gregory A.; Berends, Keith H.; Bush, Gregory S.; Roberts, John K.; Pierce, Mark W.; Bechtel, Jon H.; Walstra, Eric J.; Rycenga, Brock R., Image acquisition and processing methods for automatic vehicular exterior lighting control.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Yung, Wai Lam; Bittner, Wilfried Alois Anton; Lam, Yat Wah; Chan, Sai Chak; Chan, Wah Pong; Ng, Kwong Lam; Zimmermann, Thomas Paul Richard, Method and apparatus for locating and measuring the distance to a target.
Trissel Richard G. (Cardiff CA) DeFoe Douglas N. (Escondido CA), Optical collimating device employing cholesteric liquid crystal and a non-transmissive reflector.
Bottesch H. Werner (R.D. #6 ; Box 374 Danville PA 17821) Freas David A. (P.O. Box 324 Mifflinville PA 18631), Passive vehicle presence detection system.
Bamji,Cyrus, RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Stam, Joseph S.; Bechtel, Jon H.; Reese, Spencer D.; Roberts, John K.; Tonar, William L.; Poe, G. Bruce; Newhouse, Douglas J., System for controlling exterior vehicle lights.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.