[미국특허]
Multi-camera image stitching calibration system
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-007/80
B60R-001/00
G06T-003/40
G06T-007/33
H04N-007/18
출원번호
US-0046174
(2013-10-04)
등록번호
US-9723272
(2017-08-01)
발명자
/ 주소
Lu, Yuesheng
Byrne, Steven V.
출원인 / 주소
MAGNA ELECTRONICS INC.
대리인 / 주소
Gardner, Linn, Burkhart & Flory, LLP
인용정보
피인용 횟수 :
0인용 특허 :
214
초록▼
A vision system of a vehicle includes a plurality of cameras with exterior overlapping fields of view. The vision system is operable to calibrate at least one of the cameras and includes a plurality of targets disposed at locations near the vehicle and within the fields of views of the cameras. A di
A vision system of a vehicle includes a plurality of cameras with exterior overlapping fields of view. The vision system is operable to calibrate at least one of the cameras and includes a plurality of targets disposed at locations near the vehicle and within the fields of views of the cameras. A display device is operable to display images captured by the cameras of the vision system. A plurality of user inputs may be provided and a user may selectively actuate the user inputs to manipulate images captured by the cameras to align portions of a target in overlapping regions of the captured images of adjacent cameras to calibrate the cameras. The vision system may calibrate cameras automatically, such as responsive to the targets moving into the fields of views of the cameras, whereby the vision system may calibrate the cameras as the vehicle is moved along an assembly line.
대표청구항▼
1. A vision system of a vehicle, said vision system comprising: a plurality of cameras having fields of view exterior a vehicle equipped with the vision system, wherein said plurality of cameras comprises a forward camera having a forward field of view, a rearward camera having a rearward facing vie
1. A vision system of a vehicle, said vision system comprising: a plurality of cameras having fields of view exterior a vehicle equipped with the vision system, wherein said plurality of cameras comprises a forward camera having a forward field of view, a rearward camera having a rearward facing view, a left side camera having a left sideward field of view at the left side of the equipped vehicle and a right side camera having a right sideward field of view at the right side of the equipped vehicle;wherein a portion of said forward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view and wherein a portion of said rearward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view;a display device operable to display images captured by said cameras of said vision system;a plurality of user inputs actuatable by a user to manipulate the displayed images;wherein, during manual camera calibration, a plurality of targets are disposed at ground locations near the vehicle and within the overlapping portions of the fields of views of said cameras of said vision system;wherein, when the displayed image of one of said cameras is misaligned with a displayed image of another of said cameras, a user actuates said user inputs to manipulate the displayed image of said one of said cameras to align portions of at least one of the targets in an overlapping region of the displayed images of said one of said cameras and said another of said cameras to correct for misalignment of said one of said cameras;wherein, during manual camera calibration, the displayed image captured by a first calibrated camera of said plurality of cameras is manipulated so that lines are displayed as straight and rectangles are displayed in their right shapes and scales; andwherein, after the displayed image captured by the first calibrated camera is manipulated to reach a satisfactory state, the first calibrated camera is used as a calibration reference for a second calibrated camera, and wherein the displayed image captured by the second calibrated camera is manipulated so that lines are displayed as straight and rectangles are displayed in their right shapes and scales and so that at the border of the displayed images captured by the first and second calibrated cameras, the patterns transition smoothly and continuously across the border or overlapping areas of the displayed images. 2. The vision system of claim 1, wherein said user inputs are part of or associated with said display device. 3. The vision system of claim 1, wherein said user inputs are part of another system of the vehicle. 4. The vision system of claim 1, wherein said display device comprises a remote display device that is remote from the vehicle. 5. The vision system of claim 1, wherein said user inputs are actuated to manipulate the displayed images by stretching or compressing or rotating or shifting the displayed images to align portions of the captured image with at least one of said targets. 6. The vision system of claim 1, wherein said user inputs are actuated to manipulate the displayed images by at least one of (i) stretching the displayed images, (ii) compressing the displayed images, (iii) rotating the displayed images and (iv) shifting the displayed images. 7. A vision system of a vehicle, said vision system comprising: a plurality of cameras having fields of view exterior a vehicle equipped with the vision system, wherein said plurality of cameras comprises a forward camera having a forward field of view, a rearward camera having a rearward facing view, a left side camera having a left sideward field of view at the left side of the equipped vehicle and a right side camera having a right sideward field of view at the right side of the equipped vehicle;wherein each of said cameras comprises a lens and an imager;wherein a portion of said forward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view and wherein a portion of said rearward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view;a display device operable to display images captured by said cameras of said vision system;wherein said vision system is operable to calibrate said cameras to correct for mis-alignment of at least one of said cameras;wherein, during camera calibration, a plurality of targets are disposed at ground locations near the vehicle and within the overlapping portions of the fields of views of said cameras;wherein the plurality of targets comprises at least two longitudinal lines extending longitudinally along the direction that the vehicle is moving at either side of the vehicle, with a plurality of tick marks extending laterally from the longitudinal lines;wherein, during camera calibration, said vision system is operable to (i) utilize a math model to at least in part correct for lens distortion based on lens nominal design data, (ii) utilize a lens-to-sensor misalignment assessment and correction by determining a relationship between the degree of curvature of horizontal lines and vertical misalignment of said lens to a center of said imager for each of said cameras, (iii) utilize a perspective correction to at least in part correct for remaining rotation and translation errors and (iv) utilize a fine tuning of smaller mis-matches to achieve smooth stitching for the displayed image;wherein camera calibration is performed while the vehicle is moving; andwherein camera calibration is triggered by a target moving into a predetermined position in the camera views while the vehicle is moving. 8. The vision system of claim 7, wherein the perspective correction utilizes a math model of homographic transformation represented by a 3×3 matrix. 9. The vision system of claim 7, wherein a degree of curvature of horizontal lines can be measured by detecting edges and points of the target patterns. 10. The vision system of claim 7, wherein said fine tuning of smaller mis-matches is achieved by stretching or moving pixel data in overlapping regions of adjacent camera images. 11. A vision system of a vehicle, said vision system comprising: a plurality of cameras having fields of view exterior a vehicle equipped with the vision system, wherein said plurality of cameras comprises a forward camera having a forward field of view, a rearward camera having a rearward facing view, a left side camera having a left sideward field of view at the left side of the equipped vehicle and a right side camera having a right sideward field of view at the right side of the equipped vehicle;wherein a portion of said forward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view and wherein a portion of said rearward field of view overlaps respective portions of said left sideward field of view and said right sideward field of view;wherein said vision system is operable to calibrate said cameras to correct for mis-alignment of at least one of said cameras;wherein said vision system is operable to calibrate said cameras while the vehicle is moving;wherein a plurality of targets are disposed at ground locations near the path of the vehicle so that, when the vehicle is at or near the targets, the targets are at least partially within the overlapping portions of the fields of views of said cameras;wherein the plurality of targets comprises at least two longitudinal lines extending longitudinally along the direction that the vehicle is moving at either side of the vehicle, with a plurality of tick marks extending laterally from the longitudinal lines; andwherein camera calibration is triggered by at least one triggering target moving into a predetermined position in the field of view of at least one of said cameras as the vehicle moves relative to the targets. 12. The vision system of claim 11, wherein, responsive to triggering of said camera calibration, said cameras will capture and store image for a stitching calibration computation. 13. The vision system of claim 11, comprising a display device operable to display images captured by said cameras. 14. The vision system of claim 11, wherein each of said cameras comprises a lens and an imager. 15. The vision system of claim 14, wherein, during camera calibration, said vision system is operable to utilize a math model to at least in part correct for lens distortion based on lens nominal design data. 16. The vision system of claim 14, wherein, during camera calibration, said vision system is operable to utilize a lens-to-sensor misalignment assessment and correction by determining a relationship between the degree of curvature of horizontal lines and vertical misalignment of said lens to a center of said imager for each of said cameras. 17. The vision system of claim 11, wherein, during camera calibration, said vision system is operable to utilize a perspective correction to at least in part correct for rotation and translation errors. 18. The vision system of claim 11, wherein, during camera calibration, said vision system is operable to, following an initial calibration process, utilize a fine tuning of smaller mis-matches to achieve smooth stitching for the displayed image.
Salmeen, Irving Toivo; Miller, Ronald Hugh; DiMeo, David M.; Strumolo, Gary Steven; Dassanayake, Mahendra Somasara; Alles, Sheran Anthony; Wagner, David Anthony, Adaptive vehicle communication controlled lighting system.
Sato Jun (Cambridge GB2) Saiki Mitsuyoshi (Tsukuba JPX), Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and inc.
Chen Min-Hsiung (3F. No. 10 ; Alley 102 ; Lane 109 ; Der-Hsin E. Rd. Taipei TWX), Automatic luminosity control device for car and motor bicycle headlamps.
Wood Robert B. (Hillsboro OR) Thomas Mark A. (Lake Oswego OR) Desmond John P. (Portland OR), Automobile head-up display system with reflective aspheric surface.
Van Lente Paul S. (Holland MI) Suman Michael J. (Holland MI) Zeinstra Mark L. (Holland MI) DeVree William S. (Holland MI), Electrical control system for vehicle options.
Hatae,Yasuhiko; Usui,Shuji; Nakamura,Yoshifumi, Emergency information notifying system, and apparatus, method and moving object utilizing the emergency information notifying system.
Stein, Gideon S.; Shashua, Amnon; Gdalyahu, Yoram; Liyatan, Harel, Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications.
Wilson David T. (Torrance CA) Wreede John E. (Azusa CA) Gunther John E. (Torrance CA) Arns James A. (Saline MI), Holographic parking assistance device.
Stam, Joseph S.; Mart, Gregory A.; Berends, Keith H.; Bush, Gregory S.; Roberts, John K.; Pierce, Mark W.; Bechtel, Jon H.; Walstra, Eric J.; Rycenga, Brock R., Image acquisition and processing methods for automatic vehicular exterior lighting control.
Lelong Pierre (Nogent/Sur/Marne FRX) Dalm Govert (Veldhoven NLX) Klijn Jan (Breda NLX), Image processing method and device for constructing an image from adjacent images.
Takano Kazuaki,JPX ; Monzi Tatsuhiko,JPX ; Tanaka Yasunari,JPX ; Ondoh Eiryoh,JPX ; Shioya Makoto,JPX, Imaging system for a vehicle which compares a reference image which includes a mark which is fixed to said vehicle to su.
Denny, Patrick Eoghan; Raab, Tycho Lorenz Roland; Sharman, Lloyd Anthony, Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset.
Yung, Wai Lam; Bittner, Wilfried Alois Anton; Lam, Yat Wah; Chan, Sai Chak; Chan, Wah Pong; Ng, Kwong Lam; Zimmermann, Thomas Paul Richard, Method and apparatus for locating and measuring the distance to a target.
Trissel Richard G. (Cardiff CA) DeFoe Douglas N. (Escondido CA), Optical collimating device employing cholesteric liquid crystal and a non-transmissive reflector.
Bottesch H. Werner (R.D. #6 ; Box 374 Danville PA 17821) Freas David A. (P.O. Box 324 Mifflinville PA 18631), Passive vehicle presence detection system.
Bamji,Cyrus, RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging.
Schofield Kenneth (Holland MI) Larson Mark L. (Grand Haven MI) Vadas Keith J. (Coopersville MI), Rearview vision system for vehicle including panoramic view.
Tsuchiya Hideaki (Mitaka JPX) Hanawa Keiji (Chofu JPX) Saneyoshi Keiji (Tokyo JPX), Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof.
Schofield Kenneth (Holland) Gahan Richard J. (Holland) Schierbeek Kenneth L. (Zeeland) Larson Mark L. (Grand Haven MI), Single sensor adaptive drive circuit for rearview mirror system.
Borcherts Robert H. (Ann Arbor MI) Jurzak Jacek L. (Rochester Hills MI) Liou Shih-Ping (Ann Arbor MI) Yeh Tse-Liang A. (Rochester Hills MI), System and method for automatically steering a vehicle within a lane in a road.
Stein Gideon P. ; Shashua Amnon,ILX, System and method for directly estimating three-dimensional structure of objects in a scene and camera motion from three two-dimensional views of the scene.
Stein, Gideon P.; Mano, Ofer, System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion.
Stam, Joseph S.; Bechtel, Jon H.; Reese, Spencer D.; Roberts, John K.; Tonar, William L.; Poe, G. Bruce; Newhouse, Douglas J., System for controlling exterior vehicle lights.
Alves James F. (Camarillo CA) Cacnio Gerry R. (Los Angeles CA) Stevens David R. (Simi Valley CA), Video image processor and method for detecting vehicles.
Kenneth Schofield ; Mark L. Larson ; Keith J. Vadas, Vision system for a vehicle including an image capture device and a display system having a long focal length.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.