Efficient 3-D telestration for local and remote robotic proctoring
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-015/00
A61B-006/00
A61B-019/00
G06T-007/00
G06T-011/00
H04N-013/00
H04N-013/02
H04N-013/04
출원번호
US-0450251
(2014-08-03)
등록번호
US-9402690
(2016-08-02)
발명자
/ 주소
Zhao, Wenyi
Wu, Chenyu
Hirvonen, David
Hasser, Christopher J.
Miller, Brian E.
Mohr, Catherine J.
Zhao, Tao
Di Maio, Simon
Hoffman, Brian D.
출원인 / 주소
Intuitive Surgical Operations, Inc.
인용정보
피인용 횟수 :
0인용 특허 :
70
초록▼
An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stere
An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.
대표청구항▼
1. A medical system comprising: an image capture device for capturing stereo images of a work site, each stereo image including a first two-dimensional image and a second two-dimensional image, wherein corresponding points in the first and second two-dimensional images depict a common point in the w
1. A medical system comprising: an image capture device for capturing stereo images of a work site, each stereo image including a first two-dimensional image and a second two-dimensional image, wherein corresponding points in the first and second two-dimensional images depict a common point in the work site and exhibit a disparity between the corresponding points so as to indicate a depth of the common point relative to the image capture device;a stereo display displaying the first and second two-dimensional images so as to provide a three-dimensional viewing experience; anda processor that is programmed to: receive information of a first telestration indicative of user selected points of interest of the first two-dimensional image; selectively match the selected points of interest of the first two-dimensional image to corresponding points of the second two-dimensional image to determine matched points of interest of the second two-dimensional image by computing a plurality of image offsets wherein each of the plurality of image offsets is computed using a different image matching method than all others of the plurality of image offsets, and by using one of the computed plurality of image offsets which is selected according at least partially to one or more confidence scores of the plurality of image offsets, wherein the matched points of interest correspond to a second telestration mark; and cause the first and second telestration marks to be displayed respectively over the first and second two-dimensional images on the stereo display such that the first telestration mark and the second telestration mark appear as a single three-dimensional telestration mark on the stereo display. 2. The medical system of claim 1, further comprising: a user interface that is usable by a first user to define the first telestration by indicating the user selected points of interest of the first two-dimensional image. 3. The medical system of claim 2, wherein the user interface includes a touchscreen display for displaying the first two-dimensional image. 4. The medical system of claim 2, wherein the user interface includes a monovision display and an input device, wherein the input device is usable by the first user to define the first telestration by indicating the user selected points of interest on the first two-dimensional image being displayed on the monovision display. 5. The medical system of claim 1, wherein the plurality of image offsets includes at least two of: a global offset which the processor is programmed to compute by using a coarse-to-fine global offset image matching method, a region offset which the processor is programmed to compute by using a coarse-to-fine region image matching method, and a feature offset which the processor is programmed to compute by using a point matching method based upon feature detection and matching. 6. The medical system of claim 5, wherein the processor is programmed to compute the global offset by using normalized cross correlations to compare information of the first and second two-dimensional images. 7. The medical system of claim 5, wherein the processor is programmed to compute the region offset by matching a region of the first two-dimensional image to a region of the second two-dimensional image by using at least one of cross-correlation, two-way matching, least squares regression, and non-linear regression. 8. The medical system of claim 5, wherein the processor is programmed to compute the feature offset by matching features of the first two-dimensional image to features of the second two-dimensional image by using at least one of Harris corner detection, scale-space extrema detection, local extrema detection, and scale invariant feature transform. 9. The medical system of claim 1, wherein the processor is programmed to perform the selectively matching by at least one of selectively matching regions, selectively matching features, selectively interpolating features, and selectively interpolating previously matched points of interest. 10. The medical system of claim 9, wherein the processor is programmed to determine region match confidence scores when the regions are selectively matched, determine feature match scores when the features are selectively matched, selectively match the features in response to the region match confidence scores, selectively interpolate the features in response to the feature match scores, and selectively interpolate the previously matched points of interest in response to the feature match scores; and wherein locations of the matched points of interest are determined by using the region match confidence scores and the feature match scores.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (70)
Weiglhofer, Gerhard; Pauker, Fritz; Henkel, Rolf, 3D stereo real-time sensor system, method and computer program therefor.
Robinson, David; Nixon, Thomas R.; Hanuschik, Michael; Goldberg, Randal P.; Hemphill, Jason; Larkin, David Q.; Millman, Paul, Adaptable integrated energy control system for electrosurgical tools in robotic surgical systems.
Moll, Frederic H.; Rosa, David J.; Ramans, Andris D.; Blumenkranz, Stephen J.; Guthart, Gary S.; Niemeyer, Gunter D.; Nowlin, William C.; Salisbury, Jr., J. Kenneth; Tierney, Michael J.; Mintz, David, Arm cart for telerobotic surgical system.
Gunter D. Niemeyer ; Gary S. Guthart ; William C. Nowlin ; Nitish Swarup ; Gregory K. Toth ; Robert G. Younge, Camera referenced control in a minimally invasive surgical apparatus.
Niemeyer,Gunter D.; Guthart,Gary S.; Nowlin,William C.; Swarup,Nitish; Toth,Gregory K; Younge,Robert G., Camera referenced control in a minimally invasive surgical apparatus.
Salisbury, Jr., J. Kenneth; Niemeyer, Gunter D.; Younge, Robert G.; Guthart, Gary S.; Mintz, David S.; Cooper, Thomas G., Devices and methods for presenting and regulating auxiliary information on an image display of a telesurgical system to assist an operator in performing a surgical procedure.
Banker Robert O. (Cumming GA) Ith Cham (Duluth GA) Bacon Kinney C. (Lawrenceville GA) Burleson David B. (Roswell GA), Display system for selectively overlaying symbols and graphics onto a video signal.
Zhao, Wenyi; Wu, Chenyu; Hirvonen, David; Hasser, Christopher J.; Miller, Brian E.; Mohr, Catherine J.; Curet, Myrian J.; Zhao, Tao; Di Maio, Simon; Hoffman, Brian D., Efficient 3-D telestration for local robotic proctoring.
Cavallaro, Richard H.; Gloudemans, James R.; Lazar, Matthew T.; Meier, Kevin R.; Mozes, Alon; Peon, Roberto J.; Steinberg, Eric M., Enhancing video using a virtual surface.
Wang Yulun ; Uecker Darrin R. ; Jordan Charles S. ; Wright James W. ; Laby Keith Phillip ; Wilson Jeff D., Method and apparatus for performing minimally invasive cardiac procedures.
Han Chia Y. (Hamilton County OH) Porembka David T. (Boone County KY) Lin Kwun-Nan (Fayette County KY), Method for automatic contour extraction of a cardiac image.
Chang, William H. L.; Hameed, Salmaan; Mahadik, Amit A.; Javadekar, Kiran A.; Abello, Oretho F., Multi-function image and video capture device for use in an endoscopic camera system.
Philip C. Evans ; Frederic H. Moll ; Gary S. Guthart ; William C. Nowlin ; Rand P. Pendleton ; Christopher P. Wilson ; Andris D. Ramans ; David J. Rosa ; Volkmar Falk ; Robert G. Younge, Performing cardiac surgery without cardioplegia.
William C. Nowlin ; Gary S. Guthart ; J. Kenneth Salisbury, Jr. ; Gunter D. Niemeyer, Repositioning and reorientation of master/slave relationship in minimally invasive telesurgery.
Tierney, Michael J.; Cooper, Thomas; Julian, Chris; Blumenkranz, Stephen J.; Guthart, Gary S.; Younge, Robert G., Surgical robotic tools, data architecture, and use.
Touzawa, Yoshito; Nishigaki, Morio; Hagiwara, Hisashi, System for accurately obtaining a contour and/or quantitative information from an echo image with reduced manual operation.
Madhani Akhil J. ; Salisbury J. Kenneth, Wrist mechanism for surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitiv.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.