Focusable virtual microscopy apparatus and method
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G06K-009/32
출원번호
UP-0373156
(2003-02-24)
등록번호
US-7596249
(2009-10-12)
발명자
/ 주소
Bacus, James V.
Bacus, James W.
출원인 / 주소
Olympus America Inc.
대리인 / 주소
Volpe and Koenig, P.C.
인용정보
피인용 횟수 :
22인용 특허 :
59
초록▼
A virtual microscope slide includes images of a specimen for a given level of optical magnification which are associated and stored in a data structure. The forming of the data structure having the multiple Z-plane images preferably includes automatically focusing at a principal reference focal plan
A virtual microscope slide includes images of a specimen for a given level of optical magnification which are associated and stored in a data structure. The forming of the data structure having the multiple Z-plane images preferably includes automatically focusing at a principal reference focal plane and capturing and digitizing an optically magnified reference Z-image and then shifting the specimen relative to the lens system by a predetermined increment to capture and digitize another Z-plane image. Preferably, a multiple sequence of Z-plane images above and below the reference image and captured and digitized. For ease of retrieval and use, each reference image has its associated Z-plane images are formed in a stack that is sent over the Internet, or Intranet to a local computer storage for quick retrieval when a viewer wants to mimic a focusing up or down to better view a detail in an image. Thus, the resultant images are retrieved and displayed such that a virtual focusing capability is available to the user. The images can be formed with overlapping fields of depth, adjacent fields of depth, or wholly separated fields of depth.
대표청구항▼
What is claimed is: 1. A method of forming a virtual microscope slide to provide images of a specimen outside the depth of field of a principal focal plane, the specimen being deposited on a microscope slide located on a stage movable in X, Y, and Z directions with respect to a microscope lens, the
What is claimed is: 1. A method of forming a virtual microscope slide to provide images of a specimen outside the depth of field of a principal focal plane, the specimen being deposited on a microscope slide located on a stage movable in X, Y, and Z directions with respect to a microscope lens, the method comprising: moving the stage in the Z direction to position the specimen at the principal focal plane and then moving the stage in at least one of the X and Y directions to digitally capture an area of the specimen at a given level of magnification, wherein each stage movement allows the capture of a first set of Z-plane images formed of tiles or strips which are contiguous and non-overlapping, to produce multiple, adjacent, microscope objective optical fields of view, each captured at the principal focal plane; moving the stage in at least one of the X and Y directions over the same area of the specimen, at the given level of magnification, to digitally capture a second set Z-plane images formed of tiles or strips which are contiguous and non-overlapping, to produce multiple, adjacent, microscope objective optical fields of view of the area of the specimen captured at a different Z-plane focal distance; and associating and storing the first and second set of Z-plane images in a data structure for later retrieval of the respective images to preserve the depth of field content for the specimen. 2. A method in accordance with claim 1 comprising: shifting the specimen by predetermined increments above and below the principal focal plane to capture a multiple Z-axis sequence of images referenced to the principal focal plane. 3. A method in accordance with claim 1 comprising: transmitting at least a portion of the data structure by an Internet or Intranet transmission channel; and displaying details of the specimen at an area of interest in the second set of Z-images that cannot be seen in focus in the first set of Z-plane images captured at the principal focal plane. 4. A method in accordance with claim 3 wherein the shifting between a display of the first set of Z-plane and the second set of Z-plane images mimics a focusing up or down when using a microscope and is accomplished by moving a computer selector device such as a mouse wheel or keyboard. 5. A virtual microscope slide data structure stored on a computer readable medium comprising: a first and second set of virtual microscope slide images of a specimen deposited on a microscope slide, created by moving the specimen with respect to a microscope lens and capturing contiguous non-overlapping field of view images from substantially the entire specimen; wherein the first set of images has a plurality of images, each image being associated with an x, y planar position of the specimen taken from a principal focal plane; the second set of images has a plurality of images, each image being associated with an x, y planar position of the specimen taken from one or more different focal planes relative to the first set of images; and the data structure is organized to provide information to enable the coordination of the x, y planar positions of the first and second sets of images for retrieval and viewing at least a portion of the specimen at the principal focal plane and at one or more different focal planes relative to the principal focal plane. 6. A virtual microscope slide data structure in accordance with claim 5 wherein the data structure may be viewed over an Internet or intranet transmission channel using a generic Internet browser. 7. A virtual microscope slide data structure in accordance with claim 5 wherein: the principal image focal plane is a reference image plane and the second set of images are focused at a different focal planes spaced above and below the reference image plane. 8. A virtual microscope slide data structure in accordance with claim 5 wherein: the first and second set of images are stored in one or more tiled data structures representing a single virtual microscope slide, irrespective of the method of capture. 9. A virtual microscope slide data structure in accordance with claim 8 comprising: stored digitized image tiles or strips, representing different levels of magnification at a contiguous principal image focal plane for a given area of interest to enable zooming between images of different levels of magnification of the given area of interest. 10. A method of forming a virtual microscope slide from a specimen on a slide, the specimen exhibiting substantially varying depth in a Z-direction, the method comprising; automatically obtaining a Z-axis focusing position on a first area of the specimen at a given level of magnification to determine a first principal focus plane in the Z-direction; using the first principal focus plane as a reference plane, digitally capturing a first set of Z-plane images formed of tiles or strips which are contiguous and non-overlapping, to produce multiple, adjacent, microscope objective optical fields of view, each captured at the first principal focus plane; providing a second area on the specimen and automatically determining a second principal focus plane spaced in the Z-direction from the first principal focus plane; using the second principal focus plane as a reference plane, digitally capturing a second set of Z-plane images formed of tiles or strips which are contiguous and non-overlapping, to produce multiple, adjacent, microscope objective optical fields of view, each captured at the second principal focus plane; storing the respective images for subsequent retrieval; and storing coordinating information for the respective images to allow retrieval of the respective images to provide Z-plane images of the magnified first and second areas to preserve depth of field content of the specimen. 11. A method in accordance with claim 10 comprising: automatically stepping the microscope relative to the principal focus plane for each field of view; and capturing Z-plane images at predetermined Z-direction planes located above and below the respective principal focal plane to allow a reconstruction of adjacent areas in multiple Z-images; and forming a data structure having the respective multiple Z-plane images and having coordinating information to associate the multiple Z-plane images with their respective reference image. 12. A method in accordance with claim 10 comprising: storing a data structure of the respective images and coordinating information to allow retrieval and display at a local or remote location. 13. A method of using a virtual microscope slide having multiple Z-plane images preserving the depth of field content for the specimen, the method comprising: providing a first and second set of image tiles stored on a computer readable medium, the first and second set of image tiles being acquired from contiguous non-overlapping microscope field of view images, retrieving a first subset of image tiles over an Internet or intranet transmission channel from the first set of image tiles, the first subset of image tiles representing the principal focal plane of the virtual microscope slide, the first subset of image tiles corresponding to a requested view of a given area of the specimen and representing only a portion of the data structure of the virtual microscope slide; displaying the first subset of image tiles in a contiguous tiled image reconstruction to fill a viewing area on a display screen with the requested view; requesting, by a pointing device, a position on the requested view displayed on the display screen to retrieve over an Internet or intranet transmission channel a second subset of image tiles from the second set of image tiles, the second subset of image tiles having one or more of a plurality of Z-plane images taken from the same area but from different focal planes relative to the first subset of images; and producing on the display screen at least a portion of the second subset of image tiles to allow the user to view the additional depth of field content present in the second set of Z-plane images relative to the first set of images image tiles. 14. A method in accordance with claim 13 comprising: providing a selector for selecting the second set of image tiles from the second set of Z-plane images that were acquired either above or below a principal Z-axis focal plane. 15. A method in accordance with claim 13 comprising: caching the second subset of image tiles on a remote computer having a remote computer display screen and displaying on the remote computer display screen at least a portion of the second subset of image tiles reconstructed to fill a viewing area of the remote computer display screen. 16. A method in accordance with claim 13 comprising: prestoring in a tiled data structure individual Z-plane images from the second set of image tiles and associating them with a principal reference image so that one or more of the individual Z-plane images may be retrieved from the data structure to fill an area on a display screen. 17. A method in accordance with claim 13 comprising: transferring the second subset of image tiles in a data structure over the Internet to a remote location, and caching the second subset of image tiles in a remote computer having a remote computer display screen; and moving a computer selector device including one of a mouse, mouse wheel and keyboard to accomplish a shifting between a display of the first subset of reconstructed principal Z-plane image tiles on the remote computer display screen and a display of the second subset of images image tiles on the remote computer display screen to mimic, in a superimposed visual effect manner, a focusing up or down when using a microscope. 18. A method of using a virtual microscope slide comprising: providing data structure of images stored on a computer readable medium at a transmitting station, the set of images acquired from continuous non-overlapping microscope field of view images, using an Internet or intranet transmission channel to retrieve and display an initial optically magnified, digitized reference image at a given level of magnification for a field of view limited by the area available on a viewing screen, wherein the retrieved reference image represents only a portion of a the data structure residing at the transmitting station; sending a focus instruction signal to obtain from the transmission station over the Internet or intranet transmission channel a plurality of other images at the given level of magnification located at different Z-planes on the specimen for substantially the same field of view; retrieving over the Internet or intranet transmission channel the other images to bring into focus details not in focus in the initial reference image; retrieving over the Internet or intranet communication channel and displaying a sequence of reference images some of which are captured when using different referenced focal planes; and retrieving over the Internet or intranet communication channel a plurality of associated Z-plane images for each reference image captured after shifting the specimen incrementally in the Z-direction. 19. A method in accordance with claim 18 comprising: providing Z-plane images in focal planes located in the specimen above and below a primary focal plane for the reference image; and sending focus instructions to cause retrieval of a plurality of other Z-plane images. 20. A method in accordance with claim 19 comprising: automatically displaying the reference image to a user; and the user determining what, if any, instruction signal is to be sent to retrieve another Z-plane image to bring into focus a specimen detail which is located either above or below the principal focus plane for the reference image. 21. A method in accordance with claim 20 comprising: displaying initially a first reference image for each of the areas of the specimen selected by the user for viewing; and selectively sending a focus modification by the user when seeing a detail portion of the reference image not in focus to obtain an in focus view of the detail portion. 22. A method in accordance with claim 18 comprising: selecting another Z-plane image for display due to a difference in specimen thickness. 23. A method in accordance with claim 18 comprising: selecting another Z-plane image for display due to a Z-plane depth difference from a non-uniform microscope slide. 24. A method in accordance with claim 18 wherein each of the reference images has an equal number of Z-plane images captured in planes in the specimen located above and below in the Z-direction of the plane at which the reference image was captured. 25. A method in accordance with claim 24 wherein there are four Z-plane images above the reference image and four Z-plane images below the reference image. 26. A method for forming and using a virtual microscope slide of a specimen having features spaced in the depth in a Z-direction, the method comprising: using the same magnification level and an iterative automatic focus procedure to determine the z-plane position of a plurality of contiguous and abutted reference images and capturing several optically magnified, z-plane images of the specimen features spaced in the Z-direction relative to the reference images; displaying to a viewer one of the optically magnified Z-plane images of the feature; sending of an instruction by the viewer to display another optically magnified Z-plane image; and displaying to the viewer a feature in better focus than shown in the previously displayed image; and displaying first to the viewer a the predetermined reference image and the operator decides whether or not to display an associated Z-plane image in the sense of mimicking a focusing up or down of the displayed images to bring a detail into focus; and providing associated Z-plane images for each of the reference images with the Z-plane images each being captured at spaced increments in the Z-direction from the reference plane used when capturing the reference images. 27. A method in accordance with claim 26 wherein a substantial portion of the specimen on the slide is captured using the same magnification level for several reference plane images at different planes on the specimen. 28. A method in accordance with claim 26 comprising: providing associated Z-plane images at focal planes in the specimen located above and below the reference plane for the reference image. 29. A method in accordance with claim 26 comprising: capturing the z-plane images as adjacent tiles of a field of interest. 30. An apparatus for forming a virtual microscope slide that has a specimen on the slide exhibiting substantially varying depth in a Z-direction comprising: a scanning device for optically magnifying and digitally capturing an area of the specimen at a given level of magnification and; a control system selectively operable to cause the scanning device to acquire a principal image of the area captured at a reference Z-plane and at least one Z-plane image of the area spaced in the Z-direction and referenced to the principal image; wherein the control system moves the slide and specimen with respect to a microscope lens and the scanning device digitally captures a first set of contiguous non-overlapping field of view images, the scanning device automatically focuses at a reference Z-plane and captures a principal image for each field of view from an area of the specimen at the given level of magnification and at a principal focus plane in the Z-direction to provide a set of contiguous principal reference images; a storage device storing both the first and second images for subsequent retrieval and storing coordinating information for the respective images to allow retrieval of the respective images to provide multiple, spaced, Z-axis images of the area of the specimen at the given level of magnification. 31. An apparatus in accordance with claim 30 wherein the control system operates the scanning device to automatically capture Z-plane images at predetermined Z-step increments to allow a reconstruction of adjacent areas in multiple Z-planes; and a recorder forms a data structure having the respective multiple Z-plane images and having coordinating information to associate the multiple Z-plane images for each field of view. 32. A method of forming a virtual microscope slide comprising: moving an object with respect to a microscope lens; optically magnifying the object at a given level of magnification and at a first focus determined automatically, to provide a one or more first fields of view of various portions of the object; digitally capturing a first set of planar, abutted, non-overlapping images that correspond to at least a portion of the first fields of view and the various portions of the object at the first focus; while substantially maintaining the given level of magnification changing the z-axis position of the object relative to a the lens system of the microscope to provide a second focus; digitally capturing a second set of planar, abutted, non-overlapping images that correspond to the same area of the first field of view but at the second focus; storing at least some of both the first set of images and the second set of images for subsequent retrieval. 33. The method of claim 32 wherein digitally capturing a first set of images includes digitally capturing a first set of images that, collectively, comprise a view of substantially the entire object at the given level of magnification and the first focus. 34. The method of claim 33 wherein digitally capturing a second set of images includes digitally capturing a second set of images that, collectively, comprise a view of substantially the entire object at the given level of magnification and the second focus. 35. The method of claim 32 wherein a depth of field corresponding to the first focus is substantially adjacent to a depth of field corresponding to the second focus. 36. A method of using a focus able virtual microscope slide comprising: providing data structure having a first and second set of image tiles stored on a computer readable medium, the first and second set of image tiles being acquired from contiguous non-overlapping microscope field of view images, retrieving over an Internet or intranet transmission channel from a remote computer controlling a display device a first subset of image tiles from the first set of image tiles and creating a first image filling a viewing area on a the display device; wherein the first image is reconstituted from the first subset of image tiles but represent only a portion of the data structure and have one or more different reference focal planes, but appears in the reconstituted view to have one principal focal plane; and moving a computer interface device to cause retrieving over an Internet or intranet transmission channel and the creation of one or more second reconstituted images but represent only a portion of the data structure overlaid and positioned over the first image, to mimic the visual effect of focusing a microscope; wherein each tile of a second reconstituted image is relative to the principal focal plane of the tile it overlays; and the second set of reconstituted images appears in the view as if in one different focal plane, either above or below the principal focal plane of the first image. 37. The method of claim 36 comprising: activating a continuous display to automatically move through the one or more sets of Z stack tiles needed to cover the viewing area and produce a visual effect of moving in and out of focus on either side of the principal focal plane of the specimen in the viewing area on the display device. 38. The method of claim 36 comprising: caching the principal focal plane image and the individual Z stack images necessary to reconstruct the view required on the remote computer to allow a faster local simulation of the visual effect of focusing. 39. A method of forming a virtual microscope slide of an object, the method comprising: optically magnifying the object at a given level of magnification; moving the object with respect to a microscope lens to digitally capture a first plurality of contiguous, non-overlapping images of the object at the given level of magnification at a plurality of first principal Z-plane focal distances; while substantially maintaining the given level of magnification, digitally capturing a second set of additional images for at least some of the first plurality of images of the object at a variety of Z-plane positions on the specimen at different focal distances than the first principal Z-plane focal distances to provide an additional plurality of images of the object at the given level of magnification at varying levels of focus; and storing both the first plurality of images and the second set of additional images for subsequent retrieval. 40. The method of claim 39 wherein digitally capturing the second set of additional images includes digitally capturing at least two additional images for each of the at least some of the plurality of images of the object at a variety of focal positions. 41. The method of claim 40 wherein digitally capturing the second set of additional images includes digitally capturing at least eight additional images for each of the at least some of the plurality of images of the object at a variety of focuses. 42. The method of claim 41 wherein digitally capturing at least eight additional images includes digitally capturing at least eight additional images such that some of the additional images are more closely spaced relative to the microscope lens in the Z-direction than images in the plurality of images and some of the additional images are more distantly spaced relative to the microscope lens in the Z-direction than images in the first plurality of images. 43. The method of claim 41 wherein digitally capturing at least eight additional images includes digitally capturing at least eight additional images such that four of the additional images are above the principal focal position in the Z-direction than corresponding images in the plurality of images and four of the additional images are above the principal focal position in the Z-direction than corresponding images in the first plurality of images. 44. A focusable virtual microscope slide comprising: a first plurality of digitized microscopic image tiles derived from contiguous non-overlapping field of view images captured from a specimen deposited on a microscope slide, wherein the first plurality of digitized microscopic image tiles can be reconstituted to form a first virtual composite view of the specimen comprised of contiguous tiles with one or more different principal focal positions for each of the respective tiles relative to each other; and a second plurality of microscopic digitized image tiles wherein the second plurality of digitized image tiles represent individual sets of Z stack tiles at different focal positions referenced to the individual principal focal positions respectively of each of the first plurality of digitized image tiles, and maybe reconstituted to form composite views of the specimen at focal positions different from the first virtual composite view.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (59)
Bacus James W. (Hinsdale IL), Analysis method and apparatus for biological specimens.
Preston ; Jr. Kendall (4701 E. Glenn St. ; Bldg. 36 Tucson AZ), Apparatus and method for a multi-resolution electro-optical imaging, display and storage/retrieval system.
Gaylord William J. (Stone Mountain GA), Apparatus and method for segmenting a field of view into contiguous, non-overlapping, vertical and horizontal sub-fields.
Riley James K. ; Frost Keith L. ; Lindow William C. ; Hansen Kim J. ; Phan Tuan ; Stephanick James A., Astigmatism measurement apparatus and method based on a focal plane separation measurement.
Rutenberg Mark R. (Monsey NY) Hall Thomas L. (Pasadena CA) Domes Robert (Newbury Park CA) Chaban Richard (Ventura CA) Dulak Thomas (Newhall CA) Herriman James M. (New York NY) Porzio John (New York N, Automated cytological specimen classification system and method.
Boon Mathilde E. (Lieveren NLX) Kok Lanbrecht P. (Lieveren NLX) Mango Laurie J. (Rooevelt Island NY) Rutenberg Akiva (Monsey NY) Rutenberg Mark R. (Monsey NY), Automated specimen classification system and method.
Kennedy Ann R. (Wynnewood PA) Szuhaj Bernard F. (Fort Wayne IN), Bowman-birk inhibitor concentrate compositions and methods for the treatment of pre-malignant tissue.
Domanik Richard A. ; Dadeshidze Vladimir ; Olsson Lars J., Cytological specimen analysis system with slide mapping and generation of viewing path information.
Rutenberg Mark R. (Monsey NY) Tjon-Fo-Sang Robert (Valley Cottage NY) Strinkovsky Leonid (Spring Valley NY) Mango Laurie J. (Roosevelt Island NY) Herriman James C. (Huntington Station NY), Device and method for facilitating inspection of a specimen.
Bacus James V. ; Bacus James W., Method and apparatus for acquiring and reconstructing magnified specimen images from a computer-controlled microscope.
Bacus James W. (Hinsdale IL) Bacus James V. (Lombard IL) Wagner Robert E. (Chicago IL), Method and apparatus for automated assay of biological specimens.
Peters Richard K. (Tallmadge OH) Elmerick Donald V. (Tallmadge OH) Spayer James L. (Brecksville OH) Walter Gerald E. (Cleveland OH 4), Method and apparatus for cell analysis.
Ortyn William E. (Devall WA) Arnone Joseph (Bainbridge Island WA), Method and apparatus for checking automated optical system performance repeatability.
Kopp Richard E. (Smithtown NY) Lisa Joseph (Ronkonkoma NY) Mendelsohn Jay (Old Bethpage NY) Pernick Benjamin J. (Brooklyn NY) Stone Harvey (Flushing NY) Wohlers Martin R. (Cold Spring Harbor NY), Method and apparatus for classifying biological cells.
Bacus James W. (Hinsdale IL) Bacus Sarah S. (Hinsdale IL), Method and apparatus for determining the amount of oncogene protein product in a cell sample.
Kamentsky Louis A. (180 Beacon St. ; #17G Boston MA 02116) Kamentsky Lee D. (79 Paul Gore St. Jamaca Plain MA 02130), Method and apparatus for measuring multiple optical properties of biological specimens.
Kopp Richard E. (Smithtown NY) Lisa Joseph (Ronkonkoma NY) Mendelsohn Jay (Old Bethpage NY) Pernick Benjamin J. (Brooklyn NY) Stone Harvey (Flushing NY) Wohlers Martin R. (Cold Spring Harbor NY), Method for classifying biological cells.
Reinhardt Erich (Stuttgart DEX) Erhardt Rainer (Stuttgart DEX), Method of automatic detection of cells and determination of cell features from cytological smear preparations.
Krummey Michael J. (Madison WI) Lee Richard J. (Murrysville PA) Nesbitt James E. (Pittsburgh PA) Powers ; Jr. William J. (Pittsburgh PA) Schwoeble Alfred J. (Monroeville PA), Simultaneously recording of video image and microscope stage position data.
Crandall,Greg J.; Eichhorn,Ole; Olson,Allen H.; Soenksen,Dirk G., System and method for data management in a linear-array-based microscope slide scanner.
Ellenby John (La Jolla CA) Ellenby Thomas (La Jolla CA) Ellenby Peter (La Jolla CA), Vision systems including devices and methods for combining images for extended magnification schemes.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.