Projection-based head-mounted display with eye-tracking capabilities
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G02B-027/12
G09G-005/00
출원번호
UP-0635989
(2006-12-08)
등록번호
US-7522344
(2009-07-01)
발명자
/ 주소
Curatu, Costin E.
Rolland, Jannick P.
출원인 / 주소
University of Central Florida Research Foundation, Inc.
대리인 / 주소
Steinberger, Brian S.
인용정보
피인용 횟수 :
157인용 특허 :
5
초록▼
Methods, systems, apparatus and devices for the lens design of an HMPD with eye-tracking capabilities. The integration uses a low-level optical configuration in order to achieve a compact, comfortable, easy-to-use system. The optical system is further designed and optimized for sharing of the optica
Methods, systems, apparatus and devices for the lens design of an HMPD with eye-tracking capabilities. The integration uses a low-level optical configuration in order to achieve a compact, comfortable, easy-to-use system. The optical system is further designed and optimized for sharing of the optical path between the HMD and the Eye-Tracker with minimal performance loss for both tasks.
대표청구항▼
We claim: 1. A projection-based head-mounted display device with an optical system inside the head-mounted display for eye-tracking capability, the optical eye tracking system comprising: multiple light sources for providing an infrared light to illuminate an eye of a user for creating multiple gli
We claim: 1. A projection-based head-mounted display device with an optical system inside the head-mounted display for eye-tracking capability, the optical eye tracking system comprising: multiple light sources for providing an infrared light to illuminate an eye of a user for creating multiple glints for use in detecting a polygon formed by the multiple glints, the multiple light source consisting of a dual light source for obtaining a bright pupil image and a dark pupil image of the eye, wherein the bright pupil image and the dark pupil image are subtracted to obtain a pupil of the eye for eye tracking; a camera for recording the multiple glints from the eye, the bright pupil image and the dark pupil image alternating with each frame taken by the camera; an optics for providing an optical path for a head-mounted display path and an eye-tracker path; a first and a second optical device for imaging the eye onto the camera, the optical device including a first and second hot mirror positioned in the eye-tracker path to reflect the infrared light and transmit a visible light; an interface for coupling an external controller with the head-mounted projection display for executing a set of instructions for controlling the projection-based head-mounted display device; and a display for displaying an image to the user, the display path shared with the eye eye-tracker path without obstructing the user's view, wherein the projection-based head-mounted display with eye-tracking capabilities built into the head-mounted device is compact, light-weight, comfortable and easy-to-use. 2. The display device eye tracking optical system of claim 1, wherein the dual light source comprises: an on-axis light source aligned with a semi-transparent second hot mirror, wherein a first infrared light from the on-axis light source is on the same axis as the camera to obtain the bright pupil image; and an off-axis light source aligned with the first hot mirror and having at least two infrared LEDs, wherein a second infrared light from the off-axis light source is off-axis from the camera to create at least three glints for the dark pupil image, wherein the bright pupil image and the dark pupil image alternate with each frame taken by the camera. 3. The display device eye tracking optical system of claim 2, wherein the optics includes: a beam splitter in a shared portion of both the head-mounted display path and the eye-tracker path. 4. The display device eye tracking optical system of claim 3, wherein each of the off-axis and the on-axis light sources comprises: a first and a second infrared light emitting diode mounted around the beam splitter inside the head-mounted display. 5. The display device eye tracking optical system of claim 2, wherein the set of instructions comprises: a set of pupil extraction instructions for synchronizing the camera and the light source for obtaining the bright pupil and dark pupil images and performing a subtraction of the two images to extract the pupil to remove the background to enable thresholding and artifact removal. 6. The display device eye tracking optical system of claim 1, wherein the optics include: an ideal lens module having a effective focal length to yield a diagonal full field of view of approximately 40 degrees for the head mounted projection display. 7. The display device eye tracking optical system of claim 6, wherein the effective focal length is approximately 33 mm. 8. The display device eye tracking optical system of claim 1, wherein the optics include: a double Gauss lens scaled to have an effective focal length to optimize the head-mounted projection display path. 9. The display device eye tracking optical system of claim 8, wherein the effective focal length is approximately 33 mm. 10. The display device eye tracking optical system of claim 8, wherein the double Gauss lens has a weight of less than approximately 9 grams and does not contain aspheric or diffractive elements. 11. A method for providing eye-tracking inside a projection-based head-mounted display comprising the steps of: providing plural infrared light beams from each of a first and a second different infrared light source for illuminating a pupil of a user's eye; using an optical device to create a head-mounted display optical path and an eye-tracking optical path, wherein the optical display path and the eye-tracking optical paths are shared to reduce weight of the projection based head-mounted display; using a first and second hot mirror coupled with the first and second light source, respectively, to image the eye onto a camera; alternatingly recording multiple glints in a bright pupil image and a dark pupil image created by the at least two infrared light sources using the camera; and extracting an eye movement data from the bright pupil image and the dark pupil image using a pupil/glint method. 12. The method of claim 11, wherein the eye movement extracting comprises the steps of: creating plural glints in an eye of a user using the visible light transmitted by the first and second hot mirrors; and calculating a centroid of the shape formed by the plural glints from the bright pupil image and the dark pupil image for tracking the eye movement. 13. The method of claim 11, further comprising: using a first and a second infrared light source having a first and a second infrared light output, respectively; positioning the first infrared light source and first hot mirror to create plural glints to provide a first infrared light on-axis with a point of view of the camera; positioning the second infrared light source and a semitransparent second hot mirror to provide a second infrared light off-axis with the point of view of the camera; recording the bright pupil image and the dark pupil image of a user's eye using the camera, wherein the on-axis light source creates the bright pupil image and the off-axis light source creates the dark pupil image; and subtracting the bright pupil image and the dark pupil image for use tracking eye movement, wherein the bright and the dark pupil images are synchronized to alternate with each frame taken by the camera to improve eye-tracking during rapid eye movement. 14. The method of claim 11, further comprising the step of: simultaneously optimizing the optical device for the head-mounted display path and the eye-tracking path using a zoomed configuration. 15. The method of claim 14, wherein the optimizing step comprises the steps of: adjusting respective wavelength weights for the head-mounted display path and the eye-tracking path according to a spectral eye response and a infrared wavelength; and weighting an extended visible-infrared spectrum across the zoomed configuration to optimize obtain best balanced performance for both the head-mounted display path and the eye-tracking path. 16. The method of claim 11, wherein the using an optical device step includes the step of: selecting a double Gauss lens having a weight of less than approximately 9 grams and not containing aspheric or diffractive elements. 17. A system for providing eye-tracking capabilities within a projection-based head-mounted display comprising: a dual light source for providing a first and second infrared light, respectively; an optical device to create a head-mounted display optical path and an eye-tracking optical path, wherein the head-mounted display and the eye-tracking optical paths are shared to reduce weight of the projection based head-mounted display, the optical device including a first and second hot mirror coupled with the first and second infrared light sources, respectively, for reflecting infrared light and transmitting visible to create plural eye glints for use detecting eye movement; and a camera for recording the eye movement, wherein one source of the dual light source is on-axis and a second source of the dual light source is off-axis with the point of view of the camera; and a processing device for executing a set of instructions to calculate a position of a pupil of the user's eye based on the eye movement extracted from a bright pupil image and a dark pupil image recorded by the camera. 18. The system of claim 17, wherein the set of instructions comprises: a first subset of instructions for synchronizing the camera and the dual infrared light source; a second subset of instructions for alternately obtaining and recording the bright pupil image and the dark pupil image and calculating a difference between the bright pupil image and the dark pupil image to extract the pupil, wherein the background is reduced to enable efficient thresholding and artifact removal.
Fergason James L. (92 Adam Way Atherton CA 94025), Optical system for a head mounted display using a retro-reflector and method of displaying an image.
Osterhout, Ralph F.; Haddick, John D.; Lohse, Robert Michael; Cella, Charles; Nortrup, Robert J.; Nortrup, Edward H., AR glasses with event and sensor triggered AR eyepiece interface to external devices.
Osterhout, Ralph F.; Haddick, John D.; Lohse, Robert Michael; Cella, Charles; Nortrup, Robert J.; Nortrup, Edward H., AR glasses with event and sensor triggered control of AR eyepiece applications.
Osterhout, Ralph F.; Haddick, John D.; Lohse, Robert Michael; Cella, Charles; Nortrup, Robert J.; Nortrup, Edward H., AR glasses with event and user action control of external applications.
Alameh, Rachid Mohsen; Cauwels, Patrick J.; Paitl, Kenneth A., Electronic device with gesture detection system and methods for using the gesture detection system.
Alameh, Rachid Mohsen; Cauwels, Patrick J.; Paitl, Kenneth A., Electronic device with gesture detection system and methods for using the gesture detection system.
Alameh, Rachid Mohsen; Cauwels, Patrick J.; Paitl, Kenneth A., Electronic device with gesture detection system and methods for using the gesture detection system.
Perez, Kathryn Stone; Vaught, Benjamin I.; Lewis, John R.; Crocco, Robert L.; Kipman, Alex Aben-Athar, Enhancing an object of interest in a see-through, mixed reality display device.
Robbins, Steve; McEldowney, Scott; Lou, Xinye; Nister, David; Steedly, Drew; Miller, Quentin Simon Charles; Bohn, David D; Terrell, Jr., James Peele; Goris, Andrew C.; Ackerman, Nathan, Eye-tracking system using a freeform prism and gaze-detection light.
Osterhout, Ralph F.; Haddick, John D.; Lohse, Robert Michael; Border, John N.; Miller, Gregory D.; Stovall, Ross W., Eyepiece with uniformly illuminated reflective display.
Lewis, John R.; Wei, Yichen; Crocco, Robert L.; Vaught, Benjamin I.; Kipman, Alex Aben-Athar; Perez, Kathryn Stone, Gaze detection in a see-through, near-eye, mixed reality display.
Lewis, John R.; Wei, Yichen; Crocco, Robert L.; Vaught, Benjamin I.; Kipman, Alex Aben-Athar; Perez, Kathryn Stone, Gaze detection in a see-through, near-eye, mixed reality display.
Lewis, John R.; Wei, Yichen; Crocco, Robert L.; Vaught, Benjamin I.; Kipman, Alex Aben-Athar; Perez, Kathryn Stone, Gaze detection in a see-through, near-eye, mixed reality display.
Miller, Gregory D.; Border, John N.; Osterhout, Ralph F., Grating in a light transmissive illumination system for see-through near-eye display glasses.
Popovich, Milan Momcilo; Waldern, Jonathan David; Grant, Alastair John, Method and apparatus for generating input images for holographic waveguide displays.
Lvovskiy, Matvey; Efros, Alexander; Volfson, Sofya; Iokhvidson, Simon, Method for determination of head position relative to rectangular axes for observer equipped with head-mounted module.
Miller, Gregory D.; Border, John N.; Osterhout, Ralph F., Optical imperfections in a light transmissive illumination system for see-through near-eye display glasses.
Liu, James Chia-Ming; Andrews, Anton Oguzhan Alford; Maitlen, Craig R.; Novak, Christopher M.; Bennett, Darren A.; Small, Sheridan Martin, Portals: registered objects as virtualized, personalized displays.
Border, John N.; Bietry, Joseph; Osterhout, Ralph F., See-through near-eye display glasses including a curved polarizing film in the image source, a partially reflective, partially transmitting optical element and an optically flat film.
Border, John N.; Haddick, John D.; Osterhout, Ralph F., See-through near-eye display glasses including a partially reflective, partially transmitting optical element.
Border, John N.; Osterhout, Ralph F., See-through near-eye display glasses including an auto-brightness control for the display brightness based on the brightness in the environment.
Border, John N.; Bietry, Joseph; Osterhout, Ralph F., See-through near-eye display glasses wherein image light is transmitted to and reflected from an optically flat film.
Border, John N.; Osterhout, Ralph F., See-through near-eye display glasses with a fast response photochromic film system for quick transition from dark to clear.
Border, John N.; Haddick, John D.; Osterhout, Ralph F., See-through near-eye display glasses with a light transmissive wedge shaped illumination system.
Border, John N.; Haddick, John D.; Lohse, Robert Michael; Osterhout, Ralph F., See-through near-eye display glasses with the optical assembly including absorptive polarizers or anti-reflective coatings to reduce stray light.
Lankford, Chris; Mulholland, II, Timothy; McKinley, Charles, Systems and method of providing automatic motion-tolerant calibration for an eye tracking device.
Publicover, Nelson G.; Torch, William C.; Amayeh, Gholamreza; Leblanc, David, Systems and methods for identifying gaze tracking scene reference locations.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.