Augmented reality systems having a virtual image overlaying an infrared portion of a live scene
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06T-011/00
G02B-027/01
G06T-019/00
H04N-005/33
출원번호
US-0142380
(2016-04-29)
등록번호
US-9995936
(2018-06-12)
발명자
/ 주소
Macannuco, David J.
Beard, Wayne A.
출원인 / 주소
Lockheed Martin Corporation
대리인 / 주소
Withrow & Terranova, PLLC
인용정보
피인용 횟수 :
0인용 특허 :
113
초록▼
A head-mounted display (HMD) for an augmented reality system allows a user to view an augmented scene comprising a real-world portion of a live scene combined with virtual images overlaying an infrared (IR) portion of the live scene. The HMD includes a head-wearable frame, a lens defining a user fie
A head-mounted display (HMD) for an augmented reality system allows a user to view an augmented scene comprising a real-world portion of a live scene combined with virtual images overlaying an infrared (IR) portion of the live scene. The HMD includes a head-wearable frame, a lens defining a user field of view (FOV), and a camera. The lens permits the user to view a live scene corresponding to the user FOV, and the camera is configured to capture a captured image of the live scene containing data indicative of an infrared (IR) portion of the live scene. The IR portion of the live scene includes reflected IR light above a predetermined threshold, such as an IR reflective background surface in a simulation environment. Based on the IR portion, a virtual image is displayed on an interior surface of the lens so that the virtual image overlays the IR portion.
대표청구항▼
1. A head-mounted display (HMD) comprising: a head-wearable frame configured to be worn by a user;a lens coupled to the frame, the lens defining a user field of view (FOV), the lens comprising an interior surface configured to face the user;a camera coupled to the frame, the camera having a camera F
1. A head-mounted display (HMD) comprising: a head-wearable frame configured to be worn by a user;a lens coupled to the frame, the lens defining a user field of view (FOV), the lens comprising an interior surface configured to face the user;a camera coupled to the frame, the camera having a camera FOV registered to the user FOV, the camera configured to capture a captured image of a live scene corresponding to the user FOV, the captured image comprising data indicative of an IR portion of the live scene, the IR portion comprising an infrared (IR) light in a predetermined IR wavelength band reflected above a predetermined threshold;a processing device configured to: detect the IR portion of the live scene based on the captured image;determine the virtual image based on the IR portion by: determining a relative position and orientation of the user FOV;determining a virtual FOV within a virtual environment model based on the relative position and orientation of the user FOV, the virtual FOV defining a virtual scene; andregistering a portion of the virtual scene to the IR portion of the live scene to determine the virtual image; anda display device configured to display the virtual image on the interior surface of the lens, the virtual image registered to the IR portion of the live scene based on the captured image, the virtual image configured to overlay the IR portion of the live scene in the user FOV. 2. The HMD of claim 1, wherein the predetermined threshold is an IR brightness threshold, the processing device further configured to: determine, for a plurality of predefined sections of the captured image, whether an IR brightness of each section in the predetermined IR wavelength band is above the IR brightness threshold; andregister each section of the captured image having an IR brightness above the IR brightness threshold to the IR portion of the live scene. 3. The HMD of claim 2, wherein the plurality of predefined sections of the captured image is a plurality of pixels, each pixel mapped to a predetermined location of the interior surface of the lens. 4. The HMD of claim 1, wherein the interior surface of the lens comprises a reflective surface, and wherein the display device is configured to project the virtual image on the reflective surface of the lens, the virtual image configured to overlay the IR portion of the live scene. 5. An augmented reality system comprising: a head-mounted display (HMD) comprising: a head-wearable frame configured to be worn by a user;a lens coupled to the frame, the lens defining a user field of view (FOV), the lens comprising an interior surface configured to face the user;a camera coupled to the frame, the camera having a camera FOV registered to the user FOV, the camera configured to capture a captured image of a live scene corresponding to the user FOV; anda display device configured to display a virtual image on the interior surface of the lens;a simulation environment comprising an infrared (IR) reflective surface disposed in the live scene, the IR reflective surface configured to reflect IR light to define an IR portion of the live scene; anda processing device configured to: detect the IR portion of the live scene based on the captured image by: determining, for a plurality of predefined sections of the captured image, whether an IR brightness of each section in a predetermined IR wavelength band is above an IR brightness threshold; andregistering each section of the captured image having an IR brightness above the IR brightness threshold to the IR portion of the live scene; anddetermine the virtual image based on the IR portion;wherein the virtual image is registered to the IR portion of the live scene on the interior surface of the lens, the virtual image configured to overlay the IR portion of the live scene in the user FOV. 6. The augmented reality system of claim 5, wherein the IR reflective surface in the live scene comprises a window region of the simulation environment. 7. The augmented reality system of claim 5, further comprising an object disposed in the live scene in a line of sight between a portion of the IR reflective surface in the live scene and the HMD. 8. The augmented reality system of claim 5, wherein the IR reflective surface in the live scene is configured to reduce a translucency of the virtual image. 9. The augmented reality system of claim 8, wherein the IR reflective surface in the live scene is configured to absorb visible light to reduce the translucency of the virtual image. 10. The augmented reality system of claim 5, wherein the plurality of predefined sections of the captured image is a plurality of pixels, each pixel mapped to a predetermined location of the interior surface of the lens. 11. The augmented reality system of claim 1, wherein the interior surface of the lens comprises a reflective surface, and wherein the display device is configured to project the virtual image on the reflective surface of the lens, the virtual image configured to overlay the IR portion of the live scene. 12. A method of providing a virtual image to a head-mounted display (HMD), the method comprising: determining an infrared (IR) portion of a live scene based on a captured image of the live scene, the live scene corresponding to a user field of view (FOV) defined by a lens of the HMD, the IR portion comprising IR light in a predetermined IR wavelength band reflected above a predetermined threshold;determining a relative position and orientation of the user FOV;determining a virtual FOV within a virtual environment model based on the relative position and orientation of the user FOV, the virtual FOV defining a virtual scene;registering a portion of the virtual scene to the IR portion of the live scene to determine the virtual image; anddisplaying, via a display device of the HMD, the virtual image on an interior surface of the lens, the virtual image registered to the IR portion of the live scene based on the captured image, the virtual image configured to overlay the IR portion of the live scene in the user FOV. 13. The method of claim 12, wherein the IR portion corresponds to an IR reflective surface disposed in the live scene, the IR reflective surface configured to reflect IR light in the predetermined IR wavelength band. 14. The method of claim 13, further comprising an object disposed in the live scene in a line of sight between a portion of the IR reflective surface in the live scene and the HMD. 15. The method of claim 13, wherein the IR reflective surface in the live scene is configured to reduce a translucency of the virtual image overlaying the IR portion of the live scene in the user FOV. 16. The method of claim 15, wherein the IR reflective surface in the live scene is configured to absorb visible light to reduce the translucency of the virtual image overlaying the IR portion of the live scene in the user FOV. 17. A head-mounted display (HMD) comprising: a head-wearable frame configured to be worn by a user;a lens coupled to the frame, the lens defining a user field of view (FOV), the lens comprising an interior surface configured to face the user;a camera coupled to the frame, the camera having a camera FOV registered to the user FOV, the camera configured to capture a captured image of a live scene corresponding to the user FOV, the captured image comprising data indicative of an IR portion of the live scene, the IR portion comprising an infrared (IR) light in a predetermined IR wavelength band reflected above an IR brightness threshold;a processing device configured to: determine, for a plurality of predefined sections of the captured image, whether an IR brightness of each section in the predetermined IR wavelength band is above the IR brightness threshold;register each section of the captured image having an IR brightness above the IR brightness threshold to the IR portion of the live scene; anddetermine the virtual image based on the IR portion; anda display device configured to display the virtual image on the interior surface of the lens, the virtual image registered to the IR portion of the live scene based on the captured image, the virtual image configured to overlay the IR portion of the live scene in the user FOV.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (113)
Peng,Guolin; Steffensmeier,Martin J., Advanced compact head up display.
Friedrich,Wolfgang; Wohlgemuth,Wolfgang, Augmented-reality system for situation-related support of the interaction between a user and an engineering apparatus.
Yamasaki, Masafumi, Camera and photographing method for setting focal distance of photographing optical system so as to correspond to information that indicates photographic range inputted via an operation section.
Lewis, John R.; Wei, Yichen; Crocco, Robert L.; Vaught, Benjamin I.; Kipman, Alex Aben-Athar; Perez, Kathryn Stone, Gaze detection in a see-through, near-eye, mixed reality display.
Janssen, Holger, Head-up display system and method for carrying out the location-correct display of an object situated outside a vehicle with regard to the position of the driver.
Zaenglein ; Jr. William G. (1235 Skyline Dr. Laguna Beach CA 92651), Shooting simulating process and training device using a virtual reality display screen.
Barger, James Edwin; Mullen, Richard James; Cruthirds, Daniel Ramsay; Coleman, Ronald Bruce, Systems and methods for disambiguating shooter locations with shockwave-only location.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.