초록 ▼

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The system creates a volumetric model of

A system that includes a head mounted display device and a processing unit connected to the head mounted display device is used to fuse virtual content into real content. In one embodiment, the processing unit is in communication with a hub computing device. The system creates a volumetric model of a space, segments the model into objects, identifies one or more of the objects including a first object, and displays a virtual image over the first object on a display (of the head mounted display) that allows actual direct viewing of at least a portion of the space through the display.

대표청구항 ▼

1. A method for fusing virtual content into real content, comprising: creating a volumetric model of a space;segmenting the model into objects, the segmenting the model into objects includes accessing one or more depth images and accessing one or more visual images, the segmenting the model into obj

1. A method for fusing virtual content into real content, comprising: creating a volumetric model of a space;segmenting the model into objects, the segmenting the model into objects includes accessing one or more depth images and accessing one or more visual images, the segmenting the model into objects includes detecting one or more persons using the one or more depth images and the one or more visual images, the segmenting the model into objects includes detecting edges based on the one or more depth images and the one or more visual images, the segmenting the model into objects includes detecting the objects based on the edges and updating the model to store information about the detection of the objects and the one or more persons;identifying one or more of the objects including a first object, the identifying one or more of the objects includes matching at least one of the detected objects to a user identity and accessing a data structure of shapes, the identifying one or more of the objects includes matching one or more of the detected objects to shapes in the data structure and updating the model to reflect the matching of the at least one of the detected objects to the user identity and the matching of the one or more of the detected objects to shapes in the data structure, the identifying one or more of the objects includes visually displaying an unmatched object to a user of a display and requesting the user to identify the unmatched object, the identifying one or more of the objects includes receiving a new identification for the unmatched object and updating the data structure to reflect the new identification, the identifying one or more of the objects includes updating the model to store information about the new identification for the unmatched object; andautomatically displaying a virtual image over the first object on the display that allows actual direct viewing of at least a portion of the space through the display. 2. The method of claim 1, wherein the creating a volumetric model comprises: receiving depth images from multiple perspectives; andusing data from the depth images to create and store a three dimensional description of the space. 3. The method of claim 1, further comprising: specifying that the first object is to be replaced; andidentifying the virtual image to replace the first object. 4. The method of claim 1, wherein the automatically displaying the virtual image over the first object further comprises: identifying a light source; andadjusting the virtual image based on the light source. 5. The method of claim 1, wherein the automatically displaying the virtual image over the first object further comprises: identifying a shadow area; anddarkening a view of a real object corresponding to the shadow area. 6. The method of claim 1, wherein: the automatically displaying the virtual image over the first object on the display comprises determining a point of view of the user of the display, the determining field point of view of the user of the display comprises automatically determining location and orientation of the display using the one or more depth images, automatically refining the determined orientation of the display using inertial sensors and automatically determining position of an eye of the user of the display to identify a subset of the space viewed at the determined location and orientation. 7. The method of claim 1, wherein the displaying of the virtual image over the first object on the display further comprises: automatically determining that the first object is moving;automatically tracking movement of the first object;automatically determining a change in shape of the first object; andblurring the virtual image in response to movement of the first object. 8. The method of claim 1, wherein the displaying of the virtual image over the first object on the display further comprises: identifying a portion of the display that shows the first object; andselectively blocking light from passing through the portion of the display that shows the first object, the virtual image is projected on the portion of the display that shows the first object. 9. The method of claim 1, wherein: the automatically displaying the virtual image includes projecting the virtual image on the display without projecting the portion of the space on the display; andthe first object is a portion of a person. 10. The method of claim 1, wherein: the display comprises a head-mounted display.