초록 ▼

In the multi-modal interface apparatus of the present invention, a gaze object detection section always detects a user's gaze object. The user inputs at least one medium of sound information, character information, image information and operation information through a media input section. In order t

In the multi-modal interface apparatus of the present invention, a gaze object detection section always detects a user's gaze object. The user inputs at least one medium of sound information, character information, image information and operation information through a media input section. In order to effectively input and output information between the user and the apparatus, a personified image presentation section presents a personified image to the user based on the user's gaze object. A control section controls a reception of the inputted media from the media input section based on the user's gaze object.

대표청구항 ▼

[ What is claimed is:] [9.] Multi-modal interface apparatus, comprising:image input means for inputting a user's image including face and hand;recognition means for extracting the face and hand from the user's image, and for recognizing a face position as the user's view position, a finger end posit

[ What is claimed is:] [9.] Multi-modal interface apparatus, comprising:image input means for inputting a user's image including face and hand;recognition means for extracting the face and hand from the user's image, and for recognizing a face position as the user's view position, a finger end position as the user's gesture part, and a reference object pointed by the finger end as the user's gesture input in a world coordinate space;a location information memory means for storing a representative position and a direction by unit of a label information in the world coordinate space, wherein the label information is one of a presentation position of a personified image, the face, the finger end and the reference object, a plurality of the presentation positions being predetermined, the face, the finger end and the object being recognized by said recognition means;control means for deciding a visual relation between the presentation position and the user's recognized position by referring to said location information memory means, wherein the user's recognized position includes the face position and the finger end position, and for generating the personified image expressing at least one of looks and action based on the visual relation; andpersonified image presentation means for presenting the personified image at the presentation position to the user as a feedback information of the gesture input;wherein said control means decides whether the user's gesture part is watched from a current presentation position of the personified image and whether the personified image is watched from the user's view position, and generates the personified image which gazes the user's gesture part if the user's gesture part is watched from the current presentation position of the personified image and the personified image is watched from the user's view position.