Virtual keyboard for a non-tactile three dimensional user interface
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06F-017/28
G06F-017/20
G06F-017/27
G06F-017/21
G06F-003/01
G06F-003/048
G06F-003/02
G06F-003/043
G06F-003/041
G06F-003/033
G06F-003/045
G06F-003/023
G09G-005/08
G09G-005/00
G06T-011/20
G06K-009/00
G06K-009/46
G06K-009/66
출원번호
US-0244490
(2011-09-25)
등록번호
US-8959013
(2015-02-17)
발명자
/ 주소
Galor, Micha
Or, Ofir
Litvak, Shai
Sali, Erez
출원인 / 주소
Apple Inc.
대리인 / 주소
D. Kligler I.P. Services Ltd
인용정보
피인용 횟수 :
1인용 특허 :
89
초록▼
A method, including presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard including multiple virtual keys, and capturing a sequence of depth maps over time of a body part of a human subject. On the display, a
A method, including presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard including multiple virtual keys, and capturing a sequence of depth maps over time of a body part of a human subject. On the display, a cursor is presented at positions indicated by the body part in the captured sequence of depth maps, and one of the multiple virtual keys is selected in response to an interruption of a motion of the presented cursor in proximity to the one of the multiple virtual keys.
대표청구항▼
1. A method, comprising: presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys;capturing a sequence of depth maps, via a 3D capturing device, over time of a hand of a human sub
1. A method, comprising: presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys;capturing a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane;presenting, on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane; andselecting one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor's trajectory path segment over the virtual keys, wherein the trajectory of the user's hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs. 2. The method according to claim 1, wherein selecting one of the multiple virtual keys comprises using a language model. 3. The method according to claim 2, wherein the language model is selected from a list consisting of a dictionary, a statistical dictionary, an n-gram model, a Markov model and a dynamic Bayesian network. 4. The method according to claim 2, wherein the language model applies rules specific to a given language. 5. The method according to claim 4, wherein the rules are selected from a list consisting of word rules, short phrase rules, parts of speech rules and grammatical rules. 6. The method according to claim 2, wherein the language model utilizes a custom dictionary based on text previously entered by a user interacting with the non-tactile three dimensional user interface. 7. The method according to claim 2, wherein the language model utilizes a custom dictionary specific to an application executing on the computer system. 8. The method according to claim 2, and comprising the language model selecting none of the virtual keys if none of the virtual keys that are in proximity to the cursor are sufficiently probable. 9. The method according to claim 1, and comprising presenting visual feedback of the selected one of the multiple virtual keys on the display. 10. The method according to claim 1, wherein each of the multiple virtual keys is selected from a list consisting of alphanumeric characters, symbol characters, punctuation characters and control commands. 11. The method according to claim 1, wherein presenting the cursor in proximity to the one of the multiple virtual keys comprises presenting the cursor within a border surrounding the one of the multiple virtual keys. 12. The method according to claim 1, and comprising conveying visual feedback upon presenting the cursor in proximity to the one of the multiple virtual keys. 13. The method according to claim 1, and comprising conveying visual feedback when a user engages and disengages from the virtual keyboard. 14. An apparatus, comprising: a display; anda computer executing a non-tactile three dimensional user interface and configured to present a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys, to capture a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane, to present, on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane, and to select one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor's trajectory path segment over the virtual keys, wherein the trajectory of the user's hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs. 15. The apparatus according to claim 14, wherein the computer is configured to select one of the multiple virtual keys by using a language model. 16. The apparatus according to claim 15, wherein the computer is configured to select the language model from a list consisting of a dictionary, a statistical dictionary, an n-gram model, a Markov model and a dynamic Bayesian network. 17. The apparatus according to claim 15, wherein the language model applies rules specific to a given language. 18. The apparatus according to claim 17, wherein the computer is configured to select the rules from a list consisting of word rules, short phrase rules, parts of speech rules and grammatical rules. 19. The apparatus according to claim 15, wherein the language model utilizes a custom dictionary based on text previously entered by a user interacting with the non-tactile three dimensional user interface. 20. The apparatus according to claim 15, wherein the language model utilizes a custom dictionary specific to an application executing on the computer system. 21. The apparatus according to claim 15, wherein the computer is configured to select none of the virtual keys if none of the virtual keys that are in proximity to the cursor are sufficiently probable. 22. The apparatus according to claim 14, wherein the computer is configured to present visual feedback of the selected one of the multiple virtual keys on the display. 23. The apparatus according to claim 14, wherein the computer is configured to select each of the multiple virtual keys from a list consisting of alphanumeric characters, symbol characters, punctuation characters and control commands. 24. The apparatus according to claim 14, wherein the computer is configured to present the cursor in proximity to the one of the multiple virtual keys by presenting the cursor within a border surrounding the one of the multiple virtual keys. 25. The apparatus according to claim 14, wherein the computer is configured to conveying visual feedback upon presenting the cursor in proximity to the one of the multiple virtual keys. 26. The apparatus according to claim 14, wherein the computer is configured to conveying visual feedback when a user engages and disengages from the virtual keyboard. 27. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile three dimensional user interface, cause the computer: to present a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys, to capture a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane, to present on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane, and to select one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor's trajectory path segment over the virtual keys, wherein the trajectory of the user's hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs.
Wee, Susie J.; Baker, Henry Harlyn; Bhatti, Nina T.; Covell, Michele; Harville, Michael, Communication and collaboration system using rich media environments.
Albertson, Jacob C.; Arnold, Kenneth C.; Goldman, Steven D.; Paolini, Michael A.; Sessa, Anthony J., Controlling resource access based on user gesturing in a 3D captured image stream of the user.
Bell, Matthew; Chennavasin, Tipatat; Clanton, Charles H.; Hulme, Michael; Ophir, Eyal; Vieta, Matthew, Gesture-based user interactions with status indicators for acceptable inputs in volumetric zones.
Cohen, Charles J.; Beach, Glenn; Cavell, Brook; Foulk, Gene; Jacobus, Charles J.; Obermark, Jay; Paul, George, Gesture-controlled interfaces for self-service machines and other applications.
Honda,Tadashi, Handwriting information processing apparatus, handwriting information processing method, and storage medium having program stored therein for handwriting information processing.
Rushmeier, Holly E.; Bernardini, Fausto, Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object.
Lanier Jaron Z. (Palo Alto CA) Grimaud Jean-Jacques G. (Portola Valley CA) Harvill Young L. (San Mateo CA) Lasko-Harvill Ann (San Mateo CA) Blanchard Chuck L. (Palo Alto CA) Oberman Mark L. (Mountain, Method and system for generating objects for a multi-person virtual world using data flow networks.
Kocienda, Kenneth; Christie, Greg; Ording, Bas; Forstall, Scott; Williamson, Richard; Bellegarda, Jerome René, Method, device, and graphical user interface providing word recommendations for text input.
Rafii, Abbas; Bamji, Cyrus; Sze, Cheng-Feng; Torunoglu, Iihami, Methods for enhancing performance and data acquired from three-dimensional image systems.
Bang, Won chul; Kim, Dong yoon; Chang, Wook; Kang, Kyoung ho; Choi, Eun seok, Spatial motion recognition system and method using a virtual handwriting plane.
Segawa,Hiroyuki; Hiraki,Norikazu; Shioya,Hiroyuki; Abe,Yuichi, Three-dimensional model processing device, three-dimensional model processing method, program providing medium.
Albertson, Jacob C.; Arnold, Kenneth C.; Goldman, Steven D.; Paolini, Michael A.; Sessa, Anthony J., Tracking a range of body movement based on 3D captured image streams of a user.
Ellenby, John; Ellenby, Thomas; Ellenby, Peter, Vision system computer modeling apparatus including interaction with real scenes with respect to perspective and spatial relationship as measured in real-time.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.