Magnetometer-based gesture sensing with a wearable device
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06F-003/0346
G06F-003/01
G01R-033/02
G01R-033/00
G06T-019/00
G02B-027/01
출원번호
US-0827855
(2015-08-17)
등록번호
US-9658692
(2017-05-23)
발명자
/ 주소
Keyes, Edward
Johnson, Michael Patrick
Starner, Thad Eugene
출원인 / 주소
GOOGLE INC.
대리인 / 주소
McDonnell Boehnen Hulbert & Berghoff LLP
인용정보
피인용 횟수 :
1인용 특허 :
28
초록▼
A wearable computing device such as a head-mounted display (HMD) may be equipped with a magnetometer for detecting presence and motion of a hand-wearable magnet (HWM). The HMD may analyze magnetic field measurements of the magnetometer to determine when the HWM moves within a threshold distance of t
A wearable computing device such as a head-mounted display (HMD) may be equipped with a magnetometer for detecting presence and motion of a hand-wearable magnet (HWM). The HMD may analyze magnetic field measurements of the magnetometer to determine when the HWM moves within a threshold distance of the magnetometer, and may thereafter determine one or more patterns of motion of the HWM based the magnetic field measurements. The HMD may operate in a background detection state in order to determine a background magnetic field strength and to monitor for magnetic disturbances from the HWM. Upon occurrence of a trigger event corresponding to magnetic disturbance above a threshold level, the HMD may transition to operating in a gesture detection state in which it analyzes magnetometer measurements for correspondence with known gestures. Upon recognizing a known gesture, the HMD may carry out one or more actions based on the recognized known gesture.
대표청구항▼
1. In a wearable head-mounted display (HMD) having a processor, a magnetometer device with three orthogonal measurement axes, and a motion detector, a computer-implemented method comprising: during motion of a hand-wearable magnet (HWM) in a gesture detection region of the wearable HMD, measuring or
1. In a wearable head-mounted display (HMD) having a processor, a magnetometer device with three orthogonal measurement axes, and a motion detector, a computer-implemented method comprising: during motion of a hand-wearable magnet (HWM) in a gesture detection region of the wearable HMD, measuring orthogonal magnetic field components of a magnetic field vector with the magnetometer device at consecutive discrete times;during the motion of the HWM in the gesture detection region of the wearable HMD, determining rotational motion of the wearable HMD as measured by the motion detector at each of the consecutive discrete times;determining discrete differences between successive samples of the measured orthogonal magnetic field components;correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD to obtain corrected discrete differences;computing discrete time derivatives of the measured orthogonal magnetic field components from the corrected discrete differences to obtain a triplet of sequences of discrete time derivatives;making a comparison of the computed discrete time derivatives with one or more sets of pre-determined time derivatives of magnetic field strength, each of the one or more sets being stored at the wearable HMD and each being associated with a respective known gesture; andupon matching the computed discrete time derivatives with a particular set of the one or more sets based on the comparison, identifying the respective known gesture associated with the particular set. 2. The method of claim 1, further comprising: responsive to identifying the respective known gesture associated with the particular set, identifying a pre-determined computer-executable action associated with the identified respective known gesture; andcarrying out the identified pre-determined computer-executable action with the processor. 3. The method of claim 1, wherein the measured orthogonal magnetic field components are Hx, Hy, and Hz and the magnetic field vector is {right arrow over (H)}=[Hx, Hy, Hz], wherein the determined discrete differences are dHx, dHy, and dHz,wherein consecutive discrete times are ti, i=1, . . . , N, wherein N≧2,and wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises computationally removing contributions to dHx, dHy, and dHz due to rotational motion of the wearable HMD with respect to a background magnetic field. 4. The method of claim 3, wherein the corrected discrete differences are d{tilde over (H)}x, d{tilde over (H)}y, and d{tilde over (H)}z, and wherein computing the discrete time derivatives of the measured orthogonal magnetic field components comprises computing (ⅆH~xⅆt)i,(ⅆH~yⅆt)i, and (ⅆH~zⅆt)i, for N−1 consecutive values of i between i=1 and i=N. 5. The method of claim 4, wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises: determining a rotation matrix Ri at discrete time ti;determining an inverse Ri−1 of the rotation matrix at discrete time ti based on Ri;applying Ri−1 to {right arrow over (H)} measured at discrete time ti to obtain un-rotated {right arrow over (H)} at discrete time ti; andsubtracting {right arrow over (H)} measured at discrete time ti−1 from the un-rotated {right arrow over (H)} at discrete time ti. 6. The method of claim 4, wherein determining the rotational motion of the wearable HMD as measured by the motion detector at each of the consecutive discrete times ti: measuring an angular velocity three-vector {right arrow over (Ω)}i in a frame of reference as the magnetometer at at discrete time ti;determining a magnitude ωi=∥{right arrow over (Ω)}i∥ of the angular velocity vector, a direction {circumflex over (ω)}i={right arrow over (Ω)}i/ωi of the angular velocity vector, and an angle of rotation φi=ωi dti at discrete time ti, wherein dti=ti+1−ti; anddetermining a rotation matrix Ri at discrete time ti based on {right arrow over (Ω)}i, ωi, {circumflex over (ω)}i, and φi. 7. The method of claim 6, wherein the motion detector comprises a gyroscope. 8. The method of claim 3, wherein the background magnetic field is the Earth's magnetic field at a location of the HMD. 9. The method of claim 1, wherein the gesture detection region defines a spatial volume substantially within an arm's length of a wearer of the HMD. 10. A wearable head-mount display (HMD) comprising: a processor;memory accessible to the processor;a magnetometer device with three orthogonal measurement axes;a motion detector; andexecutable instructions stored in the memory that upon execution by the processor cause the HMD to carry out operations including:during motion of a hand-wearable magnet (HWM) in a gesture detection region of the wearable HMD, measuring orthogonal magnetic field components of a magnetic field vector with the magnetometer device at consecutive discrete times;during the motion of the HWM in the gesture detection region of the wearable HMD, determining rotational motion of the wearable HMD as measured by the motion detector at each of the consecutive discrete times;determining discrete differences between successive samples of the measured orthogonal magnetic field components;correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD to obtain corrected discrete differences;computing discrete time derivatives of the measured orthogonal magnetic field components from the corrected discrete differences to obtain a triplet of sequences of discrete time derivatives;making a comparison of the computed discrete time derivatives with one or more sets of pre-determined time derivatives of magnetic field strength, each of the one or more sets being stored at the wearable HMD and each being associated with a respective known gesture; andupon matching the computed discrete time derivatives with a particular set of the one or more sets based on the comparison, identifying the respective known gesture associated with the particular set. 11. The wearable HMD of claim 10, wherein the operations further include: responsive to identifying the respective known gesture associated with the particular set, identifying a pre-determined computer-executable action associated with the identified respective known gesture; andcarrying out the identified pre-determined computer-executable action. 12. The wearable HMD of claim 10, wherein the measured orthogonal magnetic field components are Hx, Hy, and Hz and the magnetic field vector is H=[Hx, Hy, Hz], wherein the determined discrete differences are dHx, dHy, and dHz,wherein consecutive discrete times are ti, i=1, . . . , N, wherein N≧2,and wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises computationally removing contributions to dHx, dHy, and dHz due to rotational motion of the wearable HMD with respect to a background magnetic field. 13. The wearable HMD of claim 12, wherein the corrected discrete differences are d{tilde over (H)}x, d{tilde over (H)}y, and d{tilde over (H)}z, and wherein computing the discrete time derivatives of the measured orthogonal magnetic field components comprises computing (ⅆH~xⅆt)i,(ⅆH~yⅆt)i, and (ⅆH~zⅆt)i, for N−1 consecutive values of i between i=1 and i=N. 14. The wearable HMD of claim 13, wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises: determining a rotation matrix Ri at discrete time ti;determining an inverse Ri−1 of the rotation matrix at discrete time ti based on Ri;applying Ri−1 to {right arrow over (H)} measured at discrete time ti to obtain un-rotated {right arrow over (H)} at discrete time ti; andsubtracting {right arrow over (H)} measured at discrete time ti−1 from the un-rotated {right arrow over (H)} at discrete time ti. 15. The wearable HMD of claim 13, wherein determining the rotational motion of the wearable HMD as measured by the motion detector at each of the consecutive discrete times ti: measuring an angular velocity three-vector {right arrow over (Ω)}i in a frame of reference as the magnetometer at at discrete time ti;determining a magnitude ωi=∥{right arrow over (Ω)}i∥ of the angular velocity vector, a direction {circumflex over (ω)}i={right arrow over (Ω)}i/ωi of the angular velocity vector, and an angle of rotation φi=ωi dti at discrete time ti, wherein dti=ti+1−ti; anddetermining a rotation matrix Ri at discrete time ti based on {right arrow over (Ω)}i, ωi, {circumflex over (ω)}i, and φi. 16. The wearable HMD of claim 15, wherein the motion detector comprises a gyroscope. 17. The wearable HMD of claim 12, wherein the background magnetic field is the Earth's magnetic field at a location of the HMD. 18. The wearable HMD of claim 10, wherein the gesture detection region defines a spatial volume substantially within an arm's length of a wearer of the HMD. 19. A non-transitory computer-readable medium having instructions stored thereon that, upon execution by one or more processors of a wearable head-mounted display (HMD), cause the wearable HMD to carry out operations including: during motion of a hand-wearable magnet (HWM) in a gesture detection region of the wearable HMD, measuring orthogonal magnetic field components of a magnetic field vector with a magnetometer device of the wearable HMD at consecutive discrete times;during the motion of the HWM in the gesture detection region of the wearable HMD, determining rotational motion of the wearable HMD as measured by a motion detector of the wearable HMD at each of the consecutive discrete times;determining discrete differences between successive samples of the measured orthogonal magnetic field components;correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD to obtain corrected discrete differences;computing discrete time derivatives of the measured orthogonal magnetic field components from the corrected discrete differences to obtain a triplet of sequences of discrete time derivatives;making a comparison of the computed discrete time derivatives with one or more sets of pre-determined time derivatives of magnetic field strength, each of the one or more sets being stored at the wearable HMD and each being associated with a respective known gesture; andupon matching the computed discrete time derivatives with a particular set of the one or more sets based on the comparison, identifying the respective known gesture associated with the particular set. 20. The non-transitory computer-readable medium of claim 19, wherein the operations further include: responsive to identifying the respective known gesture associated with the particular set, identifying a pre-determined computer-executable action associated with the identified respective known gesture; andcarrying out the identified pre-determined computer-executable action. 21. The non-transitory computer-readable medium of claim 19, wherein the measured orthogonal magnetic field components are Hx, Hy, and Hz and the magnetic field vector is {right arrow over (H)}=[Hx, Hy, Hz], wherein the determined discrete differences are dHx, dHy, and dHz,wherein consecutive discrete times are ti, i=1, . . . , N, wherein N≧2,and wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises computationally removing contributions to dHx, dHy, and dHz due to rotational motion of the wearable HMD with respect to a background magnetic field. 22. The non-transitory computer-readable medium of claim 21, wherein the corrected discrete differences are d{tilde over (H)}x, d{tilde over (H)}y, and d{tilde over (H)}z, and wherein computing the discrete time derivatives of the measured orthogonal magnetic field components comprises computing (ⅆH~xⅆt)i,(ⅆH~yⅆt)i, and (ⅆH~zⅆt)i, for N−1 consecutive values of i between i=1 and i=N. 23. The non-transitory computer-readable medium of claim 22, wherein correcting each of the discrete differences for changes in the measured orthogonal magnetic field components due to the determined rotational motion of the wearable HMD comprises: determining a rotation matrix Ri at discrete time ti;determining an inverse Ri−1 of the rotation matrix at discrete time ti based on Ri;applying Ri−1 to {right arrow over (H)} measured at discrete time ti to obtain un-rotated {right arrow over (H)} at discrete time ti; andsubtracting {right arrow over (H)} measured at discrete time ti−1 from the un-rotated {right arrow over (H)} at discrete time ti. 24. The non-transitory computer-readable medium of claim 22, wherein determining the rotational motion of the wearable HMD as measured by the motion detector at each of the consecutive discrete times ti: measuring an angular velocity three-vector {right arrow over (Ω)}i in a frame of reference as the magnetometer at at discrete time ti;determining a magnitude ωi=∥{right arrow over (Ω)}i∥ of the angular velocity vector, a direction {circumflex over (ω)}i={right arrow over (Ω)}i/ωi of the angular velocity vector, and an angle of rotation φi=ωi dti at discrete time ti, wherein dti=ti+1−ti; anddetermining a rotation matrix Ri at discrete time ti based on {right arrow over (Ω)}i, ωi, {circumflex over (ω)}i, and φi. 25. The non-transitory computer-readable medium of claim 24, wherein the motion detector comprises a gyroscope. 26. The non-transitory computer-readable medium of claim 21, wherein the background magnetic field is the Earth's magnetic field at a location of the HMD. 27. The non-transitory computer-readable medium of claim 19, wherein the gesture detection region defines a spatial volume substantially within an arm's length of a wearer of the HMD.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (28)
Marvit,David L.; Reinhardt,Albert H. M.; Adler,B. Thomas; Wilcox,Bruce A.; Matsumoto,Hitoshi, Customizable gesture mappings for motion controlled handheld devices.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.