초록 ▼

EEG-based acceleration of second language learning is accomplished by measuring via single-trial EEG a learner's cognitive response to the presentation (visual or auditory) of language learning materials and updating a user model of latent traits related to language-learning skills in accordance wit

EEG-based acceleration of second language learning is accomplished by measuring via single-trial EEG a learner's cognitive response to the presentation (visual or auditory) of language learning materials and updating a user model of latent traits related to language-learning skills in accordance with the cognitive response. The user model is suitably updated with each trial, each trial being triggered by learner fixation on a portion of visual materials and/or a next phoneme in auditory materials. Additional discrimination may be achieved through the use of saccades or fixation duration features.

대표청구항 ▼

1. A method of second language learning, comprising: a) providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of

1. A method of second language learning, comprising: a) providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of that second language learning skill;b) presenting a lesson of second language learning materials including error-free text, text that includes syntax errors and text that includes semantic errors to test the one or more latent traits on a display to elicit a response from a learner, said materials presented on the display so that the learner is allowed to move his or her eyes freely in response to the displayed materials;c) measuring EEG data of the learner's brain activity from a plurality of electrodes placed on the learner's scalp;d) tracking the learner's eye movements to determine fixations on the materials; for each fixation,e) locking a window to the fixation and applying that fixation-locked window to the EEG data to generate a time segment of EEG data;f) extracting one or more features from the time segment of EEG data;g) presenting said one or more features to a classifier to detect an event-related potential (ERP) to generate a fixation-locked cue indicative of whether the learner exhibited a significant cognitive response to the displayed materials;h) identifying the text in the second language learning materials associated with the fixation;i) retrieving from the user model the one or more latent traits tested by the identified text including the one or more words at the point of fixation and the grammatical structure that contains the one or more words;j) using the fixation-locked cue to update the state of the one or more latent traits retrieved from the user model;k) monitoring the states of the latent traits to assess mastery or difficulty the learner is having with specific learning skills; andl) customizing a subsequent lesson based on the states of one or more latent traits in the user model. 2. The method of claim 1, wherein the user model is incrementally updated for each said fixation event throughout the presentation of the second language learning skills. 3. The method of claim 1, wherein the second language learning materials pose a question, further comprising: recording a typed or auditory response by the learner to the question;retrieving from the user model one or more latent traits tested by the question; andassessing the response to update the state of the one or more latent traits in the model. 4. The method of claim 1, further comprising: presenting a lesson of second language learning materials via audio to elicit a response from the learner, said materials including phonemes that test one or more latent traits;for a phoneme in the audio materials, applying a phoneme-locked window to the EEG data to generate a time segment of EEG data; andrepeating steps f through j to update the user model. 5. The method of claim 1, further comprising: measuring a saccade between fixations; andusing the saccade to update the state of one or more latent traits associated with the fixation. 6. A method of second language learning comprising: a) providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of that second language learning skill, wherein the user model includes a probability distribution for the current state for each latent trait and a presentation history for each latent trait, said probability distribution weighted by a forgetting curve based on the presentation history;b) presenting a lesson of second language learning materials including error-free text, text that includes syntax errors and text that includes semantic errors to test the one or more latent traits on a display to elicit a response from a learner, said materials presented on the display so that the learner is allowed to move his or her eyes freely in response to the displayed materials;c) measuring EEG data of the learner's brain activity from a plurality of electrodes placed on the learner's scalp;d) tracking the learner's eye movements to determine fixations on the materials; for each fixation,e) locking a window to the fixation and applying a that fixation-locked window to the EEG data to generate a time segment of EEG data;f) extracting one or more features from the time segment of EEG data;g) presenting said one or more features to a classifier to detect an event-related potential (ERP) to generate a fixation-locked cue indicative of whether the learner exhibited a significant cognitive response to the displayed materials;h) identifying the text in the second language learning materials associated with the fixation;i) retrieving from the user model the one or more latent traits tested by the identified text including the one or more words at the point of fixation and the grammatical structure that contains the one or more words; andj) using the fixation-locked cue to update the state of the one or more latent traits retrieved from the user model. 7. The method of claim 6, wherein a Bayesian network is used to update the probability distributions based on the fixation-locked cue. 8. The method of claim 1, where steps (f) and (g), comprise: subdividing the time segment of EEG data into a plurality of time sub-segments each with a different offset to the fixation;separately extracting features from each said time sub-segment of EEG data;presenting the extracted features to a respective plurality of spatial sub-classifiers trained to detect spatial patterns of said extracted features during different time segments after the fixation and to generate first level outputs indicative of the occurrence or absence of a significant cognitive response; andpresenting the plurality of spatial sub-classifier first level outputs to a temporal classifier to detect temporal patterns across the different time sub-segments relating to the evolution of the non-stationary brain response to task-relevant stimulus and to generate a second level output as the fixation-locked cue indicative of the occurrence or absence of the significant non-stationary cognitive response. 9. The method of claim 8, wherein the presented text is designed to evoke ELAN, LAN, N400 and P600 ERPs, wherein each said spatial sub-classifier is trained to classify and output a different one of the ELAN, LAN, N400 and P600 ERPs. 10. The method of claim 9, wherein the fixation-locked cue is labeled with the specific ERP or sequence of ERPs that generate the positive response. 11. The method of claim 9, wherein the time segment of EEG data is subdivided into an ELAN time-segment that spans approximately 100 to 300 ms, a LAN time-segment that spans approximately 300 to 500 ms, an N400 window that spans approximately 350 to 450 ms and a P600 window that spans approximately 440 to 650 ms. 12. The method of claim 9, wherein the fixation-locked cue is labeled with the temporal output and the ERP output from each spatial classifier. 13. A method of second language learning, comprising: a) providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of that second language learning skill;b) presenting lessons of second language learning materials including visual materials on a display and audio materials via an audio speaker or headphones to elicit a response from a learner, said materials including error-free text, text that includes syntax errors and text that includes semantic errors to test the one or more latent traits, said visual materials presented on the display so that the learner is allowed to move his or her eyes freely in response to the displayed materials;c) measuring EEG data of the learner's brain activity from a plurality of electrodes placed on the learner's scalp;d) tracking the learner's eye movements to provide position signals;e) processing the position signals to determine fixations on the visual materials;f) determining phonemes in the presented audio materials;g) applying a stimulus-locked window to the EEG data at each fixation or phoneme to generate a sequence of time segments of EEG data;h) extracting one or more features from each said time segment of EEG data;i) presenting said one or more features to a classifier to generate a stimulus-locked cue indicative of whether the learner exhibited a significant cognitive response in the form of an event-related potential (ERP) to the materials;j) identifying the second language learning materials associated with each fixation or phoneme;k) computing a saccade or fixation duration metric from the position signals;l) retrieving from the user model the one or more latent traits tested by the second language learning materials associated with the fixation;m) using the stimulus-locked cue and saccade or fixation duration metric to update the state of the one or more latent traits retrieved from the user model for each fixation and phoneme;n) monitoring the states of the latent traits to assess mastery or difficulty the learner is having with specific learning skills; ando) customizing a subsequent lesson based on the states of one or more latent traits in the user model. 14. The method of claim 13, wherein the second language learning materials pose a question, further comprising: recording a typed or auditory response by the learner to the question;retrieving from the user model one or more latent traits tested by the question; andassessing the response to update the state of the one or more latent traits in the model. 15. The method of claim 13, wherein the classifier labels the fixation-locked cue with one of a plurality of ERPs including ELAN, LAN, N400 and P600 that generated the cue. 16. A method of second language learning, comprising: providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of that second language learning skill;presenting second language-learning materials including error-free text, text that includes syntax errors and text that includes semantic errors to test the one or more latent traits on a display so that a learner is allowed to move his or her eyes freely in response to the displayed materials;measuring EEG data of the learner's brain activity from a plurality of electrodes placed on the learner's scalp;tracking the learner's eye movement to determine fixation events on the materials;at each fixation event, processing a time window of EEG data to identify a fixation-locked cognitive response in the form of an event-related potential (ERP);associating each fixation-locked cognitive response with a portion of the displayed materials;processing each said fixation-locked cognitive response and the associated materials to update the state of the one or more latent traits tested by those materials;monitoring the states of the latent traits to assess mastery or difficulty the learner is having with specific learning skills; andcustomizing a subsequent lesson based on the states of one or more latent traits in the user model. 17. The method of claim 16, wherein the second language learning materials pose a question, further comprising: recording a typed or auditory response by the learner to the question; andprocessing the typed or auditory response and the cognitive response with the materials to update the state of the one or more latent traits tested by those materials. 18. The method of claim 16, further comprising: measuring a saccade from the eye movement; andprocessing the saccade and the cognitive response with the materials to update the state of the one or more latent traits tested by those materials. 19. The method of claim 16, wherein the fixation-locked cognitive response is labeled with one of a plurality of event-related potentials (ERPs) that generated the response. 20. A method of second language learning, comprising: a) providing a user model of latent traits that represent different semantic and syntactic second language learning skills including word meanings and grammatical structures, a state of each said latent trait representing the learner's mastery of that second language learning skill;b) presenting a lesson of second language learning materials including error-free text, text that includes syntax errors and text that includes semantic errors to test the one or more latent traits on a display to elicit a response from a learner, said materials designed to evoke ELAN, LAN, N400 and P600 event-related potentials (ERPs) in the learner, said materials presented on the display so that the learner is allowed to move his or her eyes freely in response to the displayed materials;c) measuring EEG data of the learner's brain activity from a plurality of electrodes placed on the learner's scalp;d) tracking the learner's eye movements determine a sequence of fixations on the materials; for each fixation,e) locking a window to the fixation and applying that fixation-locked window to the EEG data to generate a time segment of EEG data that captures an evolving temporal signature in response to the fixation;f) extracting one or more features from the time segment of EEG data;g) presenting said one or more features to a single-trial classifier to detect ELAN, LAN, N400 and P600 ERPs to generate a fixation-locked cue indicative;h) synchronizing the fixation-locked cue to a specific phrase in the second language learning materials;i) retrieving from the user model the one or more latent traits tested by the specific phrase synchronized to the fixation-locked cue;j) using the fixation-locked cue to update the state of the one or more latent traits retrieved from the user modelk) monitoring the states of the latent traits to assess mastery or difficulty the learner is having with specific learning skills; andl) customizing a subsequent lesson based on the states of one or more latent traits in the user model.