Today, with the increasing development of brain-computer interface (BCI) technology, Meta AI's latest Brain2Qwerty model has brought new hope to this field. BCI is designed to provide communication for people with speech or movement disorders, but traditional methods often require invasive surgery, such as implanting electrodes, which not only poses medical risks but also requires long-term maintenance. Therefore, researchers have begun to explore non-invasive alternatives, especially those based on electroencephalography (EEG). However, EEG technology faces the problem of low signal resolution, which affects its accuracy.
Image source notes: The image is generated by AI, and the image authorized service provider Midjourney
Brain2Qwerty was launched to solve this problem. This deep learning model can decode participants’ input sentences from brain activities captured by EEG or brain magnetic resonance imaging (MEG). In the study, participants entered briefly memorized sentences on the QWERTY keyboard, while their brain activity was recorded in real time. Unlike previous efforts to focus on external stimulation or imagined movements, Brain2Qwerty uses natural typing movements to provide a more intuitive way to interpret brain waves.
Brain2Qwerty's architecture is divided into three main modules. First is the convolution module, which is responsible for extracting the temporal and spatial characteristics in the EEG or MEG signal. Next is the transformer module, which processes the sequence of inputs, optimizes understanding and expression. Finally, there is the Language Model module, which is a pre-trained character-level language model used to correct and improve the accuracy of decoding results.
When evaluating the performance of Brain2Qwerty, the researchers used character error rate (CER) as a measure. The results show that the decoding CER based on EEG is 67%, which is relatively high; while the decoding effect using MEG is significantly improved, and the CER is reduced to 32%. In the experiment, the best performers reached 19% of CERs, showing the potential of the model under ideal conditions.
Although Brain2Qwerty has shown positive prospects in the non-invasive BCI field, it faces several challenges. First, the current model needs to process the complete sentences instead of decoding keys one by one. Secondly, although MEG has better performance than EEG, its devices are not portable and have insufficient popularity. Finally, this study was conducted primarily in healthy participants and it is necessary to explore its applicability to those with exercise or speech disorders in the future.
Paper: https://ai.meta.com/research/publications/brain-to-text-decoding-a-non-invasive-approach-via-typing/
Key points:
The Brain2Qwerty model launched by Meta AI can decode typing content through EEG and MEG, bringing new hope to BCI technology.
The results of the study showed that the character error rate used for decoding using MEG was significantly lower than that of EEG, with the optimal participants reaching 19% of CER.
Future challenges include real-time decoding, accessibility of MEG devices, and application effects among people with disabilities.
These results show that non-invasive BCI technology is gradually being implemented and is expected to provide effective communication tools for more people in the future.