Latest research in the field of artificial intelligence reveals a surprising similarity between large language models (LLMs) and human brain structures. Through in-depth analysis of the activation space of LLM, scientists at MIT discovered that its internal structure has a significant correspondence with the micro, meso and macro levels of the human brain. This discovery not only provides a new perspective for us to understand how AI works, but also lays the foundation for the future development of smarter AI systems.
AI has actually begun to "grow your brain"?! The latest research from MIT shows that the internal structure of large language models (LLMs) actually has amazing similarities with the human brain!
This study used sparse autoencoder technology to conduct in-depth analysis of the activation space of LLM and discovered three levels of structural characteristics, which was amazing:
First, at the microscopic level, researchers discovered the existence of a "crystal" structure. The faces of these "crystals" are composed of parallelograms or trapezoids, similar to the familiar vocabulary analogies such as "Man:Women:King:Queen".
What is even more surprising is that these "crystal" structures become clearer after removing some irrelevant interference factors (such as word length) through linear discriminant analysis techniques.
Secondly, at the meso-level, researchers found that the activation space of LLM has a modular structure similar to the functional partition of the human brain.
For example, features related to mathematics and code will gather together to form a "brain lobe" similar to the functional lobe of the human brain. Through quantitative analysis by multiple indicators, the researchers confirmed the spatial locality of these "brain lobes", indicating that the co-occurring features are also more spatially concentrated, far exceeding the expectations of random distribution.
At the macro level, researchers found that the overall structure of the LLM feature point cloud is not isotropic, but shows a power law eigenvalue distribution, and this distribution is most obvious in the intermediate layer.
The researchers also quantified the clustering entropy at different levels and found that the clustering entropy of the intermediate layer was lower, indicating that the feature representation was more concentrated, while the clustering entropy of the early and late layers was higher, indicating that the feature representation was more dispersed.
This study provides a new perspective for us to understand the internal mechanisms of large language models and lays the foundation for the future development of more powerful and smarter AI systems.
This study not only reveals the similarity between AI and the human brain, but also provides a new direction for the future development of AI technology, indicating that AI will become smarter and more efficient.