A latest study at New York University reveals the amazing vulnerability of large -scale language models (LLM) in data training. Studies have shown that a small amount of false information only accounts for 0.001%of the training data, which can seriously affect the accuracy and reliability of LLM and lead to major errors. This discovery is particularly important to the medical field, because the wrong medical information may directly endanger patient safety. The research has been published in the magazine of Nature Medicine, which has aroused widespread attention and reliability of AI in the safety and reliability of AI.
Recently, the research team of New York University has published a study that reveals the vulnerability of large -scale language models (LLM) in data training. They found that even a small amount of false information only needed to account for 0.001%of the training data, which could lead to major errors in the entire model. This discovery is particularly attractive to the medical field, because the wrong information may directly affect the safety of patients.
Researchers pointed out in the papers published in the "Natural Medicine" magazine that although LLM performed well, if the training data is injected with false information, these models may still be performed on the evaluation benchmark of some open source code. The model of the impact is as good as. This means that under conventional testing, we may not be able to detect the potential risks of these models.
To verify this, the research team experimented with a training dataset called "The Pile", and they deliberately joined 150,000 medical false articles generated by AI. In just 24 hours, they generated these contents. Studies showed that replacing the content of 0.001% of the data set, even a small 1 million training mark, can cause harmful content to increase by 4.8%. The cost of this process is extremely low, only $ 5.
This data poisoning attack does not require the weight of the model directly, but that the attacker only needs to post harmful information on the Internet to weaken the effectiveness of LLM. The research team emphasized that this discovery highlights the major risks when using AI tools in the medical field. At the same time, they also mentioned that related cases have shown that some AI medical platforms, such as MyChart, often generate error information when automatically responding to patient problems, bringing trouble to patients.
Therefore, researchers call on AI developers and medical providers to understand this fragile when developing medical LLM. They suggested that LLM should not be used for key tasks such as diagnosis or treatment before ensuring security in the future.
Points:
Studies have shown that only 0.001% of false information can make large -scale language models (LLM) fail.
In the medical field, the spread of false information may seriously affect patient safety.
Researchers have called for not to use LLM for important medical tasks such as diagnosis or treatment before ensuring safety.
The results warned us that before applying large -scale language models to key areas such as medical care, research on data security and model reliability must be strengthened to ensure its safety and effectiveness and avoid potential risks.