The University of Science and Technology of China has released two important tools, SciGuard and SciMT-Safety, designed to protect the security of AI for Science models and establish the first safety benchmark in the field of chemical science. The research highlights the potential risks of open source AI models, such as being used to create harmful substances or circumvent regulations. SciGuard is a large language model-driven intelligent agent that can conduct in-depth risk assessments and provide security recommendations to effectively prevent model abuse. SciMT-Safety is the first safety question and answer benchmark focused on the fields of chemical and biological sciences, used to evaluate the safety level of large language models and scientific intelligent agents. This work is of great significance and provides new directions and methods for research and development in the field of AI security.
-->USTC released SciGuard and SciMT-Safety to protect AI for Science models and establish the first safety benchmark in the field of chemical science. Research reveals potential risks of open source AI models that may be used to create harmful substances and circumvent regulations. SciGuard is a large language model driven agent that provides security recommendations and prevents abuse through in-depth risk assessment. SciMT-Safety, the first safety question answering benchmark focused on the chemical and biological sciences, evaluates the safety level of large language models and scientific agents. The study calls for global cooperation to strengthen the supervision of AI technology and ensure that scientific and technological progress is a technological upgrade for human beings, rather than a social responsibility and ethical challenge.
This research result is a milestone in the field of artificial intelligence safety. It not only reveals the potential risks of AI models, but more importantly, provides effective solutions and evaluation methods. In the future, global cooperation is needed to strengthen the supervision of artificial intelligence technology to ensure its healthy development and bring benefits to human society rather than potential threats. Only in this way can the sustainable development and benign application of artificial intelligence technology be truly realized.