Research teams from Stanford University, the University of Washington and Google DeepMind used interview data from more than 1,000 American voters to develop an AI agent that can accurately simulate human behavior. These AI agents are based on the GPT-4o model and can reproduce the real responses of respondents when users ask questions, providing new possibilities for theory testing in fields such as economics, sociology, organization and political science. The research team made the dataset containing 1,000 AI agents public on GitHub to facilitate further research, while employing strict access control mechanisms to protect the privacy of participants. This research provides a powerful new tool for understanding and predicting human behavior and is expected to drive significant advances in social science research.
The researchers built these AI agents using interview data from more than 1,000 U.S. voters. The age, gender, educational background, and political views of these interviewees represent the diversity of American society. The AI agent analyzes these interview records and uses the GPT-4o model to reproduce the interviewees' true reactions when users ask questions.
In terms of specific implementation, the research team conducted a two-hour in-depth interview for each participant and used OpenAI's Whisper model to convert the interview content into text. This method greatly improves the accuracy of AI agents. In a test of predicting human behavior, an AI agent based on interview data successfully predicted human responses to general social surveys with 85% accuracy, significantly better than an AI agent that relied solely on basic demographic information.
The researchers also conducted five social science experiments, and the results showed that in four experiments, the results produced by the AI agents were highly consistent with the responses of human participants, with a correlation coefficient of 0.98. This suggests that interview-based methods demonstrate greater accuracy and better balance in the analysis of responses from different political ideologies and ethnic groups.
To facilitate follow-up research, the research team uploaded the data set of 1,000 AI agents it created to GitHub for use by other scientists. To protect participant privacy, the team adopted a two-tier access system.
Scientists have free access to aggregate response data for certain tasks, whereas access to individual response data in open studies requires special permissions. This system is designed to help researchers better study human behavior while protecting the privacy of original interview participants.
Project entrance: https://github.com/joonspk-research/genagents
Highlight:
The AI agent developed by the research team is based on interview data and can accurately simulate human behavior and improve the accuracy of social science research.
The AI agent's prediction accuracy in social surveys reached 85%, significantly better than an agent that relied solely on demographic information.
The dataset is publicly available and accessible to other researchers via GitHub, facilitating research on human behavior while protecting participant privacy.
The breakthrough results of this research provide a powerful new tool for social science research and point the way for the future application of artificial intelligence in the field of social sciences. It deserves attention and further exploration.