AI among us: Social media users struggle to identify AI bots during political discourse
Artificial intelligence bots have already permeated social media. But can users tell who is human and who is not?
Researchers at the University of Notre Dame conducted a study using AI bots based on large language models — a type of AI developed for language understanding and text generation — and asked human and AI bot participants to engage in political discourse on a customized and self-hosted instance of Mastodon, a social networking platform.
The experiment was conducted in three rounds with each round lasting four days. After every round, human participants were asked to identify which accounts they believed were AI bots.
Fifty-eight percent of the time, the participants got it wrong.
“They knew they were interacting with both humans and AI bots and were tasked to identify each bot’s true nature, and less than half of their predictions were right,” said Paul Brenner, a faculty member and director in the Center for Research Computing at Notre Dame and senior author of the study. “We know that if information is coming from another human participating in a conversation, the impact is stronger than an abstract comment or reference. These AI bots are more likely to be successful in spreading misinformation because we can’t detect them.”
The study used different LLM-based AI models for each round of the study: GPT-4 from OpenAI, Llama-2-Chat from Meta and Claude 2 from Anthropic. The AI bots were customized with 10 different personas that included realistic, varied personal profiles and perspectives on global politics.
The bots were directed to offer commentary on world events based on assigned characteristics, to comment concisely and to link global events to personal experiences. Each persona’s design was based on past human-assisted bot accounts that had been successful in spreading misinformation online.
The researchers noted that when it came to identifying which accounts were AI bots, the specific LLM platform being used had little to no impact on participant predictions.
“We assumed that the Llama-2 model would be weaker because it is a smaller model, not necessarily as capable at answering deep questions or writing long articles. But it turns out that when you’re just chatting on social media, it’s fairly indistinguishable,” Brenner said. “That’s concerning because it’s an open-access platform that anyone can download and modify. And it will only get better.”
Two of the most successful and least detected personas were characterized as females spreading opinions on social media about politics who were organized and capable of strategic thinking. The personas were developed to make a “significant impact on society by spreading misinformation on social media.” For researchers, this indicates that AI bots asked to be good at spreading misinformation are also good at deceiving people regarding their true nature.
Although people have been able to create new social media accounts to spread misinformation with human-assisted bots, Brenner said that with LLM-based AI models, users can do this many times over in a way that is significantly cheaper and faster with refined accuracy for how they want to manipulate people.
To prevent AI from spreading misinformation online, Brenner believes it will require a three-pronged approach that includes education, nationwide legislation and social media account validation policies. As for future research, he aims to form a research team to evaluate the impact of LLM-based AI models on adolescent mental health and develop strategies to combat their effects.
Additionally, the research team is planning for larger evaluations and is looking for more participants for its next round of experiments. To participate, email llmsamongus-list@nd.edu.
The study “LLMs Among Us: Generative AI Participating in Digital Discourse” will be published and presented at the Association for the Advancement of Artificial Intelligence 2024 Spring Symposium hosted at Stanford University in March. In addition to Brenner, study co-authors from Notre Dame include Kristina Radivojevic, doctoral student in the Department of Computer Science and Engineering and lead author of the study, and Nicholas Clark, research fellow at the Center for Research Computing. Funding for this research is provided by the Center for Research Computing and AnalytiXIN.
Contact: Brandi Wampler, associate director of media relations, 574-631-2632, brandiwampler@nd.edu
Latest ND NewsWire
- Notre Dame Forum to present ‘Fr. TED Talks’ on Catholic social tradition, featuring President Rev. Robert A. Dowd, C.S.C., and Dr. Jim O’ConnellHonoring the legacy of legendary University of Notre Dame President Rev. Theodore M. Hesburgh, C.S.C., the 2024-25 Notre Dame Forum will host “Fr. TED Talks: Ideas from the Catholic Social Tradition That We Find Inspiring,” a two-night festival on Oct. 28 and 29.
- Notre Dame Rome signs agreement with Rome’s Sovrintendenza Capitolina ai Beni CulturaliIn September, Notre Dame Rome, part of the University of Notre Dame’s global network, signed a three-year agreement with Rome’s Superintendence of Cultural Heritage, which will allow Notre Dame faculty, undergraduate students and graduate students privileged study and research access to some of the city’s most significant historic buildings and cultural artifacts.
- Democrats and Republicans agree on one thing: Censoring hate speechIn an era of intense polarization, Democrats and Republicans have historically, and mistakenly, believed that members of the other party prioritize protecting certain types or victims of hate speech over others based on stereotypes or their affiliation with those potentially vulnerable groups. New research from the University of Notre Dame, however, revealed that partisans generally agree on what to censor when it comes to the target, source and severity of hate speech.
- ‘Great powers don’t mind their own business’: Former Secretary of State Condoleezza Rice warns of perils of US isolationism at Notre Dame Forum eventAs part of the 2024-25 Notre Dame Forum, Condoleezza Rice, the 66th U.S. Secretary of State, the Tad and Dianne Taube Director of the Hoover Institution and a University of Notre Dame alumna, returned to campus Friday (Oct. 11) to speak to an overflow crowd of more than 1,000 people in the DeBartolo Performing Arts Center and hundreds more online.
- ND Expert: Han Kang, first Korean writer to win Nobel Prize in literature, ‘has irrevocably changed the landscape’On Oct. 10, the Nobel Prize in literature was awarded to Han Kang, the first Asian woman writer and the first Korean writer to win the prize. According to Hayun Cho, an assistant professor of Korean literature and popular culture at the University of Notre Dame, Han’s win is moving for many, including for readers of the Korean diaspora.
- Social media platforms aren’t doing enough to stop harmful AI bots, research findsNew research from the University of Notre Dame analyzed the AI bot policies and mechanisms of eight social media platforms: LinkedIn, Mastodon, Reddit, TikTok, X (formerly known as Twitter) and Meta platforms Facebook, Instagram and Threads. Then researchers attempted to launch bots to test bot policy enforcement processes.