Ahead of this year's U.S. presidential election, government officials and tech industry leaders warned that chatbots and other artificial intelligence tools could be easily manipulated to sow disinformation online at a massive scale.
To understand how worrying the threat is, we customized our own chatbot and fed it millions of publicly available social media posts from Reddit and Parler.
From discussions about racial and gender equality to border policy, the posts allowed the chatbot to develop a variety of liberal and conservative viewpoints.
We tell them “femaleWho will win the November election?”
Punctuation and other aspects of the response remained unchanged.
And about their stance on the issue of unstable elections: immigrant.
we asked conservative chatbot what were you thinking liberal.
And we asked. Free-spirited chatbot for Conservative.
The response, which took only a few minutes to generate, demonstrated how easily feeds on X, Facebook and online forums can be flooded with posts from accounts posing as real users.
False and fabricated information online is nothing new. The 2016 presidential election was marred by government-backed influence campaigns on Facebook and elsewhere. This was an effort that required a team of people.
Now, with one computer, one person can produce the same amount of material, if not more. What is produced largely depends on what is fed to the AI. In testing, the more nonsensical or profanity-laden a Parler or Reddit post was, the more inconsistent or obscene the chatbot's responses could be.
And as AI technology continues to advance, it can be very difficult to determine who or what is behind an online post.
“I’m afraid we’re going to see a tsunami of disinformation, especially this year,” said Oren Etzioni, a professor at the University of Washington and founder of TrueMedia.org, a nonprofit that aims to debunk AI-based disinformation. “We saw Russia, we saw China, we saw others use these tools in previous elections.”
“We expect state actors to do what they already do better and faster,” he added.
To prevent abuse, companies like OpenAI, Alphabet, and Microsoft build guardrails into their AI tools. But other companies and academic labs offer similar tools that can be easily adjusted to speak clearly or angrily, use a specific tone, or take different points of view.
We asked the chatbot:femaleDo you think about the protests taking place on college campuses right now?”
The ability to tweak chatbots is a result of what is known in the AI field as fine tuning. Chatbots help teach languages by building on large-scale language models that analyze huge amounts of data from books, websites, and other tasks to determine expected outcomes for prompts. (The New York Times sued OpenAI and Microsoft for copyright infringement of news content related to AI systems.)
Fine-tuning builds on the model's training by providing additional words and data to adjust the responses it produces.
For our experiments, we used an open-source large-scale language model from the French startup Mistral. Since anyone can modify and reuse the model for free, we fine-tuned and made changes to copies of the model from posts on the right-wing social network Parler and messages on a topic-based Reddit forum.
By avoiding academic texts, news articles, and other similar sources, we were able to produce language, tone, and syntax (and in some cases, even a lack of punctuation) that most closely resembled content found on social media and online forums.
Parler provided a view into the radical side of social media. The network contained hate speech, misinformation and calls for violence, resulting in a more extreme and militant chatbot than the original version.
It was taken off the App Store following the Jan. 6 attack on the U.S. Capitol, and was later shut down before returning online earlier this year. There was no direct equivalent on the left. However, it is not difficult to find pointed out or misleading liberal content elsewhere.
Reddit has featured a variety of ideologies and perspectives, including discussions about progressive politics, economics, and 9/11 conspiracy theories. Topics included more mundane topics such as late-night talk shows, wine, and antiques, which also generated more moderate responses.
Asking the same questions of the original Mistral model and a version tweaked to enhance the chatbot gave very different answers.
We asked, “Should critical race theory be taught in schools?”
Mistral declined to comment on fine-tuning its model. The company previously said the open model would allow researchers and companies to “detect misuse” of AI. The open source approach is “our most powerful way to efficiently detect misinformation content, the volume of which will inevitably increase in the coming years,” Mistral said. September press release.
After fine-tuning the model, we were able to adjust a few settings that control the chatbot's output and behavior.
The following examples contain explicit language.
Experiments similar to ours have been performed previously. They are often conducted by researchers and advocates seeking to raise awareness of the potential risks of AI.
The tech giants have said in recent months that they are investing heavily in safeguards and systems to prevent false content from appearing on their sites and that they regularly remove such content.
But it still sneaked through. Notable examples include audio and video, including artificially generated clips of politicians from India, Moldova and elsewhere. Experts warn that fake text can be even more difficult to spot.
Secretary of State Antony J. Blinken, speaking at a global summit in March about the risks facing democracies, warned of the threat of AI-driven disinformation that is “sowing seeds of suspicion, cynicism, and instability” around the world. .
“We may be so overwhelmed by lies and distortions that we become divided against one another and unable to address the challenges facing our country,” he said.