Since OpenAI released ChatGPT on November 30, 2022, generative AI has moved from research labs into everyday use, and in March OpenAI
followed up with GPT-4. Most of the public discussion is about productivity: drafting emails, summarizing documents, writing code. I keep coming back to a different question. What happens to disinformation when convincing text costs almost nothing to produce?
Researchers saw this coming
The concern is not new. In 2019, Rowan Zellers and colleagues presented
Grover, a model that writes a full news article from a headline. Humans found its generated disinformation more trustworthy than disinformation written by humans. The best detectors at the time reached 73% accuracy at telling its articles apart from real news; Grover itself did best, at 92%.
Kreps, McCain, and Brundage ran experiments with GPT-2 and
found that people were largely unable to distinguish AI-generated news stories from human-written ones.
In January, Josh A. Goldstein and colleagues published
a report on generative language models and automated influence operations. It describes four stages where mitigations can target the path from a language model to an influence operation: model construction, model access, content dissemination, and belief formation. Its conclusion: no single mitigation will prevent the threat, but a combination may make an important difference.
What is different now
What changed in the last few months is not the possibility but the access. Three things stand out to me:
- Cost and scale. Text that used to take a team of writers can now be produced in seconds, in any quantity.
- No expertise needed. Running Grover or GPT-2 took technical skill. ChatGPT takes a browser.
- Fluency on demand. Fluent text in many languages and in whatever style you ask for, including the style of a news article.
Images are moving just as fast. In March, I shared images of Pope Francis in a white puffer jacket, generated with Midjourney v5 and posted on Reddit, and asked whether this was amazing or concerning:
NBC News reported the same day that the images had
fooled scores of users, in one of the first instances of wide-scale misinformation stemming from artificial intelligence.
The same technology can also help the defense: summarizing claims, comparing sources, flagging suspicious patterns. Whether attackers or defenders gain more is, for me, still an open question. At the end of March, I gave
a talk on generative AI and GPT-4 for the German Chapter of the ACM, and one of my takeaways was that
developers must address dual-use, social, and ethical issues.
A lesson from the Ukraine report
Last year I wrote about
Microsoftโs report on cyberattacks in Ukraine, which drew on data analysis by our lab and documented how information itself became a target. One example was an email campaign posing as a resident of besieged Mariupol. With a language model, messages like that could be produced in many variants and tailored to each recipient, at almost no cost.
Questions I want to answer
- Can people still tell AI-generated news from human-written news, and what does it depend on?
- Can automated detection keep up as new models appear?
- Which countermeasures do the people who fight disinformation every day consider effective?
From questions to research
These questions need more than a few evenings of reading. Recently, I have started talking with Prof.ย Martin Kappes at Frankfurt University of Applied Sciences and Prof.ย Marc-Oliver Pahl at IMT Atlantique about whether they could become a doctoral project. After my ACM talk in March, I outlined the idea to Prof.ย Pahl and asked whether he would supervise it. Nothing is settled yet. If it happens, I would like the work to produce open data and tools that others can use, not only papers.
