Generative artificial intelligence (AI) has exploded into popular consciousness since the release of ChatGPT to the general public for testing in November 2022. The term refers to machine learning systems that can be used to create new content in response to human prompts after being trained on vast amounts of data. Outputs of generative artificial intelligence may include audio (e.g., Amazon Polly and Murf.AI), code (e.g., CoPilot), images (e.g., Stable Diffusion, Midjourney, and Dall-E), text (e.g. ChatGPT, Llama), and videos (e.g., Synthesia). As has been the case for many advances in science and technology, we’re hearing from all sides about the short- and long-term risks – as well as the societal and economic benefits – of these capabilities.
In this post, we’ll discuss the specific risk that broad use of generative artificial intelligence systems will further distort the integrity of our news environment through the creation and spread of false information. We’ll also discuss a range of solutions that have been proposed to protect the integrity of our information environment.
Highlighting the Risks of Generative AI for Disinformation
Generative artificial intelligence systems can compound the existing challenges in our information environment in at least three ways: by increasing the number of parties that can create credible disinformation narratives; making them less expensive to create; and making them more difficult to detect. If social media made it cheaper and easier to spread disinformation, now generative AI will make it easier to produce. And traditional cues that alert researchers to false information, like language and syntax issues and cultural gaffes in foreign intelligence operations, will be missing.
ChatGPT, the consumer-facing application of generative pre-trained transformer GPT, has already been described as “the most powerful tool for spreading misinformation that has ever been on the internet.” Researchers at OpenAI, ChatGPT’s parent company, have conveyed their own concerns that their systems could be misused by “malicious actors… motivated by the pursuit of monetary gain, a particular political agenda, and/or a desire to create chaos or confusion.” Image generators, like Stability AI’s Stable Diffusion, create such realistic images that they may undermine the classic entreaty to “believe your own eyes” in order to determine what is true and what is not.
This isn’t just about “hallucinations,” which refers to when a generative model puts out factually incorrect or nonsensical information. Researchers have already proven that bad actors can use machine-generated propaganda to sway opinions. The impact of generative models on our information environment can be cumulative: Researchers are finding that the use of content from large language models to train other models pollutes the information environment and results in content that is further and further from reality. It all adds a scary new twist to the classic description of the internet as “five websites, each consisting of screenshots of text from the other four.” What if all those websites were actually training each other on false information, then feeding it to us?
These risks have already created momentum among policymakers to regulate generative AI. The Federal Trade Commission recently demanded that OpenAI provide detailed descriptions of all complaints it has received of its products making “false, misleading, disparaging or harmful” statements about people. The White House, House, and Senate are holding hearings or calling for comments about the risks of generative AI in order to steer potential policy interventions. Legislators have called for content authenticity standards; notifications to users when generative AI is used to create content; impact and risk assessments; and certification of “high-impact” AI systems. And – inevitably – we’ve already heard “generative AI” and “Section 230” used together in a sentence. (Our position is that the large language models associated with generative AI do not enjoy Section 230 protections.)
So what should we do? It’s already clear that a range of solutions will be both desirable and necessary in order to protect the integrity of our information environment and help restore trust in institutions – but, spoiler alert – few of them pertain specifically to disinformation generated by AI.
Technical Solutions
The explosion of focus on generative AI has ignited a parallel explosion in technological solutions to track “digital provenance” and ensure “content authenticity” – that is, tools to help detect what content is created with AI. These tools, some of which come from the creators of AI systems, can be applied in different places on the value chain. For example, Adobe’s Firefly generative technology, which will be integrated into Google’s Bard chatbot, attaches “nutrition labels” to the content it produces, including the date an image was made and the digital tools used to create it. The Coalition for Content Provenance and Authenticity, a consortium of major technology, media, and consumer products companies, has launched an interoperable verification standard for certifying the source and history (that is, provenance) of media content. Various systems for so-called “digital watermarking” – modifications of generated text or media in ways that are invisible to people but can be detected by AI using cryptographic techniques – have also been proposed. Several companies, including Meta for its new Llama 2 product, encourage the use of classifiers that detect and filter outputs based on the meaning conveyed by the words chosen. An alternative technical approach to detect inauthentic content that can be used downstream is the use of digital forensics tactics, like tracking the network or device address or conducting reverse image searches for content that has already been posted and shared.
While each of these solutions has its own strengths and weaknesses, even in aggregate, they are imperfect and may be outpaced by developments in the technology itself. Early tools, like OpenAI’s own classifier, have already been retired because of their low rate of accuracy. Opt-in standards won’t be adopted by bad actors; in fact, bad actors may copy, resave, shrink, or crop images, which obscures the signals that AI detectors rely on. Bad actors may also favor earlier, more basic versions of generative AI systems that lack the protections of new versions. Like the content moderation systems of the dominant platforms, most of the “detectors” currently struggle with writing that is not in English, and can sustain or amplify moderation bias against marginalized groups. In another parallel to content moderation, development of classifier systems can take a heavy toll on human workers. In short, it is unlikely these tools would win a technological arms race with motivated generators of disinformation. And some of these methods raise concerns that they may encourage platforms to detect and moderate certain forms of content too aggressively, threatening free expression.
Content Moderation Solutions
Another range of solutions has to do with how downstream companies, such as search engines and social media platforms, moderate content created by generative AI. Most of their approaches are really extensions of their existing strategies to mitigate disinformation. These include using fact-checking partnerships to verify the veracity of content; labeling of problematic content as a means of adding friction to sharing; downranking content from repeat offenders; upranking trusted sources of information; and fingerprinting and sharing of known AI-created content across platforms (similar to processes that already exist for fingerprinting non-consensual intimate images and child sexual abuse materials). In their efforts to avoid partisan debates about censorship and bias, several of the major platforms have also shifted their emphasis from the content of posts to account and behavioral signals, like detecting networks of accounts that amplify each other’s messages, large groups of accounts that are created at the same time, and hashtag flooding.
All of these methods may be helpful if lower cost, higher volume and more difficult detection are the hallmarks of generative AI in disinformation. They may also use risk assessments to determine where the potential harms are severe enough to warrant specific policies related to AI-generated content. (Elections and public health information are the most prevalent examples. When the stakes are that high, it may warrant prohibitions on certain uses of generative AI or manipulated media.) They could add information about AI-generated content (such as its prevalence, or the type moderated) to existing transparency reports. We would also favor policies that call for more accountability, including legal liability, for paid advertising. We don’t have the same concerns about over-moderation of commercial speech.
But all these methods carry the same limits and risks as they do for other forms of content. That includes the risk of over-moderation, which invariably has a particular impact on marginalized communities. As generative AI comes into broader use, users may actually be posting content that is beneficial and entertaining, making strict moderation policies by search and social media platforms undesirable as well as legally problematic. Even when strict policies and enforcement are warranted, their value depends on platforms’ willingness and ability to enforce them, including in languages other than English. Do we really want platforms to be the main line of defense against harmful narratives of disinformation given the platforms’ history, including on topics of enormous public importance like COVID-19 and elections?
AI Industry Self Regulation
Until or unless there are government regulations, the field of AI will be governed largely by the ethical frameworks, codes, and practices of its developers and users. (There are exceptions, such as when AI systems have outcomes that are discriminatory.) Virtually every AI developer has articulated their own principles for responsible AI development. These principles may encompass each stage of the product development process, from pretraining and training of data sets to setting boundaries for outputs, and incorporate principles like privacy and security, equity and inclusion, and transparency. They also articulate use policies that ostensibly govern what users can generate. For example, OpenAI’s usage policies “disallow” disinformation, as well as hateful, harassing, or violent content and coordinated inauthentic behavior, among other things.
But these policies, no matter how well-intentioned, have significant limits. For example, researchers recently found that the “guardrails” of both closed systems, like ChatGPT, and open-sourced systems, like Meta’s Llama 2 product, can be “coaxed” into generating biased, false, and violative responses. And, as in every other industry, voluntary standards and self-regulation are subject to daily trade-offs with growth and profit motives. This will be the case even when voluntary standards are agreed to collectively (as is a new industry-led body to develop safety standards) or secured by the White House (as is a new set of commitments announced last week). For the most part, we’re talking about the same companies – even some of the same people – whose voluntary standards have proven insufficient to safeguard our privacy, moderate content that threatens democracy, ensure equitable outcomes, and prohibit harassment and hate speech.
Regulatory Solutions
Any discussion of how to regulate disinformation in the United States – no matter how virulent, and no matter how it’s created – is bounded by the simple fact that most of it is constitutionally protected speech. Regardless, policymakers are actively exploring whether, or how, to regulate generative (and other) AI. New research shows public support for the federal government taking steps to restrict false information and extremely violent content online. In Public Knowledge’s view: Proceed with caution. While there may be room and precedent for content standards for the most destructive “lawful but awful” disinformation (such as networked disinformation that threatens national security and public health and safety), in general user speech is protected speech and free expression values are paramount.
One framework – which begins by comparing AI to nuclear weapons – is grounded in the idea of incremental regulation; that is, regulation that recognizes and accounts for a breadth of use cases and potential benefits as well as harms. It encourages us to focus on applications of the technology, not bans or restrictions on the technology itself. Every sector and use case comes with its own set of ethical dilemmas, technical complexities, stakeholders and policy challenges, and potential transformational benefits from AI. For example, in the case of disinformation, Public Knowledge advocates for solutions that address the harms associated with disinformation whether they originate with generative AI, Photoshop, troll farms, or your uncle Frank. The resulting policy solutions would encompass things like requirements for risk assessment frameworks and mitigation strategies; transparency on algorithmic decision-making and its outcomes; access to data for qualified researchers; guarantee of due process in content moderation; impact assessments that show how algorithmic systems perform against tests for bias; and enforcement of accountability for the platform’s business model (e.g., paid advertising).
We also need to account for the rapidity of innovation in this sector. One solution that Public Knowledge has favored is an expert and dedicated administrative agency for digital platforms. A dedicated agency should have the authority to conduct oversight and auditing of AI and other algorithmic decision making products in order to protect consumers and promote civic discourse and democracy. But such an agency should also have broader authorities, including to enhance competition and empower the public to choose platforms and services whose policies align with their values. Data privacy protections are also relevant here, as they would disallow the customization and targeting of content that can make disinformation narratives so potent and so polarizing. But let’s implement protections that cover all the data collection, exploitation, and surveillance uses we’ve discussed for so many years.
The Best Time To Act
To paraphrase an old expression, the best time to act to protect the integrity of our information environment is, well, in 2016; but the second-best time is now. There’s been a lot of freaking out about the heightened risks of disinformation due to generative AI as the United States and 49 other countries enter another election cycle for 2024. But generative AI is only one of the new threats in our information environment.
Virtually all of the major platforms have rolled back disinformation policies and protections before the 2024 election cycle. A U.S. District Court judge recently issued a ruling and preliminary injunction limiting contact between Biden administration officials and social media platforms over certain online content, even some content relating to national security and public health and safety. There is a powerful new counter-narrative in Congress and the judicial system about the government’s role in content moderation and an equation with censorship. Social media platforms, and media in general, seem to be fragmenting. This could be good or bad: Will the popularity of alternative, sometimes highly partisan, platforms send the conspiracy theorists back underground, made less dangerous because they are less able to find one another, connect, and communicate? Could more cohesive online communities with more in common increase the civility of these platforms? Or will the end of a few dominant digital gatekeepers mean even greater sequestering and polarization? And what happens if Twitter – or X – does implode like the Titan submersible, and its wonky, highly influential user base of journalists, politicians and experts disbands and can’t find one another to connect the dots on world events?
It will take a whole-of-society approach to restore trust in our information environment, and we need to accelerate solutions that have already been proposed. We favor solutions that equip civil society to identify false information and allow all Americans to make informed choices about what information they share. We should enable research into how disinformation is seeded and spread and how to counteract it. Policymakers should create incentives for the technology platforms to change their policies and product design and they should foster more competition and choice among media outlets. Civil society should convene stakeholders, including from the communities most impacted by misinformation, to research and design solutions – all while protecting privacy and freedom of expression. And we should use policy to solve the collapse of local news, since it has opened information voids that disinformation rushes in to fill.
Let’s not waste a crisis, even if it’s a false one. Let’s focus the explosion of attention on generative AI and its threats to democracy into productive solutions to the challenges and harms of disinformation we’ve been facing for years.