OpenAI Says It Blocked 250,000 Election Deepfake Requests

ChatGPT rejected more than 250,000 requests to generate DALL·E images of President-elect Trump, Vice President Harris, Vice President-elect Vance, President Biden, and Governor Walz in the month leading up to the U.S. Election Day, OpenAI disclosed on November 8, 2024.

The figure comes from an update to OpenAI's blog post on its 2024 worldwide elections approach, published after the U.S. vote. It offers the company's most concrete accounting yet of how its safety guardrails performed during the first U.S. presidential election held since ChatGPT's launch. The stakes for the industry were high: regulators, researchers, and campaigns spent 2024 watching generative AI for evidence that it could distort voting behavior at scale.

Directing users to authoritative sources

Throughout 2024, OpenAI worked with the National Association of Secretaries of State (NASS) to route voting-related questions in ChatGPT to CanIVote.org, the authoritative website on U.S. voting information. In the month before the election, roughly 1 million ChatGPT responses directed people to that site, OpenAI reported.

Starting on Election Day, people who asked ChatGPT for election results received responses encouraging them to consult news sources such as the Associated Press and Reuters, or their state or local election board. Around 2 million ChatGPT responses included this message on Election Day and the day after.

OpenAI also stated that it worked to ensure ChatGPT did not express political preferences or recommend candidates, even when asked explicitly.

Deepfake prevention

OpenAI applied safety measures to ChatGPT to refuse requests to generate images of real people, including politicians. The company described these guardrails as especially important in an elections context and a key part of broader efforts to prevent its tools being used for deceptive or harmful purposes.

The company applied the same principle abroad. Ahead of the 2024 European Parliament election, OpenAI introduced a ChatGPT experience directing users to the European Parliament's official voting information source, elections.europa.eu, when asked certain questions about the election process such as where to vote. That effort mirrored the NASS collaboration in the United States.

Disrupting influence operations

A central part of OpenAI's global elections work in 2024 involved identifying and disrupting attempts to use its tools to generate content for covert influence operations. In May 2024, the company began publicly sharing information on its disruptions, and it published additional reports in August and October.

OpenAI said its teams continued to monitor its services closely in the lead-up to Election Day and have not seen evidence of U.S. election-related influence operations attracting viral engagement or building sustained audiences through the use of its models.

Provenance and transparency tools

In a May 14, 2024 update, OpenAI said it had begun providing researchers with early access to a new tool that can help identify images created by DALL·E 3. The company also joined the Steering Committee of C2PA, the Coalition for Content Provenance and Authenticity, a widely used standard for digital content certification developed and adopted by software companies, camera manufacturers, and online platforms.

The company went further on policy. In May, OpenAI endorsed the "Protect Elections from Deceptive AI Act," a bipartisan bill proposed by Senators Klobuchar, Hawley, Coons, Collins, Ricketts, and Bennet in the U.S. Senate. The bill would ban the distribution of deceptive AI-generated audio, images, or video relating to federal candidates in political advertising, while including exceptions to protect First Amendment rights.

"We do not want our technology—or any AI technology—to be used to deceive voters and we believe this legislation represents an important step to addressing this challenge in the context of political advertising," OpenAI wrote.

Usage policies and organizational structure

OpenAI framed its election work around three pillars: elevating accurate voting information, enforcing measured policies, and improving transparency. A cross-functional effort brings together expertise from the company's safety systems, threat intelligence, legal, engineering, and policy teams to investigate and address potential abuse.

The company's Usage Policies for ChatGPT and the API include election-specific restrictions:

• OpenAI does not allow people to build applications for political campaigning and lobbying while it works to understand how effective its tools might be for personalized persuasion.
• Builders cannot create chatbots that pretend to be real people, such as candidates, or institutions, such as local government.
• Applications that deter participation in democratic processes are prohibited, including those misrepresenting voting processes and qualifications or discouraging voting, for example by claiming a vote is meaningless.
• With its custom GPTs, users can report potential violations to OpenAI.

The company said it red teams new systems before release, engages users and external partners for feedback, and builds safety mitigations to reduce potential harm. DALL·E has guardrails to decline requests for image generation of real people, including candidates.

Why it matters

2024 brought elections across the world's largest democracies, and OpenAI's disclosures represent a test case for how AI vendors police their own platforms during democratic contests. The company's own framing acknowledged the limits of unilateral action. "Protecting the integrity of elections requires collaboration from every corner of the democratic process, and we want to make sure our technology is not used in a way that could undermine this process," OpenAI wrote.

In an October 31 update, before the vote, the company said its teams were actively testing the safeguards put in place over the past year and monitoring for attempts to evade them, with protective measures adjusted as needed.

The company also acknowledged uncertainty. "Like any new technology, these tools come with benefits and challenges. They are also unprecedented, and we will keep evolving our approach as we learn more about how our tools are used," OpenAI wrote. That evolution will face its next tests as election cycles continue worldwide and as regulators weigh legislation like the Protect Elections from Deceptive AI Act, which OpenAI has already backed.