San Francisco City Attorney David Chiu has sent cease-and-desist letters to Apple and Google demanding that they remove 13 apps from their stores that can be used to make AI-generated nude images of people without consent, according to letters seen by WIRED.
The notices target eight apps in Apple’s App Store and five in Google Play. Chiu’s office says the apps are marketed broadly as face-swapping tools, while also allowing users to create explicit deepfake images after they start using them. That distinction matters because app store review systems can miss tools that look harmless at the front door and become abuse engines once installed.
Chiu told WIRED that making nonconsensual intimate images is illegal and harmful, and that Apple and Google have a duty to keep apps that enable sexual abuse off their platforms. His office argues that California law bars support for services that create deepfake pornography and says the companies should cut ties with the app developers, not just tidy up search results after researchers point at the mess.
The money trail is part of the city’s case. The letters say the apps use in-app payments, where Apple and Google take fees. Chiu’s office claims the companies have probably earned millions of dollars from apps offering nudification features.
How the apps work
The apps identified by the city are described as face-swapping software. In practice, WIRED reports, users can upload or select a person’s image and use AI tools to generate sexualized images or videos. Some services online produce results in seconds and charge small fees for additional output, while others offer limited free generation.
One app examined by the city had more than 1 million downloads and promoted AI image styles with sexualized descriptions, according to WIRED. Another claimed on its website that it could produce free, uncensored videos. WIRED did not publish the app names, citing the risk of driving users to them.
The broader category has become a recurring problem for app stores. Researchers and journalists have repeatedly found apps in Apple’s and Google’s stores that can generate explicit AI images, including some that were rated as suitable for children. Apple and Google both maintain developer rules against pornography, abuse and harassment, and both have removed deepfake or nudify apps after outside reports.
Google spokesperson Dan Jackson told WIRED that Google has removed hundreds of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office. Jackson said Google Play bans sexual content, investigates reported violations and has restricted search terms such as “nudify.” Apple did not comment before WIRED published its report.
Researchers keep finding the same failure
The Tech Transparency Project reported in January and April that it found around 100 apps across the App Store and Google Play, along with ads for nudifying tools. The group estimated those apps had about 480 million downloads combined and may have generated roughly $120 million in revenue. Jackson told WIRED that Google removed most of the apps identified by the group.
Katie Paul, the Tech Transparency Project’s director, told WIRED that the group did not expect to see the problem persist after its first report, yet found it again in similar or worse form. She said Apple and Google market their stores as safe and trusted, while the findings show a different result.
A May preprint paper by researchers at Cornell University and Georgetown University identified 420 general face-swapping apps in the two app stores. The researchers tested 155 of them for whether they could be used to put faces onto nude images. They found that 70 percent allowed it and lacked safeguards to stop that use. The researchers described such apps as dual-use tools that can pass platform moderation because they appear benign while retaining harmful capabilities.
Chiu told WIRED his office plans to keep pursuing the issue and wants Apple and Google to remove the apps and improve screening so similar tools do not reach users. If the companies do not act, he said the city will consider its legal options.
This story draws on original reporting from WIRED.
