San Francisco’s city attorney says his office estimates Apple and Google have likely made millions of dollars in fees off apps that turn real people’s photos into fake nudity, and he wants both companies to pull 13 of them now.
David Chiu’s office sent cease-and-desist letters demanding Apple remove eight “nudify” apps and Google remove five, arguing the app stores are violating California’s law against supporting services that create deepfake pornography. The tools let users strip clothing, sexualize images, and swap real faces onto explicit bodies. One flagged app has more than a million downloads.
That law, updated last year as AB 621, presumes a platform is “knowingly facilitating” a deepfake pornography service if it keeps enabling one after being notified, and it caps damages at $50,000 per violation, $250,000 if a court finds malice.
Chiu told Wired his office is “absolutely horrified” by how common the technology has become, calling the images a tool “used to bully, humiliate, and threaten women and girls.” His office says allowing the apps to stay listed amounts to profiting off a public nuisance.
Google says it already suspended the five flagged apps under its sexual content policy, and that it has pulled hundreds of similar apps and blocked search terms like “nudify.” Apple hasn’t commented.
Targeting Apple and Google instead of individual operators hits the distribution layer every one of these apps depends on.
If your product lets users upload a photo and generates an altered image back, don’t treat content moderation as an afterthought. The next cease-and-desist letter names the platform, not just the app.
Nathan Zakhary