A new AI Forensics report says Hugging Face deepfakes are being generated through image-editing models hosted on the platform, with little platform-level blocking of requests to create nonconsensual intimate images. The European nonprofit said seven of the nine most popular image-editing models it tested on Hugging Face complied with simple requests to remove clothing from images of women.
Hugging Face is a major repository for open-source and open-weight AI models, as well as “Spaces,” hosted demo apps that let users run models in a browser. That makes it useful infrastructure for researchers and developers. It also means weak controls can become a distribution problem, not a niche model problem.
How is Hugging Face being used for deepfakes?
AI Forensics said its researchers tested popular Hugging Face image-editing models with direct prompts asking the systems to alter images into topless versions while preserving pose and identity. The group said it did not use elaborate prompt workarounds or coded language to bypass safeguards.
That finding matters because mainstream AI products from companies such as Google and OpenAI commonly block requests to sexualize or undress real people. AI Forensics said the Hugging Face models it tested generally did not behave that way.
The nonprofit also set up honeypot image-editing Spaces on Hugging Face to observe what users would submit. AI Forensics said those Spaces were built so they would not actually produce requested images. Over seven days, the honeypots received more than 1,000 prompts and images.
According to AI Forensics, 73 percent of those submissions were sexual in nature. Of the sexual requests, 83 percent sought to undress a person in an image. The group said 95 percent of those undressing requests targeted women, and nearly 7 percent of sexual requests targeted children.
What do Hugging Face rules say?
Hugging Face’s content policy bars harmful or abusive content, including sexual content made without explicit consent and nudity involving minors. AI Forensics said it is not claiming Hugging Face created the models, but argued that the company can do more to filter activity on the platform it runs.
Paul Bouchaud, a lead researcher at AI Forensics, told Wired that most of the tested Spaces could be used to generate nonconsensual intimate images and that users were doing so. “No safeguards at all are being implemented at a platform level,” Bouchaud said, adding that safety measures are left to individual developers and that many do not add them.
The report recommends that Hugging Face add prompt-level filtering and output scanning across Spaces that generate images or video. In plain English: block requests that ask for sexualized edits before a model runs, then check the generated media before it reaches the user. That is not exotic AI safety magic. It is the basic moderation plumbing a platform needs when it hosts tools that can modify images of real people.
AI Forensics said those safeguards would help prevent sexualized editing requests and harmful outputs. The report does not say such measures would undo harms from images that may already have been created through insufficient protections.
This story draws on original reporting from The Verge.
