San Francisco Orders Meta to Stop AI-Generated Child Sexual Abuse Ads
In April 2024, San Francisco ordered Meta to stop AI-generated child sexual abuse ads.
Why it matters: Legal teams must understand how emerging AI content risks trigger enforcement actions and compliance challenges for social platforms.
- San Francisco City Attorney David Chiu issued a cease-and-desist ordering Meta to end AI-generated child sexual abuse ads on its platforms.
- Over 350 such ads appeared on Facebook and Instagram, reaching more than 29,000 accounts worldwide, including EU and U.S. users.
- Some ads used real children's images, prompting severe criticism; Meta says many ads had under 200 views and spent less than $5,000 total on advertising.
- Meta argues San Francisco lacks jurisdiction because ads were not shown in the city, raising questions about legal reach for online content enforcement.
San Francisco City Attorney David Chiu formally demanded in April 2024 that Meta stop AI-generated child sexual abuse material (CSAM) ads on Facebook and Instagram. The cease-and-desist letter accused Meta of allowing ads that violated its own policies banning child exploitation and nudity.
The Tech Transparency Project (TTP) documented 350+ ads that used AI to create explicit videos involving minors. These ads reached roughly 29,000 users across multiple countries, including the EU, U.S., Australia, and India.
TTP found some ads included images of real children, drawing particular concern. Meta stated many ads were removed quickly, with typical views under 200 and total ad spend less than $5,000. However, TTP observed ads persisted for days after reporting and new ads reused images of the same children.
Meta rejected claims that the company facilitated child exploitation, emphasizing its policies against such content. It also argued San Francisco City Attorney lacks legal jurisdiction since the ads did not appear within city limits, highlighting a key legal complexity for regulating digital content across regions.
This case illustrates how legal teams must prepare for evolving AI-related compliance risks as regulators increasingly pursue enforcement actions tied to automated harmful content. Understanding jurisdictional questions and ad monitoring remains critical for corporate counsel advising tech platforms.
By the numbers:
- 350+ AI-generated child sexual abuse ads documented — Tech Transparency Project
- 29,000+ accounts reached globally on Facebook and Instagram — TTP analysis
- Under $5,000 total ad spend for these ads — Meta statement
Yes, but: Meta argues San Francisco lacks jurisdiction because the ads were not accessed within the city, raising questions about legal reach in online enforcement.
What's next: Potential legal responses include Meta's formal reply to the cease-and-desist and possible further actions by other jurisdictions focusing on AI-generated harmful content.